In a mesh wireless network, gateway access points are wired directly to the network, while repeater access points rely on wireless mesh links to get network connectivity. If a repeater AP goes down or has an unreliable connection, it will likely be unreachable from Dashboard. However, helpful troubleshooting information can be gathered both on-site and from nearby nodes in Dashboard.
This document outlines steps to troubleshoot repeater connectivity from the perspective of its online mesh neighbors.
The Mesh Networking and Mesh Deployment Guide discuss important concepts which should be taken into account before using this troubleshooting guide. Therefore, review those articles prior to using this guide.
The following steps are recommended to troubleshoot problems with mesh connectivity:
During normal operation a repeater will appear as a green icon in Dashboard. In this state, a repeater will:
Note: Dashboard mesh stats are not updated in real-time. There can be considerable delay between what is happening in real-time and what is reported in Dashboard. Furthermore, if a repeater cannot communicate with Dashboard, the information on its details page is likely stale.
A working mesh connection is shown below. Repeater “Roberta” has a 1-hop route to reach the “Rosehill Pole” gateway AP. In this instance, both AP's are mesh neighbors because they share a common channel (channel 11 circled in red) and are within range of each other and are located in the same Dashboard network.
Figure 1: Meshing APs operating on channel 11.
Figure 2: The repeater as listed in the mesh neighbors table of the gateway.
Figure 3: The repeater listing its 1-hop route to the gateway, showing the gateway in its mesh neighbors table.
There are three common Dashboard connectivity warning messages:
When a repeater loses connection to the Internet and Dashboard, it will appear as a red icon with a status of “unreachable” on the AP details page:
Figure 4: An unreachable repeater.
A repeater can mesh through Meraki AP's located outside of its own network/organization if it is unable to find a mesh route to the Internet/Dashboard through AP's within its own Dashboard network. In this state, the AP will appear as a yellow icon with an “Unable to find a gateway to the Internet” status on the AP details page:
Figure 5: Repeater without a gateway.
Note: Repeaters that have a mesh route outside the network will not advertise SSIDs configured in Dashboard or forward client traffic.
A repeater can get into a bad connectivity state where there is one-way communication between itself and the cloud. In this state, the cloud receives management traffic from the repeater, but when it replies, it does not receive acknowledgement from the repeater. In this state the repeater will alert "bad connectivity to the controller, possible firewall or NAT issues":
Figure 6: A repeater with limited Dashboard connectivity.
The historical connectivity status of a repeater can be seen by toggling the Connectivity graph on the AP details page. Place the mouse pointer over the colored lines on the graph, to view the timeframe of mesh connectivity status changes.
Figure 7: A problematic repeater's historical connectivity graph.
If two AP's operate on the same channel and are within range of each other, they can become mesh neighbors. Mesh neighbors exchange mesh route information allowing repeaters to discover the best metric route to the gateway. If two AP's are in the same Dashboard network and mesh neighbors, they may establish a mesh routing relationship where they send and receive client traffic of the mesh link.
This information can be immensely helpful in narrowing down bad mesh links and determining next steps.
AP's that are mesh neighbors will appear in each other’s mesh neighbors table (on the AP details page, under Wireless > Monitor > Access points > The repeater experiencing issues). When a repeater is offline, the first step is to confirm whether it shows up in the mesh neighbors table of other AP's it should route through.
Figure 8: A repeater "Cedar" is a mesh neighbor of the "Roof" AP, as it appears in Roof's mesh neighbors table.
The Neighbors page of the Local Status Page shows all of the neighbors of a given Meraki AP. Access this page on an online AP and check the to see if the offline repeater MAC address appears in the table. If it does, the offline repeater is active and within range of the online AP.
Once a mesh neighbor relationship is confirmed between an offline repeater and its mesh neighbor, the next step is to confirm a mesh route exists. A repeater’s mesh route table shows the path traffic must take to reach a specific gateway. This path can include multiple hops, meaning there may be intermediate repeaters which traffic must pass through to reach the gateway. The route from the gateway back to the repeater might not follow the same path, Dashboard only shows the repeater’s mesh route table.
If a repeater is having mesh connectivity issues, check the mesh route table on that AP's details page (under Wireless > Monitor > Access points > The repeater experiencing issues) in Dashboard. For each route reported, there should be a mesh neighbor one hop away. Since the repeater will communicate directly with those APs to get online, it is important to ensure that those routes are healthy. After determining which AP is the repeater's first hop, confirm that the reported mesh neighbor is online, because it is not possible to gather Dashboard data from an offline node.
Repeaters have logic to determine which route to the gateway is best. It is calculated into a metric using a combination of hop count, data rates, RSSI, and loss rate (FWD and REV rates). A perfect metric has a score of 1179, and AP's can dynamically change to use a route with a better metric (if one is available).
Below is an example of a Dashboard mesh route table for repeater “Cedar”. This repeater has two possible routes to gateway “Roof” one which goes through mesh neighbor “pole” or a direct route to the gateway. If “Cedar” is offline you would start troubleshooting the from the closest network segments by gathering data from "pole" and "Roof".
Figure 9: The route table on Cedar can be used to show which network segments are closest.
After confirming a mesh route exists between an offline repeater and its mesh neighbor, the next step is to check for radio configurations or environmental factors which adversely affect mesh. These factors are:
For any two AP's to become mesh neighbors, they must be able to hear each other. This requires that AP's on each side of the link operate on at least one common channel and have enough transmit power for transmissions to be received by its neighbor. Simply put, the distance between nodes should not exceed the distance the radio signals can travel. You will need to confirm the channel selection and transmit power configured on each AP in a mesh link will meet this requirement.
The first section of this document provides an example of how to compare the channel information using the details page of both AP's. Additionally, the channel and transmit power configuration on the Radio settings page should be also be verified.
Figure 10: Two APs, each AP is set to a different 2.4GHz channel and the transmit power is off. These devices will not mesh.
Figure 11: Two APs manually configured to use channel 36 with a transmit power of 5dBm. The transmit power level could be an issue if the nodes are too far apart, but if they are close enough they should become mesh neighbors.
Note: Dashboard will calculate the distance between neighboring APs when placed on Google map. The distance is based on the lat/long (from Dashboard) of a mesh repeater and the neighbors it sees.This distance is then reported in the Dashboard mesh neighbors table.
In certain regulatory domains, 5GHz WiFi channels are shared with radar. DFS regulation in these environments require that an AP change it's channel whenever it detects a radar pulse.
Channel changes, including those due to DFS, will bring down a mesh link. Therefore if 5GHz is being used for mesh, non-DFS channels are preferred. However, this may not be an option depending on the country of operation (regulatory domain). Therefore, always check for DFS as a possible cause of mesh link instability.
The Channel planning section of the Radio settings page has the regulatory domain of the network, which is based on the selected Country, and an option to allow DFS channels.
Figure 12: A network that allows DFS channels. AP's in this network could be subject to automatic channel switching.
DFS channel change events are reported in the Dashboard Event log. If you suspect DFS could be causing the mesh link to drop, the Event log is a good place to check.
If an AP has a dedicated Air Marshal radio, it can report real-time RF information in Dashbord under Wireless > Monitor > RF spectrum. Check this page to find any indications of high channel utilization. If there is high utilization on certain channels, a manual channel adjustment may be necessary.
If an offline repeater is not showing up as a mesh neighbor of nearby AP's, there may be a physical issue that requires on-site troubleshooting. In particular, it may be necessary to spot check mesh AP's for signs that something in the deployment has changed or was installed incorrectly.
Monitor the LED of the offline node and note the color and activity patterns. The MR installation guide has the LED code definitions.
Outdoor MR units use an externally attached antenna. If installed incorrectly, this could be the cause of poor/unreliable mesh links.
When troubleshooting a mesh link, packet captures may be required in order to analyze traffic between AP's. These promiscuous-mode packet captures must be taken using external monitoring stations to effectively capture all nearby wireless traffic.
A monitor should be placed near each AP, and configured to capture packets on the channels being used for the mesh link. Once the packet captures been gathered, they can be used to identify the issue, or provide helpful troubleshooting information for Cisco Meraki Support.
Figure 13: In this example, the gateway and repeater on are channel 1. Therefore, a monitor is placed near each AP and configured to capture on channel 1.