Home > Wireless LAN > Other Topics > Troubleshooting Mesh Communication between Access Points

Troubleshooting Mesh Communication between Access Points

In a mesh wireless network, gateway access points are wired directly to the network, while repeater access points rely on wireless mesh links to get network connectivity. If a repeater AP goes down or has an unreliable connection, it will likely be unreachable from Dashboard. However, helpful troubleshooting information can be gathered both on-site and from nearby nodes in Dashboard.

This document outlines steps to troubleshoot repeater connectivity from the perspective of its online mesh neighbors.

The Mesh Networking and Mesh Deployment Guide discuss important concepts which should be taken into account before using this troubleshooting guide. Therefore, review those articles prior to using this guide. 

 

The following steps are recommended to troubleshoot problems with mesh connectivity:

  1. Determining Dashboard status
  2. Check for Mesh Neighbors
  3. Check Mesh Routes
  4. Check radio settings
  5. Physical inspection
  6. Packet captures

Determining Dashboard Status

During normal operation a repeater will appear as a green icon in Dashboard. In this state, a repeater will:

  • Advertise SSIDs configured in Dashboard
  • Allow clients to associate and pass traffic.
  • Update Dashboard with statistics and download its configuration.
  • Allow for full use of supported Live tools

Note: Dashboard mesh stats are not updated in real-time. There can be considerable delay between what is happening in real-time and what is reported in Dashboard. Furthermore, if a repeater cannot communicate with Dashboard, the information on its details page is likely stale. 

Normal Operation

A working mesh connection is shown below. Repeater “Roberta” has a 1-hop route to reach the “Rosehill Pole” gateway AP. In this instance, both AP's are mesh neighbors because they share a common channel (channel 44 highlighted in yellow) and are within range of each other and are located in the same Dashboard network.

 

Figure 1: Meshing APs operating on channel 44.

 

Figure 2: The repeater as listed in the mesh neighbors table of the gateway.

 

Figure 3: The repeater listing its 1-hop route to the gateway, showing the gateway in its mesh neighbors table. 

Connectivity Problems

There are three common Dashboard connectivity warning messages:

  • Unreachable
  • Unable to find a gateway
  • Repeater warning

Unreachable

When a repeater loses connection to the Internet and Dashboard, it will appear as a red icon with a status of “unreachable” on the AP details page:

 

Figure 4: An unreachable repeater.

Unable to Find a Gateway

A repeater can mesh through Meraki AP's located outside of its own network/organization if it is unable to find a mesh route to the Internet/Dashboard through AP's within its own Dashboard network. In this state, the AP will appear as a yellow icon with an “Unable to find a gateway to the Internet” status on the AP details page:

 

Figure 5: Repeater without a gateway.

 

Note: Repeaters that have a mesh route outside the network will not advertise SSIDs configured in Dashboard or forward client traffic. 

Repeater Warning

A repeater can get into a bad connectivity state where there is one-way communication between itself and the cloud. In this state, the cloud receives management traffic from the repeater, but when it replies, it does not receive acknowledgement from the repeater. In this state the repeater will alert "bad connectivity to the controller, possible firewall or NAT issues":

 

Checking Historical Status of a Repeater

The historical connectivity status of a repeater can be seen by toggling the Connectivity graph on the AP details page. Place the mouse pointer over the colored lines on the graph, to view the timeframe of mesh connectivity status changes.

 

Check for Mesh Neighbors

If two AP's operate on the same channel and are within range of each other, they can become mesh neighbors. Mesh neighbors exchange mesh route information allowing repeaters to discover the best metric route to the gateway. If two AP's are in the same Dashboard network and mesh neighbors, they may establish a mesh routing relationship where they send and receive client traffic of the mesh link.

This information can be immensely helpful in narrowing down bad mesh links and determining next steps.

Check the Mesh Neighbors table

AP's that are mesh neighbors will appear in each other’s mesh neighbors table (on the AP details page, under Wireless > Monitor > Access points > The repeater experiencing issues). When a repeater is offline, the first step is to confirm whether it shows up in the mesh neighbors table of other AP's it should route through.

 

Figure 6: A repeater "Repeater 1" is a mesh neighbor of the "Gateway" AP, as it appears in Roof's mesh neighbors table.

Check the Local Status Page

The Neighbors page of the Local Status Page shows all of the neighbors of a given Meraki AP. Access this page on an online AP and check the to see if the offline repeater MAC address appears in the table. If it does, the offline repeater is active and within range of the online AP.

Check Mesh Routes

Once a mesh neighbor relationship is confirmed between an offline repeater and its mesh neighbor, the next step is to confirm a mesh route exists. A repeater’s mesh route table shows the path traffic must take to reach a specific gateway. This path can include multiple hops, meaning there may be intermediate repeaters which traffic must pass through to reach the gateway. The route from the gateway back to the repeater might not follow the same path, Dashboard only shows the repeater’s mesh route table.

 

If a repeater is having mesh connectivity issues, check the mesh route table on that AP's details page (under Wireless > Monitor > Access points > The repeater experiencing issues) in Dashboard. For each route reported, there should be a mesh neighbor one hop away. Since the repeater will communicate directly with those APs to get online, it is important to ensure that those routes are healthy. After determining which AP is the repeater's first hop, confirm that the reported mesh neighbor is online, because it is not possible to gather Dashboard data from an offline node.

 

Repeaters have logic to determine which route to the gateway is best. It is calculated into a metric using a combination of hop count, data rates, RSSI, and loss rate (FWD and REV rates). A perfect metric has a score of 1179, and AP's can dynamically change to use a route with a better metric (if one is available).

Checking the Mesh Routes Table

Below is an example of a Dashboard mesh route table for repeater “Meraki Repeater”. 

 

Figure 7: The route table on Meraki Repeater can be used to show which network segments are closest. 

          

Check Radio Settings

After confirming a mesh route exists between an offline repeater and its mesh neighbor, the next step is to check for radio configurations or environmental factors which adversely affect mesh. These factors are:

  • Channel selection and transmit power
  • DFS
  • RF spectrum

Channel Selection and Transmit Power

For any two AP's to become mesh neighbors, they must be able to hear each other. This requires that AP's on each side of the link operate on at least one common channel and have enough transmit power for transmissions to be received by its neighbor. Simply put, the distance between nodes should not exceed the distance the radio signals can travel. You will need to confirm the channel selection and transmit power configured on each AP in a mesh link will meet this requirement.

 

The first section of this document provides an example of how to compare the channel information using the details page of both AP's. Additionally, the channel and transmit power configuration on the Radio settings page should be also be verified. 

 

Figure 8: Two APs, each AP is set to a different 2.4GHz channel and the transmit power is off. These devices will not mesh.

 

Figure 9: Two APs manually configured to use channel 36 with a transmit power of 5dBm. The transmit power level could be an issue if the nodes are too far apart, but if they are close enough they should become mesh neighbors.

 

Note: Dashboard will calculate the distance between neighboring APs when placed on Google map. The distance is based on the lat/long (from Dashboard) of a mesh repeater and the neighbors it sees.This distance is then reported in the Dashboard mesh neighbors table.

Dynamic Frequency Selection (DFS)

In certain regulatory domains, 5GHz WiFi channels are shared with radar. DFS regulation in these environments require that an AP change it's channel whenever it detects a radar pulse. 

 

Channel changes, including those due to DFS, will bring down a mesh link. Therefore if 5GHz is being used for mesh, non-DFS channels are preferred. However, this may not be an option depending on the country of operation (regulatory domain). Therefore, always check for DFS as a possible cause of mesh link instability.

 

The Channel planning section of the Radio settings page has the regulatory domain of the network, which is based on the selected Country, and an option to allow DFS channels.

 

Figure 10: A network that allows DFS channels. AP's in this network could be subject to automatic channel switching.

 

DFS channel change events are reported in the Dashboard Event log. If you suspect DFS could be causing the mesh link to drop, the Event log is a good place to check.

RF spectrum

If an AP has a dedicated Air Marshal radio, it can report real-time RF information in Dashbord under Wireless > Monitor > RF spectrum. Check this page to find any indications of high channel utilization. If there is high utilization on certain channels, a manual channel adjustment may be necessary.

Physical Inspection

If an offline repeater is not showing up as a mesh neighbor of nearby AP's, there may be a physical issue that requires on-site troubleshooting. In particular, it may be necessary to spot check mesh AP's for signs that something in the deployment has changed or was installed incorrectly.

Check the LED

Monitor the LED of the offline node and note the color and activity patterns. The MR installation guide has the LED code definitions.

  • If there are no LED, does the AP have a working power source?
  • If there are no LED, try resetting the offline unit. After a reset, if there is still no LED activity, verify the offline AP has a working power source and check the cabling and hardware for damage.
  • If the LED shows the offline node in a scanning state, it may be unable to find a path to the gateway AP. This could be attributed to antenna placement or orientation which should be checked on both ends of the link.

Check the Antenna (outdoor APs)

Outdoor MR units use an externally attached antenna. If installed incorrectly, this could be the cause of poor/unreliable mesh links.

  • Confirm the antenna type and radio band being used is correct on both ends of the link.
  • Confirm antennas are attached securely to the correct radios, i.e. the 5GHz antenna is connected to the 5GHz radio posts or that an antenna is installed at all.
  • Confirm antennas are oriented correctly based on their coverage patterns. Meraki antenna coverage patterns are in the datasheets for each antenna model. For example, if a directional antenna is being used, make sure it is aiming at its target with direct line of sight. If an omni antenna is used, verify it is at the correct elevation.

Packet Captures

When troubleshooting a mesh link, packet captures may be required in order to analyze traffic between AP's. These promiscuous-mode packet captures must be taken using external monitoring stations to effectively capture all nearby wireless traffic. 

A monitor should be placed near each AP, and configured to capture packets on the channels being used for the mesh link. Once the packet captures been gathered, they can be used to identify the issue, or provide helpful troubleshooting information for Cisco Meraki Support.

 

Figure 11: In this example, the gateway and repeater on are channel 1. Therefore, a monitor is placed near each AP and configured to capture on channel 1.

You must to post a comment.
Last modified
08:03, 20 Jul 2017

Tags

This page has no custom tags.

Classifications

This page has no classifications.

Article ID

ID: 4142

Contact Support

Most questions can be answered by reviewing our documentation, but if you need more help, Cisco Meraki Support is ready to work with you.

Open a Case