Cisco Meraki VPN peers can use Automatic NAT Traversal to establish a secure IPsec tunnel through a firewall or NAT. When ACLs on an upstream firewall block source ports or more likely the case destination UDP ports in the range 32768-61000 on outbound traffic, a peer will not be able to punch a hole in the firewall and establish a tunnel with other remote peers.
In the example below, the firewall blocks peer1 from sending outbound UDP packets in the necessary destination port range. This prevents a hole punch. Now when peer2 tries to send inbound packets, its packets are dropped on the outside interface of the firewall because they do not match an existing outbound session. In this instance the tunnel will not be established between peer1 and peer2.
Failed connectivity tests or a VPN status of 'disconnected' indicates a tunnel failure between peers in Dashboard.
In a site-to-site VPN if two peers are unable to establish a VPN connection they will show as disconnected on each other's VPN status page. In this case, a packet capture should be taken on the primary Internet interface of both peers to analyze which firewall is blocking IPsec communication.
MR to MX Concentrator Testing
With an MR to MX concentrator connection type, use the Test connectivity button on the MR network. Running the test will report which MRs "failed to connect to the concentrator." Please note if the issue is on the concentrator side, it is likely that all MRs will fail the test. A packet capture should be taken on the wired interface of each MR that failed to connect to the concentrator. Another capture should be taken from the primary Internet interface of the MX. These captures can be analyzed to determine which site's firewall is blocking outbound IPsec communication.
Analyzing a Packet Capture for IPsec Connectivity
Packet captures can be taken from Dashboard and downloaded as a .pcap file for analysis and filtering using Wireshark packet analyzer. They are invaluable for troubleshooting connections between hosts and isolating connectivity issues.
In the example below there is an MR to VPN concentrator tunnel that will not establish. We take packet captures from different points in the path to help determine which firewall is blocking the peer-to-peer communication.
The first capture, shown below, was taken from the wired interface of MR 10.0.8.99. We can see the MR attempting to punch a hole in its local upstream firewall by sending packets to 18.104.22.168, which is the outside IP address of the NAT that the VPN concentrator sits behind. Notice the the MR is sending traffic to the concentrator but there is no return traffic in the capture from the MX appliance behind the NAT.
MR 10.0.8.99:45540 -> MX 22.214.171.124:53654
A second capture, shown below, was taken from the inside interface of the MX firewall upstream from the VPN concentrator. The VPN concentrator uses IP address 10.0.50.246 on the LAN. We can see the VPN concentrator sending packets to 126.96.36.199 which is the outside IP address of the NAT that the MR sits behind in an attempt to punch a hole in its local upstream firewall. Notice the VPN concentrator is sending traffic to the MR but no return traffic is present from the MR behind the NAT.
MX 10.0.50.246:53654 -> MR 188.8.131.52:45540
A third capture was then taken, this time from the outside interface of the MX firewall upstream from the VPN concentrator. We can the see the VPN concentrator's traffic has been translated to 184.108.40.206, which is the firewall's outside IP address, and that it is being forwarded onto the Internet. This indicates the firewall is not blocking outbound IPsec traffic in the VPN concentrator site. However, we do not see any traffic originating from 220.127.116.11, the IP address of the NAT device the MR sits behind. From this we can conclude that the firewall upstream from the MR is blocking outbound IPsec traffic within the UDP port range 32768-61000.
MX 18.104.22.168:53654 -> MR 22.214.171.124:45540
To confirm, we take a final capture from the outside interface of the MX firewall upstream from the MR, shown below. This capture shows packets originating from the VPN concentrator at 126.96.36.199 and arriving at the MR firewall's outside interface at 188.8.131.52. We still do not see any traffic originating from the MR being sent from the outside interface. This indicates the MX firewall is in fact blocking outbound IPsec traffic on the inside interface, specifically destination UDP port range 32768-61000.
MX 184.108.40.206:53654 -> MR 220.127.116.11:45540
Once we reconfigure the firewall upstream from the MR to allow outbound destination port range 32768-61000, peers are able to form a tunnel. Although the first 4 captures are filtered by UDP ports 53654 and 45540, once the firewall is opened two-way traffic can occur on any dynamically chosen ports as shown below on a packet capture taken from the wired interface of the MR. Now the MR is registered with and using with port 41091 for VPN communication.
MR 10.0.8.99:41091 -> MX 18.104.22.168:53654
MR 10.0.8.99:41091 <- MX 22.214.171.124:53654
Below are two examples of ACLs that could be used to allow peer-to-peer communication between Cisco Meraki VPN peers. For the second option, X.X.X.X/32 represents the IP address of the Cisco Meraki device.
allow inside to outside, protocol: udp, source ip: any, src port: any, dst ip: any, dst port: 32768-61000 allow outside to inside established (may not be necessary with stateful firewalls)
allow inside to outside, protocol: udp, source ip: X.X.X.X/32, src port: 32768-61000, dst ip: any, dst port: 32768-61000 allow outside to inside established (may not be necessary with stateful firewalls)
Cisco Meraki VPN peers must be able to use high number UDP ports to communicate with each other. Security systems such as firewalls that disallow this traffic may prevent successful traffic flow over the VPN. Please follow the diagnostic and troubleshooting steps above to resolve such issues.