Auto VPN Static Routing Principles and Scaling Techniques
Overview
This article explains Auto VPN static routing principles and details Meraki enhancements for scaling and controlling static Auto VPN routing. It also describes the impact of scaling techniques on network environments. All scaling features described can be enabled or disabled only through Meraki Support.
The dashboard screenshot below shows an example of static Auto VPN routes. The VPN mode parameter determines whether Dashboard propagates the route to other MX devices in the organization. Enabled means Dashboard propagates the static route to the network.
Before requesting changes to the existing network, it is recommended to discuss any modifications with a Cisco Solutions Engineer (SE).
Components
-
Hardware requirements: Any Meraki MX hardware or virtual appliance
-
Licensing requirements: Any Meraki MX license level
-
Software/Firmware requirements: Any supported Meraki MX software version
The topology shown below is used to explain the concepts in this article. All subsequent examples and outputs use the devices in this topology.
Technical architecture and terminology
Meraki Auto VPN fabric
Auto VPN is a proprietary technology developed by Meraki that enables the creation of VPN tunnels between Meraki WAN appliances at different network branches through a simple, cloud-based process. Auto VPN automates the tasks typically required for manual VPN configuration.
Auto VPN fabric static routing principles
MX roles in Auto VPN fabric
- Hub - in Auto VPN, hubs are central points to which remote sites (spokes) establish connections (tunnels). Traffic between remote sites is routed through the hubs. Hubs also implicitly establish connections (tunnels) to all other hubs in the organization, while spokes establish connections only to the hubs specified in their settings.
- Spoke - in Auto VPN, spokes are sites that connect exclusively to designated hubs.
From the perspective of a given hub, all other MX devices in the Auto VPN fabric can be classified as follows:
- Adjacent hubs - Hubs that share the same spokes with the given hub.
- Non-adjacent hubs - Hubs that do not share any spokes with the given hub.
- Attached spokes - Spokes directly connected to the given hub.
- Non-attached spokes - Spokes connected to non-adjacent hubs.
From the perspective of a given spoke, all other MX devices in the Auto VPN fabric can be classified as follows:
- Attached hubs - Hubs to which the given spoke is connected.
- Non-attached hubs - Hubs to which the given spoke is not connected.
- Adjacent spokes - Spokes that share the same hub or hubs with the given spoke.
- Non-adjacent spokes - Spokes connected to different hubs.
The following topology diagram shows the role assignment in the test topology.
Auto VPN Static route propagation
The MX appliance receives all possible routes to networks enabled for Auto VPN within the organization as part of the configuration file distributed by the dashboard.
Based on the MX role in Auto VPN, Dashboard applies the following rules.
For spokes - Dashboard propagates all networks with VPN mode enabled in the organization. This includes networks behind:
- Attached Hubs
- Non-attached Hubs
- Adjacent Spokes
- Non-adjacent Spokes
The figure below shows examples of all mentioned network types from the perspective of the spoke in the SITE-8 network.
For hubs - Dashboard propagates all networks with VPN mode enabled in the organization. This includes networks behind:
- Adjacent Hubs
- Non-adjacent Hubs
- Attached Spokes
- Non-attached Spokes
The figure below shows examples of all mentioned network types from the perspective of the hub in the Auto VPN DC-A-HA-Pair network.
Auto VPN static route parameters
Like other routing mechanisms, Auto VPN static routing provides two main services - best path selection and traffic failover during an outage on the best path. During route propagation, Dashboard uses two parameters that hubs and spokes later use for best path selection and route failover.
1. Metric - Dashboard sets the metric value and MX appliances use it to select the best route when multiple paths are available for the same network address and prefix.
2. Peer-tracking - Dashboard can set peer tracking to true or false. When peer tracking is enabled, MX appliances monitor the reachability of the remote tunnel endpoint. If the remote MX becomes unreachable, the local MX marks all routes attached to that tunnel as unreachable. Routing can then switch to the route with the next best metric.
- "true" - Dashboard sets peer tracking to true when more than one path is available for a given network. For example, a spoke connected to multiple hubs has multiple paths to networks behind other spokes in the organization. Each hub provides a separate path. In this case, Dashboard sets peer tracking to true for all routes except the route with the highest metric, which is the least preferred route.
- "false" - Dashboard sets peer tracking to false when no alternative path is available or when the path is the least preferred.
Dashboard defines both the metric and peer tracking attributes. Administrators have no direct control over them. However, certain Dashboard settings, such as the order and number of hubs in the spoke settings, directly affect these parameters. Understanding how Dashboard assigns these parameters is essential for design, operation, and troubleshooting.
Configuration Used for the Test Topology
Auto VPN-SITE-10 –
Auto VPN-SITE-3 –
Auto VPN-SITE-4 –
Auto VPN-SITE-8 –
Best path selection and failover
Best path selection and failover on Spoke
On the spoke side, the metric is determined by the hub position in the Hubs section of the spoke’s Security & SD-WAN > Site-to-site VPN page – routes through the top hub have metric 0 (lower is better), and the metric increases by one for each subsequent position.
Explicit metric values are not visible in the dashboard and can only be viewed by Meraki Support.
The peer-tracking value "true" is assigned to routes through all hubs except the last one in the hub list when the network is available behind all of them. The route through the last hub in the list receives the peer-tracking value "false."
The example below demonstrates a topology and associated network settings:
Topology –
Settings –
Best path selection and failover for networks behind other spokes and non-attached hubs
If more than one hub is defined in the spoke's network settings (Security & SD-WAN > Site-to-site VPN), routes to networks behind adjacent spokes, non-adjacent spokes, and non-attached hubs point to all configured attached hubs.
Other spokes refer to adjacent and non-adjacent spokes. The same rules apply to both.
The following are examples for all three scenarios from the perspective of the Auto VPN SITE-4 spoke.
Topology with networks behind adjacent spoke highlighted –
Route to the network behind adjacent spoke –
Topology with Non-adjacent spoke highlighted –
Route to the network behind the Non-adjacent spoke –
Topology with Non-attached hub highlighted –
Route to the network behind Non-attached hub –
In all three examples, redundant routes point to both hubs. While the metric is not visible in the dashboard, a green circle in the status column indicates the preferred route. All alternative routes are marked with a gray circle.
Proper redundancy requires connections to multiple hubs at both ends – the example below demonstrates partial redundancy, as from the perspective of SITE-4, the route to the network 10.3.0.0/24 is redundant because SITE-4 is connected to two hubs, while the routing path in the opposite direction is not redundant.
The figure below shows that the SITE-3 MX has a single route to 10.4.0.0/24 in its routing table.
The routing table output below confirms that the SITE-4 MX has two routes to 10.3.0.0/24.
Best path selection and failover for networks behind Attached hubs
In the case of static Auto VPN routes towards networks behind attached hubs, redundancy is present only when the same network is "advertised" (added/enabled for Auto VPN) by multiple hubs – the example below demonstrates this scenario.
Configuration on Auto VPN DC-A-HA-Pair –
Configuration on Auto VPN DC-A-Standalone –
Both hubs advertise 10.253.0.0/16. SITE-4 has path redundancy to this network, as shown in the figure below.
At the same time, only a single hub advertises 10.254.0.0/16 and 10.254.1.0/24. No path redundancy is available for those networks, as shown in the figure.
Best path selection and failover for overlapping networks behind Attached hubs
If the design includes data center hubs that advertise overlapping subnets for the same group of resources – for example, 10.253.0.0/16 and then 10.254.0.0/16 – routes to networks 10.254.0.0/16 and 10.254.1.0/24 have peer-tracking set to "false" as no redundant path is available. Although the subnets overlap, the network addresses are not the same. The scenario below demonstrates MX behavior for such designs.
In this setup, packets originating from Auto VPN SITE-3 to Server-A traverse the tunnel established to Auto-VPN DC-A-Standalone – this occurs due to the longest prefix match.
At the same time, return traffic on the data center side is typically routed to the primary VPN hub – in this example, Auto VPN DC-A-HA. This results in asymmetric routing.
For Server-B, the path through Auto VPN DC-A-HA-Pair is used in both directions.
An Auto VPN DC-A-Standalone hub outage in this setup prevents access to Server-A, even though both hubs can theoretically deliver packets to Server-A and Server-B – this is a result of the design, which involves advertising overlapping networks on different hubs. The decision to use overlapping networks prevents MX devices from setting peer-tracking to "true" for the routes.
Even though the path through the Standalone hub is unavailable, MX has no alternative route for subnet 10.254.1.0/24 – as a result, this route remains marked as preferred in the spoke's routing table.
Best path selection and failover on Hubs
Best path selection and failover for networks behind Attached spokes
For networks located behind attached spokes, hubs always have a metric of 0 and peer-tracking set to "false" – as there is no path redundancy. This applies to all hubs configured on the spoke side, regardless of their position in the list of hubs. Hub-to-hub tunnels are never used as a backup path for networks behind attached spokes.
The exception to this rule is a routed-mode hub configured for eBGP peering on its LAN. If the design includes such a hub, all spoke routes will have peer-tracking set to "true" – this is because some or all spoke networks may be available behind the eBGP peer, for example, when a backup path to spoke networks is available over MPLS or another MX hub in a different data center. In this case, avoid using a default route on the MX hub LAN as this can potentially result in a short-lived routing loop if the path to the spoke is no longer available but the route is still included in the MX hub eBGP advertisement.
The example below demonstrates the topology and routing table outputs on Auto VPN DC-A-HA-Pair and Auto VPN DC-A-Standalone.
Route table on Auto VPN DC-A-HA
Route table on Auto VPN DC-A-Standalone
The impact of the absence of spoke route tracking on hubs can be observed by analyzing the packet flow in a design where spokes have different primary hubs in their settings.
Example topology
Security & SD-WAN > Site-to-site VPN page of SITE-3
Security & SD-WAN > Site-to-site VPN page of SITE-4
With this configuration, an endpoint at SITE-3 sending packets to an endpoint at SITE-4 will have the request follow the red path – Auto VPN SITE-3 > Auto VPN DC-A-HA-Pair> Auto VPN SITE-4. The reply will follow the orange path – Auto VPN SITE-4 > Auto VPN DC-A-Standalone > Auto VPN SITE-3. This results in asymmetric routing in the Auto VPN environment.
Request path -
Response path -
While such a design creates an asymmetric routing flow under normal conditions, MX SD-WAN handles it without issue. To avoid path unpredictability, it is recommended to use the same order of hubs on adjacent spokes.
Best path selection and failover for networks behind Non-Attached spokes
To select the best route toward non-attached spokes, hubs use the spoke configuration. The priority of attached hubs defined on the spoke is also used by non-attached hubs. The figure below shows an example topology and spoke configuration. In this example, the DC-B hub must select the best path toward networks behind the SITE-3 and SITE-4 spokes. DC-B is a non-attached hub, so it uses the same metrics that Dashboard assigns to the attached hubs on the spoke side.
The figure below shows the DC-B routing table filtered by one subnet from SITE-3. The route through DC-A-HA-Pair MX is installed as primary. The route through DC-A-Standalone MX is used as secondary.
The figure below shows the DC-B routing table filtered by one subnet from SITE-4. The route through DC-A-Standalone MX is installed as primary. The route through DC-A-HA-Pair MX is used as secondary.
In this scenario, a change in the hub priority order on the spoke affects best path selection on non-attached hubs.
For failover, when a spoke has more than one hub selected in its settings, non-attached hubs have alternative paths. This provides failover during an outage.
Best path selection and failover for networks behind other hubs
This category of routes includes networks located behind adjacent and non-adjacent hubs. Both hub types are grouped together, as the dashboard applies the same rules during configuration generation.
This scenario can be divided into two categories:
- Metric and failover for networks behind other hubs in VPN Concentrator mode.
- Metric and failover for networks behind other hubs in Routed mode.
Each use case is detailed below.
Best path selection and failover for networks behind other hubs in VPN Concentrator mode
The priority of networks available behind hubs in concentrator mode is determined by the hub's position in the concentrator priority list – higher positions have higher priority. This list is available on the Security & SD-WAN > Site-to-site VPN page. This includes networks located directly behind concentrators and those behind attached spokes.
The screenshot below shows the concentrator priority order for the demo organization used in this document.
The following example shows a network located behind non-adjacent hubs. Network 10.253.0.0/16 is added to Auto VPN on both Auto VPN DC-A-HA-Pair and Auto VPN DC-A-Standalone hubs.
From the perspective of the Auto VPN DC-C and Auto VPN DC-B hubs, the route through Auto VPN DC-A-Standalone has a metric of zero. The route through Auto VPN DC-A-HA-Pair has a metric of one.
This can be confirmed by reviewing the routing table on the Auto VPN DC-C hub for network 10.253.0.0/16.
The following example shows network 10.10.255.0/24 located behind the spoke Auto VPN SITE-10, which is connected to DC-B and DC-C hubs.
From the perspective of the DC A hubs, the route through Auto VPN DC-C has a metric of zero. The route through Auto VPN DC-B has a metric of one.
This can be confirmed by reviewing the routing table on the Auto VPN DC-HA hub for network 10.10.255.0/24.
Best path selection and failover for networks behind other hubs in Routed mode
On the spoke side, the metric is determined by the hub order in the Hubs list on the Site-to-site VPN page.
For other hubs, the concentrator priority cannot be used to select the best route, as this list includes only MXs in concentrator mode.
Because the concentrator priority list is not available, the dashboard assigns a random persistent metric to routed mode hubs to ensure traffic path stability. This means the metric assigned to routed mode hubs remains persistent but its initial assignment is 'random'. Administrators cannot predict the exact metric value, as it's determined by the dashboard's internal processing order during configuration generation, rather than by hub-specific parameters like IP or MAC addresses.
The example below illustrates these details.
MX 26.2+: Concentrator priority is renamed to hub to hub priority, as it now also applies to routed mode MXs configured as hubs.
In this example, SITE-10, SITE-3, and SITE-4 MX hubs are configured to advertise the same subnet 10.255.0.0/24 into Auto VPN.
From the perspective of SITE-8, SITE-3 is selected as the primary path.
By removing this network from SITE-3, the administrator can verify the metrics for this network on the MXs at SITE-4 and SITE-10.
If the route to 10.255.0.0/24 is restored on SITE-3, SITE-3 MX will again be listed as the primary path. This demonstrates the persistent nature of random metric assignment.
The metric values for all three hubs are as follows – SITE-3 MX: metric 0, SITE-10 MX: metric 1, SITE-4 MX: metric 2.
Auto VPN static routing scaling best practices
Subsequent sections describe scaling techniques and best practices that can be applied to Auto VPN static routing to increase scalability and modify some default behaviors outlined above.
No Auto VPN Hub-to-Hub tunnels
By default, all hubs in the organization form full-mesh tunnels with each other. This results in all hubs having routes to subnets located behind other hubs and to all non-attached spokes. In some cases, this behavior may be undesirable. Two common examples are described below.
- Regional data centers with implemented service redundancy that have spokes inside the region connected to two or more regional hubs. Each regional hub provides access to all required services within the region. Spoke-to-spoke communication between regions is either unnecessary or handled through data center interconnect.
- An organization with regional hubs and many routes may use hub and spoke resources to maintain routing information about networks in other regions. If spoke-to-spoke routing between regions is unnecessary or implemented over a data center interconnect, hub-to-hub tunnels provide no benefit and only lead to resource constraints.
How this feature works
The dashboard disables tunnels to any other hub on an MX appliance configured as a hub.
No Auto VPN Hub-to-Hub Tunnels feature summary
- This feature is disabled by default.
- Customers can request Meraki Support to enable this feature.
- Meraki Support may recommend enabling this feature when the number of routes to resources in other regions could affect MX stability.
- This feature is organization-wide – it disables hub-to-hub tunnels for all hubs in the organization.
- After this feature is enabled, spoke-to-spoke routing is not possible between non-attached spokes – alternative paths such as data center interconnect should be used for this purpose.
No Auto VPN spoke-to-spoke routes
By default, every spoke receives routes to all other networks in the organization that are added to Auto VPN. While the "No Auto VPN Hub-to-Hub tunnels" feature removes routes to non-adjacent spokes and non-attached hubs, routes to all networks behind adjacent spokes remain. In some cases, this behavior is not desirable.
Examples include:
- Spoke-to-spoke communication must be disallowed entirely, which is common for franchising organizations or organizations with high-security standards.
- The organization has a significant number of spokes and subnets, which may result in performance issues, especially on low-end MX platforms.
How this feature works
The dashboard does not propagate routes to spoke networks behind adjacent or non-adjacent spokes.
No spoke-to-spoke routes feature summary
- This feature is disabled by default.
- Customers can request Meraki Support to enable this feature.
- Meraki Support may recommend enabling this feature when a large number of routes could affect MX stability.
- This feature is organization-wide – it removes routes to subnets located behind adjacent spokes throughout the organization.
- Summary routes must be added to the hubs after enabling this feature if spoke-to-spoke communication needs to be retained. This is typically required when the feature is used to minimize routing table sizes on the spokes.
The example below demonstrates how to use summary routes on hubs to maintain spoke-to-spoke connectivity when No spoke-to-spoke routes feature is used.
Auto VPN Track Hub originated routes
As described in the section "Failover for overlapping networks behind Attached hubs", routes added to Auto VPN by hubs are not tracked when path redundancy is absent. This occurs when overlapping routes are added on multiple hubs.
In some cases, this behavior is undesirable. For example, a different hub may need to be preferred for specific destinations while maintaining a primary hub for all other traffic.
The example below shows proxy server subnet traffic (10.254.1.0/24) explicitly routed through the DC-A-Standalone hub. This approach helps distribute load between Auto VPN hubs.
When using this setup, routing in both directions should pass through the same hub to avoid asymmetric routing. In the spoke-to-DC direction, this is achieved by adding different routes to the hubs. Hub DC-A HA-Pair advertises 10.254.0.0/16, while Hub DC-A-Standalone advertises the more specific subnet 10.254.1.0/24.
In the DC-to-spoke direction, the administrator must use policy-based routing to set Hub DC-A-Standalone as the next hop for traffic sourced from 10.254.1.0/24. If a Cisco router is used in the data center, the route map must include "ip next-hop verify-availability" to ensure proper failover if Hub DC-A Standalone is unavailable.
Example topology -
The configuration example for a Cisco router suitable for this setup is explained below.
Two IP SLA elements and corresponding track objects are created to monitor the IP of DC-A-HA-Pair (172.16.251.21) and the IP of DC-A-Standalone (172.16.251.20).
|
# IP SLA object which pings an IP of DC-A-HA-Pair Hub ip sla 10 icmp-echo 172.16.251.21 source-ip 172.16.251.17 threshold 1000 timeout 1000 frequency 5
# Schedule for DC-A-HA-Pair Hub IP SLA object ip sla schedule 10 life forever start-time now
# IP SLA object which pings an IP of DC-A-Standalone Hub ip sla 20 icmp-echo 172.16.251.20 source-ip 172.16.251.17 threshold 1000 timeout 1000 frequency 5
# Schedule for DC-A-Standalone Hub IP SLA object ip sla schedule 20 life forever start-time now
# Track object to monitor reachability of IP SLA object which belongs to DC-A-HA-Pair Hub track 10 ip sla 10 reachability
# Track object to monitor reachability of IP SLA object which belongs to DC-A-Standalone Hub track 20 ip sla 20 reachability |
Static routes are configured toward the spoke subnets. The route to the DC-A-HA-Pair hub (172.16.251.21) has a tracking element assigned, so it is removed from the routing table when the corresponding track object reports reachability as down. In this case, traffic is rerouted to the DC-A-Standalone hub (172.16.251.20), as this route has a higher administrative distance.
|
# Route towards DC-A-HA-Pair Hub with the tracking object ip route 10.0.0.0 255.0.0.0 172.16.251.21 track 10
# Route towards DC-A-Standalone Hub with a higher administrative distance ip route 10.0.0.0 255.0.0.0 172.16.251.20 2 |
An access list and corresponding route map are used to send traffic originating from 10.254.1.0/24 (proxy servers subnet) with destinations in the spoke subnets (10.0.0.0/8) toward the DC-A-Standalone hub. The next hop has a tracking element assigned, which allows traffic to be rerouted to the DC-A-HA-Pair hub if the Standalone hub becomes unavailable.
|
# ACL to match traffic originating from proxy servers' subnet ip access-list extended Proxy-Servers 20 permit ip host 10.254.1.3 10.0.0.0 0.255.255.255
# Route map that routes the traffic matching Proxy-Servers ACL to DC-A-Standalone Hub when it is available route-map Prox-Servers permit 10 match ip address Proxy-Servers set ip next-hop verify-availability 172.16.251.20 10 track 20 |
How this feature works
On the spoke side, the dashboard enables route tracking for all routes where the hub is the next hop, regardless of redundant path presence. Track hub-originated routes should always be used in combination with the No Auto VPN Hub-to-Hub tunnels feature to prevent scenarios where traffic can be blackholed in hub-to-hub tunnels.
Track hub-originated routes feature summary
- This feature is disabled by default.
- Customers can request Meraki Support to enable this feature.
- Meraki Support may recommend enabling this feature if a hub outage impacts the reachability of overlapping data center subnets during DC-to-DC failover.
- This feature is organization-wide – routes to hub subnets will be tracked on all spokes, regardless of path redundancy.
Auto VPN Route summarization
The default static routing behavior in Auto VPN means the dashboard propagates all subnets enabled for Auto VPN on all MX appliances in the organization as configured, using the same network address and mask. The next hop for these routes depends on the role of the receiving appliance. For spokes, the next hop is the hub or hubs configured on the spoke's Security & SD-WAN > Site-to-site VPN page. For hubs, the next hop is either an attached spoke or an adjacent/non-adjacent hub.
Using the default approach can result in large routing tables, which may destabilize devices in networks with many subnets on each spoke or hub. This risk increases for low-end appliances that may lack sufficient resources to process dashboard routing updates.
The risk is greater in fast-growing environments, where administrators might not notice when the number of routes becomes too large.
How this feature works
When enabled, the dashboard applies a summarization function to contiguous subnets available behind specific peers.
Contiguous subnets –
The dashboard summarizes only contiguous subnets. For example, routes 10.0.3.0/24, 10.0.2.0/24, and 10.0.0.0/24 are not summarized as 10.0.0.0/22. Instead, they are summarized as 10.0.0.0/24 and 10.0.2.0/23.
Behind specific peers –
From the spoke perspective, the dashboard summarizes routes available behind attached hubs, including routes from adjacent spokes, non-adjacent spokes, and non-attached hubs.
For hubs, subnets of attached spokes are summarized only when contiguous subnets belong to the same spoke. For non-attached spokes, summarization may occur for contiguous subnets behind different spokes if the hub can route traffic to another hub using the summary route. For adjacent hubs, summarization occurs only when contiguous subnets are located behind the same hub.
The examples below demonstrate the summarization logic in different scenarios.
Summarization of contiguous subnets located behind spokes
This is the most common scenario. Spokes receive summarized routes to adjacent spokes from the dashboard. On the hub side, contiguous subnets behind attached spokes are summarized only if they belong to the same spoke.
Summarization of contiguous subnets located behind different spokes
In this example, the dashboard propagates a summary route for 10.11.2.0/23 to SITE-8. These subnets are not summarized on the hub, as they are located behind different peers.
Summarization of contiguous subnets located behind hubs towards spokes
The dashboard summarizes subnets located behind the same hub when propagating configuration to the spokes.
Summarization towards non-adjacent hubs
In this scenario, the dashboard summarizes networks located behind non-adjacent hubs for the AutoVPN-DC-B hub only when they are behind the same non-adjacent hub. Subnets 10.11.4.0/24 and 10.11.5.0/24 are summarized because they are located behind AutoVPN-DC-A-HA, while subnets 10.11.6.0/24 and 10.11.7.0/24 are propagated individually.
Subnets of non-attached spokes are summarized even if they are located behind different spokes, such as 10.11.2.0/24 and 10.11.3.0/24. This is because the actual peers for those summary routes from the perspective of AutoVPN-DC-B remain the same with summarization.
Route summarization feature summary
- This feature is enabled by default and applies to the entire organization.
- Customers can request Meraki Support to disable this feature.
- Meraki Support may advise disabling this feature during troubleshooting of asymmetric routing or Auto VPN failover issues. Permanent disabling may be recommended if summarization is confirmed to cause these issues.
- This feature can be applied organization-wide or to individual networks.
- Summarized routes do not have metric values.
- Summarized routes are not enabled for tracking.