Skip to main content
Cisco Meraki Documentation

Datacenter Redundancy (DC-DC Failover) Deployment Guide

Overview

Cisco Meraki's Datacenter Redundancy (DC-DC Failover) feature allows for network traffic sent across AutoVPN to failover between multiple geographically distributed datacenters. This guide introduces the various components of DC-DC failover and the possible ways in which to deploy a DC-DC failover architecture.

Key Concepts

Before deploying DC-DC Failover, it is important to understand several key concepts.

Concentrator Mode

All MXs can be configured in either NAT or VPN concentrator mode. There are important considerations for both modes. For more detailed information on concentrator modes, click here.

One-Armed Concentrator

In this mode the MX is configured with a single Ethernet connection to the upstream network. All traffic will be sent and received on this interface. This is the recommended configuration for MX appliances serving as VPN termination points into the datacenter.

NAT Mode Concentrator

In this mode the MX is configured with a single Ethernet connection to the upstream network and one Ethernet connection to the downstream network. VPN traffic is received and sent on the WAN interfaces connecting the MX to the upstream network and the decrypted, unencapsulated traffic is sent and received on the LAN interface that connects the MX to the downstream network. 

VPN Topology

There are several options available for the structure of the VPN deployment.

Split Tunnel

In this configuration, branches will only send traffic across the VPN if it is destined for a specific subnet that is being advertised into the AutoVPN topology by another MX in the same Dashboard organization. The remaining traffic will be checked against other available routes, such as static LAN routes and third-party VPN routes, and if not matched will be NATed and sent out the branch MX unencrypted.

Full Tunnel

In full tunnel mode all traffic that the branch or remote office does not have another available route to is sent to a VPN hub.

Hub and Spoke

In a hub and spoke configuration, the MX security appliances at the branches and remote offices connect directly to specific MX appliances and will not form tunnels to other MX or Z1 devices in the organization. Communication between branch sites or remote offices is available through the configured VPN hubs. 

VPN Mesh

In a mesh configuration, an MX appliance at the branch or remote office is configured to connect directly to any other MXs in the organization that are also in mesh mode, as well as any spoke MXs that are configured to use it as a hub.

Architecture

A DC-DC failover architecture is as follows:

  • One-armed VPN concentrators or NAT mode concentrators in each DC
  • A subnet(s) or static route(s) advertised by two or more concentrators
  • Hub & Spoke or VPN Mesh topology
  • Split or full tunnel configuration

 

DC-DC Failover Supported Architectures

Split Tunnel

Full Tunnel

NAT Mode Concentrator One-Armed Concentrator NAT Mode Concentrator One-Armed Concentrator
Hub & Spoke
VPN Mesh

Operation and Failover

Deploying one or more MXs to act as VPN concentrators in additional datacenters provides greater redundancy for critical network services. In a dual- or multi-datacenter configuration, identical subnets are advertised from each datacenter with a VPN concentrator mode MX. 
 
In a DC-DC failover design, a remote site will form VPN tunnels to all configured VPN hubs for the network. For subnets that are unique to a particular hub, traffic will be routed directly to that hub. For subnets that are advertised from multiple hubs, spokes sites will send traffic to the highest priority hub that is reachable.

 

When an MX Security Appliance is configured to connect to multiple VPN concentrators advertising the same subnets, the routes to those subnets become tracked. Hello messages are periodically sent across the tunnels from the remote site to the VPN hubs to monitor connectivity. If the tunnel to the highest priority hub goes down, the route is removed from the route table and traffic is routed to the next highest priority hub that is reachable.

This route failover operation only applies when identical routes are advertised from multiple AutoVPN hubs.

 

Warning: Failover between datacenters typically occurs in 20 to 30 seconds after connectivity between the remote site and the datacenter is lost.  The failover time will vary if utilizing BGP, due to the iBGP hold-timer.

Datacenter Deployment

DC-DC failover can be configured with either a one-armed VPN concentrator or a NAT mode concentrator at each datacenter location. 

Configuring a One-Armed Concentrator

A Cisco Meraki MX can be provisioned as a VPN concentrator at each datacenter remote sites will need to connect to. This section will provide a brief overview of configuring a one-armed concentrators and discuss special configuration considerations. For more detailed steps on configuring a one-armed concentrator, please refer to this article.

Example Topology

The following diagram shows an example topology using a hub & spoke configuration with a one-armed VPN concentrator.

Screen Shot 2016-08-26 at 7.40.35 PM.png

IP Assignment

In the datacenter, an MX Security Appliance can operate using a static IP address or an address from DHCP. MX appliances will attempt to pull DHCP addresses by default. It is highly recommended to assign static IP addresses to VPN concentrators.

 

Static IP assignment can be configured via the device local status page.

Operating Mode Configuration

Begin by configuring the MX to operate in VPN Concentrator mode. This setting is found on the Security & SD-WAN > Configure > Addressing & VLANs page. The MX will be set to operate in NAT mode by default.
 

DC Passthrough mode.png

Site-to-site VPN Configuration

Next, configure the local networks that are accessible upstream of this VPN concentrator.

 

First, set the type to Hub (Mesh). Then for the local networks configuration name, specify a descriptive title for the subnet. For the subnet, specify the subnet to be advertised to other AutoVPN peers using CIDR notation. NAT traversal can be set to either automatic or manual. An example screenshot is included below:

   Screen Shot 2015-11-25 at 12.45.44 PM.png

Configure a NAT-Mode Concentrator

A Cisco Meraki MX can be provisioned as a VPN concentrator at each datacenter remote sites will need to connect to. This section will provide a brief overview of configuring a NAT mode concentrator and discuss special configuration considerations. For more detailed steps on configuring a NAT mode concentrator, please refer to this article.

Example Topology

The following diagram shows an example topology using a hub & spoke configuration with a NAT mode VPN concentrator.

Screen Shot 2016-09-06 at 5.19.10 PM.png

IP Assignment

In the datacenter, an MX Security Appliance can operate using a static IP address or an address from DHCP. MX appliances will attempt to pull DHCP addresses by default. It is highly recommended to assign static IP addresses to VPN concentrators.

 

Static IP assignment can be configured via the device local status page.

Operating Mode Configuration

Begin by configuring the MX to operate in Routed mode. This setting is found on the Security & SD-WAN > Configure > Addressing & VLANs page
 

DC Routed Mode.png

Static Route Configuration

DC-DC failover for NAT mode MXs requires redundant static routes be advertised into the AutoVPN topology from two or more NAT mode concentrators. Any local VLAN configured to be advertised into the AutoVPN topology on a NAT mode MX must be unique.

 

To define a static route, begin by navigating to the Security & SD-WAN > Configure > Addressing & VLANs page.

 

Click on the Add static route to open the Static Route configuration menu.

DC Static Route.png 

 

In the Static Route configuration menu, define the Name, Subnet, Next hop IPActive state, and the VPN Mode status.

DC Add Route.png

 

The in VPN configuration option on the static route configuration menu will only appear if VPN has already been enabled on the Security & SD-WAN > Configure > Site-to-site VPN page. 

 

Static routes can also be configured to be advertised into the AutoVPN topology from the Site-to-site VPN page. 

 

Please see here for more information on configuring static routes on NAT mode MXs.

Site-to-site VPN Configuration

Next, enable site-to-site VPN and configure the static routes that are allowed in the VPN.

 

First, set the type to Hub (Mesh). Then for the local networks configuration, set use VPN to yes for subnets that should be advertised to AutoVPN peers. NAT traversal can be set to either automatic or manual. An example screenshot is included below:

 

Screen Shot 2016-08-24 at 5.08.49 PM.png

 

Configuring Additional Datacenters

The same steps used above can also be used to deploy additional one-armed or NAT mode concentrators at one or more additional datacenters.

 

To provide redundant access to a specific set of datacenter resources, the subnet must be defined in the local networks for each additional datacenter and allowed to use VPN for each concentrator that can provide access to the subnet.

Concentrator Priority

When multiple VPN hubs are configured for an organization, the concentrator priority can be configured for the organization. This setting determines the order in which VPN mesh peers will prefer to connect to subnets advertised by multiple VPN concentrators.

Note: This setting does not apply to remote sites configured as VPN spokes.

Additional Datacenter Considerations

In addition to the deployment steps above, a warm spare concentrator configuration and OSPF route advertisement should be taken into consideration for MXs acting as VPN concentrators in a datacenter. 

Warm Spare

A warm spare configuration uses two MX VPN concentrators in a single datacenter, with one MX serving as a stand-by warm spare to ensure high availably. More information about MX warm spare configuration can be found here.

Branch Deployment

This section will outline the configuration and implementation of the DC-DC failover architecture in the branch.

Prerequisites 

Before configuring and building AutoVPN tunnels, there are several configuration steps that should be reviewed.

WAN Interface Configuration 

While automatic uplink configuration via DHCP is sufficient in many cases, some deployments may require manual uplink configuration of the MX security appliance at the branch. The procedure for assigning static IP addresses to WAN interfaces can be found here.

 

Some MX models have only one dedicated Internet port and require a LAN port be configured to act as a secondary Internet port via the device local status page if two uplink connections are required. This currently includes all MX models other than the MX65, MX65W, MX84, MX400, and MX600. This configuration change can be performed on the device local status page on the Configure tab.

Subnet Configuration

AutoVPN allows for the addition and removal of subnets from the AutoVPN topology with a few clicks. The appropriate subnets should be configured before proceeding with the site-to-site VPN configuration.

 

Begin by configuring the subnets to be used at the branch from the Security & SD-WAN > Configure > Addressing & VLANs page. The branch MX can be configured to only use a Single LAN, or can be configured with multiple VLANs. Each VLAN can individually be configured to be allowed in the VPN or not.

 

DC Single LAN.png

To configure the local subnet when VLANs are disabled, click on the default subnet in the LAN Config section and adjust the subnet and VLAN Interface IP.

 

To configure VLANs, set the LAN Setting to VLANs. To modify the default VLAN, click on its entry in the LAN Config section. To add additional VLANs, click the Add VLAN link besides the Subnets section and define the NameSubnet, VLAN Interface IP, and VLAN ID.

 

For more information on configuring subnets, VLANs, and static routes on an MX appliance please see this article.

Configuring AutoVPN 

Once the subnets have been configured, Cisco Meraki's AutoVPN can be configured via the Security & SD-WAN > Configure > Site-to-site VPN page in Dashboard.

Configuring Hub and Spoke VPN 

From the Security & SD-WAN > Configure > Site-to-Site VPN page: 

  1. Select spoke for the type
  2. Under Hubs, select Add a hub
  3. To connect to additional hubs, select Add a hub and select the VPN concentrator configured in the datacenter deployment steps
  4. Additional hubs can be added by repeating steps 2 and 3

 

Screen Shot 2015-12-11 at 1.49.51 PM.png

Hub priorities 

Hub priority is based on the position of individual hubs in the hubs list from top to bottom. The first hub has the highest priority, the second hub the second highest priority, and so on. Traffic destined for subnets advertised from multiple hubs will be sent to the highest priority hub that has an established VPN connection with the spoke. Traffic to subnets advertised by only one hub is sent directly to that hub.

To leverage DC-DC failover, the branch site must have at least two hubs defined in the site-to-site VPN page that advertise the same subnet into the AutoVPN topology (split tunnel) or that the default route checkbox is selected for (full tunnel).

Configuring split and full-tunnel VPN connections

To configure full tunnel VPN to a particular hub, check the default route box.

 

To configure split tunnel VPN to a particular hub, do not select the default route for the VPN hub.

DC-DC failover can be leveraged in split tunnel configurations, full tunnel configurations, as well as in mixed configurations. In a mixed configuration where the branch connects to some VPN hubs with a full tunnel connection and some as split tunnel connection, it is important to remember that redundancy is only provided for subnets that are advertised as two or more hub's local networks.

Configuring Allowed Networks 

To allow a particular subnet to communicate across the VPN, locate the local networks section in the Security & SD-WAN > Configure > Site-to-site VPN page. The list of subnets is populated from the configured local subnets and static routes in the Security & SD-WAN > Configure > Addressing & VLANs page, as well as the Client VPN subnet if one is configured.

 

To allow a subnet to use the VPN, set the use VPN drop-down to yes for that subnet.

NAT Traversal 

Please refer to the datacenter deployment steps here for more information on NAT Traversal options.

VPN Mesh Configuration

From the Security & SD-WAN > Configure > Site-to-Site VPN page: 

  • Select Hub for the type

clipboard_e56085ad90c33a6d821ac8c409cbd9a79

 

A VPN hub will form AutoVPN tunnels to all other configured VPN hubs in the organization and all of it's configured VPN spoke sites.

Configuring split and full-tunnel VPN connections

To configure full tunnel VPN to a particular hub, an exit hub will need to be defined. Click add a hub and select the desired exit hub. Multiple exit hubs can be defined.

 

Split tunnel connections are made from the branch location to all other VPN hubs that are not designated in the list of exit hubs.

Configuring Allowed Networks 

To allow a particular subnet to communicate across the VPN, locate the local networks section in the Site-to-site VPN page. The list of subnets is populated from the configured local subnets and static routes in the Addressing & VLANs page, as well as the Client VPN subnet if one is configured.

 

To allow a subnet to use the VPN, set the use VPN drop-down to yes for that subnet.

Hub priorities 

Hub priority in a VPN mesh configuration is based on the position of individual hubs in the concentrator priority list and the list of exit hubs, from top to bottom.

 

Concentrator priority is an organization-wide setting that affects the priority for split tunnel connections between hubs.

 

Screen Shot 2016-08-19 at 4.32.56 PM.png

 

Exit hub configurations are specific to an individual MX network and affect full tunnel connections between hubs. When exit hubs are defined, traffic that the branch does not have another available route to is sent to the configured exit hubs.

 

Screen Shot 2016-10-12 at 6.37.25 PM.png

 

In both cases, the first hub has the highest priority, the second hub the second highest priority, and so on. Traffic destined for subnets advertised from multiple hubs will be sent to the highest priority hub that currently has an established VPN connection with the site. Traffic to subnets advertised by only one hub will be sent directly to that hub.

To leverage DC-DC failover, the branch site must have at least two hubs defined in the site-to-site VPN page that advertise the same subnet into the AutoVPN topology (split tunnel) or that have been designated as exit hubs (full tunnel).

NAT Traversal 

Please refer to the datacenter deployment steps here for more information on NAT Traversal options.

Routing Considerations

MX security appliances perform decisions on how to forward traffic based on a routing table built from the network's configuration in the Meraki Dashboard. In determining the appropriate destination to forward traffic, the MX will prefer the most specific route to the destination IP address. 

 

For more detailed information on MX routing behavior, please see this article.

  • Was this article helpful?