This guide details the various deployment architectures that are possible for MX Warm Spare functionality in NAT mode and explains the advantages and disadvantages of each.
In most customer deployments, network downtime has a direct impact on the business and should be avoided at all costs. Warm Spare functionality prevents the network from having a single point of failure at the edge and allows for fast, automatic recovery in the event of device failure, reducing the negative impact on end-user services.
- In the event of hardware failure, network downtime will be greatly reduced or eliminated entirely depending on the architecture being used.
- No manual intervention by the network administration team will be required to facilitate recovery from a hardware failure.
For purposes of this document, it is important to understand the following terms and their meaning:
Primary: The MX that is configured as the "main" MX for the network. If both MXes are online, this is the MX that traffic should be flowing through. This is a static designation, meaning that regardless of the current state of the network the Primary will always be the Primary.
Spare: The MX that is configured as the "secondary" MX for the network. If both MXes are online, this is the MX that is the inactive warm spare. This is a static designation, meaning that regardless of the current state of the network the Secondary will always be the Secondary.
Active (Master): The MX that is currently acting as the edge firewall/security appliance for the network. This is a dynamic designation.
Passive: The MX that is currently acting as an inactive warm spare with no traffic passing through it. This is a dynamic designation.
Dual Master: Dual Master describes a scenario in which both the Primary and the Spare are in the Active state. This occurs when both MXes are online and communicating with the cloud, but the Secondary is not receiving heartbeat packets (see VRRP heartbeats in the next section) from the Primary. This can cause several issues with Dynamic DNS, VPN, and traffic processing in general and should be avoided at all costs. The Physical Architectures section of this document describes how to deploy an MX Warm Spare pair in order to minimize the chances of a Dual Master scenario occurring.
Underlying concepts and technologies
Failure detection for an MX Warm Spare pair uses VRRP heartbeat packets. These heartbeat packets are sent from the Primary MX to the Secondary MX on all configured VLANs in order to indicate that the Primary is online and functioning properly. As long as the Secondary is receiving these heartbeat packets, it functions in the spare state. If the Secondary stops receiving these heartbeat packets, it will assume that the Primary is offline and will transition into the master state. When the MX is in NAT mode, VRRP heartbeats are not sent over the WAN; there is no guarantee that the WAN interfaces can communicate with each other. Instead, we use a mechanism called "connection monitor" to determine the WAN state of the device.
Connection monitor is an uplink monitoring engine built into every MX Security Appliance. The mechanics of the engine are described in this article. When all uplinks of a Primary MX are marked as failed by connection monitor, that MX will stop sending VRRP heartbeat packets, which will initiate a Warm Spare failover. Once at least one uplink on the Primary returns to a working state, the Primary resumes sending heartbeat packets and the Secondary relinquishes the Active role back to the Primary.
To prevent a scenario in which an IP address is assigned by the Primary via DHCP and then that same address is assigned to another client by the Secondary after a failover, the DHCP lease table is synchronized regularly between the Primary and Secondary.
To configure warm spare failover for an existing Dashboard network, navigate to the Configure>Addressing and VLANs page. At the bottom of the page, select "Enabled" from the Warm spare drop-down. Enter the serial number of the Secondary MX and select the desired Uplink IP configuration, then click Save Changes to enable Warm Spare.
Use MX uplink IPs: When using this option, the current Active MX will use its distinct uplink IP or IPs when sending traffic out to the Internet. This option does not require additional public IPs for Internet-facing MXes, but also results in more disruptive failover because the source IP of outbound flows will change.
Use virtual uplink IPs: When using this option, both MXes will use a shared virtual IP (vIP) when sending traffic out to the Internet. This option requires an additional public IP per uplink but allows for seamless failover because the IP address the network is using to communicate with the Internet will be consistent. The vIP for each uplink must be in the same subnet as the IPs of the MXes themselves for that uplink, and the vIP must be different from both MX uplink IPs.
To configure a new network with warm spare failover, create the network as you would normally and add the Primary MX. Then add the Secondary MX using the process described above.
There are two physical architectures available for NAT Warm Spare deployments.
In this architecture, the Primary and Secondary MXes are connected via trunk ports with an ethernet cable run directly between them, as well as connected via the network infrastructure. This is the recommended architecture for most deployments, as it significantly reduces the probability of the network entering a Dual Master state by minimizing the points of failure for VRRP heartbeat communication.
In this architecture, the Primary and Secondary MXes are connected via the network infrastructure. In most cases this means that both MXes are connected to a core switch. However, they are not connected directly to each other. This is not recommended for most deployments, as it creates a much higher probability of the network entering a Dual Master state (as described in the Terminology section of this document). This could occur due to switch configuration issues, intermediate port or cable failure, or even severe network congestion.
For more information about NAT HA, please refer to our documentation on Troubleshooting MX Warm Spare in NAT Mode (NAT HA).