By Steve Harrison
This document covers the configuration, operation, and support of the SD-Internet feature in the MX16.X code. The function of this feature is to steer customer traffic to SaaS or public cloud-based applications over the best-performing WAN connection at the time the traffic is forwarded. Whilst this feature shares a lot of functionality with Meraki SD-WAN the nature of it being outside of any transport tunnel does yield some key differences in behavior.
Currently, Meraki supports features like dynamic path selection, performance-based routing, etc on the overlay (Meraki AutoVPN) traffic as SD-WAN. This works great for the customers who have their resources located in private data centers or in the cloud infrastructure (Amazon AWS and Microsoft Azure in particular). However, many customers are moving their resources more towards cloud-based applications, some examples are Office 365, SalesForce, Oracle DB, Google Cloud, etc. Many resources like Voice and Video conferencing solutions are also mainly cloud-based for instance our own Cisco WebEx, RingCentral, Zoom, etc. These resource providers are recommending to access them over a direct internet link (Internet broadband) for better performance.
To accommodate the needs of our customers and make our solution even more effective, we have implemented a traffic steering mechanism on the direct internet links. This feature builds upon and works with the VPN exclusion feature but does not require it, to allow customers to direct new flows matching internet traffic SD-WAN policies to the best available WAN connection.
As SD-WAN over the internet suggests, this feature will be providing the similar functionality that we have on the SD-WAN overlay deployments such as Performance-based steering.
Customers will have the ability to set a policy for the applications that are excluded from VPN and routed over direct internet access.
Customers will be able to define their custom performance class for the applications that are accessed over direct internet access (DIA).
The feature will steer traffic based on the best performing available uplinks given the configured policy. E.g. if 'Best for VoIP' is configured the uplink with the best MOS will be used.
SD-Internet policies are not as fully featured as SD-WAN policies right now and at this time they only support the definition of Layer 3 & 4 source and destination-based rules whilst allowing you to define a preferred uplink policy and a ‘failover if’ condition in the same way that you can for VPN traffic policies.
The expected behavior is that the feature will failover between uplinks within 30 seconds of ‘route change needed’ detection for all new flows.
SD-Internet is still in active development hence there are certain requirements that must be met in order for the feature to be used:
Supported MX models - MX64, MX65, MX67, MX68, MX75, MX84, MX85, MX95, MX100, MX105, MX250, MX450
Unsupported MX models - MX50, MX60, MX70, MX80, MX90, MX400, MX600, vMX (all)
The SD-Internet feature is exclusively included in the SD-WAN Plus license tier, for more details on the license tiers look here.
A firmware version (16-X) is required.
Once this has been done you will be in a position to test the feature, it is recommended that testing is done in a manner that allows for the flexible control of latency, jitter, loss, and WAN availability for both primary and secondary WAN connections. An example of such a testing setup is below:
In the above example, each WAN service is connected to an inline VM that allows for the manipulation of the required criteria. If you are interested in replicating such a setup please take a look here.
Once the feature is applied the Ui on the SD-WAN & traffic shaping page will change from:
As you can see from the highlighted area in the above screenshot, the internet traffic portion of the UI has moved under the ‘SD-WAN policies’ section. The rest of the UI is unchanged.
Custom performance classes are now shared between both VPN traffic SD-WAN policies and Internet traffic SD-WAN policies.
Internet traffic policy configuration remains consistent with VPN traffic policy configuration, albeit with different options. There are 2 different SD-Internet policies that can be configured:
- Custom expression polices
- Major application polices
Custom Expression Policy Configuration
In order to configure a custom expression policy, we simply click on the 'Add a preference' link as highlighted below:
This will launch the configuration element shown below, into which you configure the source and destination traffic that should be treated by the policy defined in the lower section of the configuration element.
With a source and destination defined we can then select a preferred uplink:
We can then select a 'failover if' policy:
We can then define the failover behavior to be performance-based or WAN state-based and finally if we choose performance-based:
We can define the custom performance class that the flows being acted upon needed to comply with.
Major Applications Policy Configuration
In order to configure a major application policy, we simply click on the 'Add a preference' link as highlighted below:
This will launch the configuration element shown below, which is very similar in form and function to the configuration element for custom expression policy configuration. The only change is that the destination portion of the configuration element is now one of the top applications that we can use in the Local internet breakout section of the 'SD-WAN & traffic shaping' page.
With a source defined for the rule, the destination application can be selected:
Only one policy per destination application can be configured
The remainder of the policy configuration is exactly the same as the process illustrated above for custom expression policies and hence will not be shown again.
Path Quality Data
In addition to configuring a policy to define how packets will be routed, with SD-WAN over the internet we can also configure the endpoint that we connect to in order to judge the quality of the internet connection being used. This is possible because the SD-WAN over Internet feature uses the IP addresses that the network has configured to use for uplink statistics, this can also be found on the SD-WAN & traffic shaping page of dashboard.
The IP address chosen as the default endpoint is pinged (ICMP echo request) by the MX and the time take to respond (latency), the number of packets not returned (loss) and the variability in the linearity of the received packets (jitter) is calculated and this is used as the performance score for the entire WAN connection. In future releases of this feature, the endpoints used will be both application-specific and automatically chosen to be the most appropriate for each network.
In order to change the default Google DNS (126.96.36.199) endpoint to something more appropriate, e.g. a known perimeter IP address to a peering point for your local ISP, or a known SaaS or IaaS endpoint simply configure the IP as per the below example (that shows the use of Cisco Umbrella's primary DNS resolver):
This will result in routing decisions for the available internet connections being made by the availability of this endpoint.
In order for the MX to be able to utilize an endpoint, it needs to respond to ICMP echo requests (ping) otherwise the functionality of SD-WAN-over-Internet will be negatively impacted.
When we think about expected behavior for SD-Internet there are 2 key factors we need to consider:
Due to the stateful nature of the MX when the performance threshold of for example WAN1 connection changes such that it is no longer considered the ‘Best for VoIP’, then all new flows matching the rule for which ‘Best for VoIP’ is the preferred uplink will route via WAN2. All existing flows that are already in the MXs flow table will remain on WAN1, this is because any such existing flow would have to be reset or terminated which could lead to unwanted behavior of the application in question.
Due to the fact that SD-Internet decisions are based on the generic state of the WAN connection as measured by the MX and not of the specific path between 2 VPN endpoints, as it is for VPN traffic SD-WAN policies. Then the MX cannot confidently re-route flows as quickly as we can with VPN traffic SD-WAN policies. Such re-routing of new flows is expected to be completed within 30 seconds
1. How can I get this feature enabled?
Please raise a ticket with Meraki support who can make and request the necessary changes to enable this feature.
2. How will SD-Internet be licensed?
The SD-Internet feature requires the MX SD-WAN plus license, if you would like to test it please reach out to your Meraki sales representative or systems engineer or check here.
3. Does my MX support SD-Internet?
The SD-Internet feature is supported on all MXs except the following - MX50, MX60, MX70, MX80, MX90, MX400, MX600, vMX (all).
4. Will SD-Internet include L7 applications?
In the future, we plan to support L7 applications.
5. Why isn’t my traffic failing over when my WAN performance changes?
Changes in SD-Internet policies only apply to new flows, not existing flows. This is due to the stateful nature of the MX, to change such existing flows we would have to disconnect or reset the flow. This could lead to unwanted application behavior.
6. Where does SD-Internet get its data about the performance of the WAN connections?
SD-Internet currently uses ICMP gathered statistics from the IP address defined by the administrator in the uplink statistics section of the ‘SD-WAN & traffic shaping’ page.
7. How can I see which of my WAN connections is being used?
Currently, the only way to see which WAN connection is being used in the UI is via the ‘Live data -> Uplink traffic’ section on the ‘Uplink’ tab of the ‘Appliance status’ page.