Home > General Administration > Firmware Upgrades > Meraki Firmware Best Practices

Meraki Firmware Best Practices

Introduction to Meraki Firmware

Cisco Meraki has always prided itself on delivering powerful networking and IT solutions in a simple, easy to manage fashion. This extends to firmware management on Meraki devices. Traditionally, firmware management is a tedious, time consuming, and risky procedure met with dread and loathing by the network administrator tasked with carrying out the upgrades, but Meraki works to limit this burden! Complexity has long plagued firmware management practices throughout the industry, spawning horror stories about experiences such as upgrades that went sideways because of a corrupted USB drive or late nights in data centers manually provisioning the new code.

Why the Cloud Makes Us Different

Meraki tackles the complex firmware narrative by leveraging the power of Meraki’s cloud-based dashboard to automate firmware scheduling. The dashboard provides unique insights into new features as they become available in new firmware releases.

From a security perspective, the benefits of the cloud are unparalleled. Today, new security vulnerabilities are constantly announced, and network infrastructure is not immune to exploits. To contain threats at this scale, flexibility and rapid software remediation is paramount. At Meraki, we have the power to immediately react to discovered exploits, patch the vulnerability, and make this firmware immediately available for customers to leverage.

Simplified Firmware Updates

In the early days of Meraki, the only firmware configuration required was to specify a convenient maintenance window for your network, such as late at night on a weekend, for example. As Meraki has grown alongside its customer base, we have incorporated tighter controls over firmware for customers who desire these while still maintaining the simplicity of cloud-based delivery. Customers can now manage firmware for each network in their organization by selecting which firmware runs on which network, while still providing the automatic upgrade functionality for customers who don’t desire this level of control.

 

image5.png

 

Meraki differentiates itself through its firmware delivery using the Meraki cloud platform, by providing an exceptionally swift and reliable way to deliver firmware upgrades. The results are evident in our users’ impressive firmware adoption rates. Even given the options for finer controls, the vast majority of our users adopt and run on our latest firmware builds almost immediately after stable release candidates are available. Our extensive testing and our beta adoption process ensures that we deliver reliable builds at a regular cadence, delivering up-to-date security and stability.

Meraki Firmware Conventions

Meraki firmware nomenclature is the same across products and consists of a major and minor number as part of the name. The firmware version is named using the format given below:

<Product Name> <Major Firmware Version>.<Minor Firmware Version>

  1. Major Versions

A new major firmware is released with the launch of new products, technologies and/or major features. New major firmware may also include additional performance, security and/or stability enhancements.

  1. Minor Versions

A new minor firmware version is released to fix any bugs or security vulnerabilities encountered during the lifecycle of a major firmware release.

 

Meraki firmware release cycle consists of three stages during the firmware rollout process namely beta, release candidate (RC) and stable firmware. This cycle is covered in more detail in the Meraki Firmware Development Lifecycle section of this document.

General Firmware Best Practices

Meraki was built on the promise of making management of devices intuitive, and this extends to Meraki firmware management. Thanks to the power of the Meraki dashboard, we are able to create and release high quality firmware that allows access to cutting-edge features and high quality, secure software. Out of the box, we recommend you let the simple, automatic and seamless updates work to your advantage. By default, your devices will be scheduled for updates when new firmware becomes available — firmware that has been robustly validated and tested before being deployed.

 

Meraki’s default firmware settings include:

  • no automatic beta firmware deployments

  • a default upgrade window

  • a default upgrade choice of Wednesdays

 

On average, Meraki deploys a new firmware version once a quarter for each product family, and this cadence ensures you get access to new features and functionalities as they become available, minimizing major changes between firmware versions to ensure high quality software.

 

Once you are scheduled for an automatic update, Meraki will notify you 2 weeks in advance of the scheduled upgrade and, within this two week time window, you have the ability to reschedule to a day and time of your choosing. We recommend selecting a time that is most convenient to your business needs, and if you want to, you can set this time as your default upgrade window under your general network settings.

 

image8.png

 

If you want to take advantage of the most advanced and newest features, we recommend that you enable the “Try beta firmware” toggle. This will give you early access to the latest Meraki firmware after it has finished the full internal automated and manual testing process in our firmware development cycle. If you are running beta firmware, you get earlier access to new features, as well as the opportunity to provide feedback on these features before they become generally available! If you have any issues on the new beta firmware you can always roll back to the previous stable version, or the previously installed version if you roll back within 14 days.

Templates

With configuration templates it is possible to push a standard configuration against multiple sites at the same time. When a network is attached to a template, the firmware is controlled by the template. It is not possible to configure a network to use a different version of firmware than what the template is configured for. In certain cases Support is able to upgrade individual devices but this should not be relied upon as this prevents normal upgrades in the future.

 


When scheduling a large template to upgrade firmware, the upgrade will automatically stagger into 15 minute intervals. This prevents upgrades from stressing the Meraki cloud by not having all devices download firmware at the same time.

Best Practice for Multi-Branch Deployments

Now that we understand how the Meraki firmware system works, let's talk about how you can leverage this to confidently manage firmware on your network.

 

image9.png

 

As shown in the diagram above, firmware should be rolled out in stages when managing a large-scale network. This approach allows you to test new features and verify stability in your production environment before rolling out new features globally.

Test Network(s) (Beta)

It is recommended to have designated network(s) to test beta firmware when released. Test networks can be a lab network or production network that is smaller but that also has enough devices to test new features. If there are production networks of different sizes in your organization, it is best to have an additional beta network of each size.

Why you Should Test Beta Firmware

Although all Meraki beta firmware undergoes rigorous testing as described in the beta release process, we recommend testing the new beta code in your designated test networks. This ensures the firmware is tested based on the needs of your unique environment and works without issues for real users.

Beta Feedback Mechanisms

If bugs are encountered during beta firmware rollout, you should contact Meraki Support to ensure the issue is documented internally, using our defined process. Opening a case will ensure the appropriate details are collected and presented to Meraki engineering teams for resolution. As part of resolution, you may be provided with a hotfix (also commonly referred to as a patch fix) to verify resolution. Once a fix is confirmed, it will be rolled into a new beta version after going through our firmware release process so customers can continue testing.

Release Candidate Network(s) (Validate)

Once beta firmware is tested, you can choose to wait until the major version reaches release candidate (RC) status or roll out beta firmware to the remaining networks if you are satisfied with the validated beta firmware. If you have a policy to only use stable firmware in production, then you can move onto the next step in the process, which is to roll out the RC firmware to designated RC networks. As mentioned in the firmware rollout process, RC is very close to stable and hence can be rolled out to a larger pool of networks in the production environment. During this phase, customers do not need to have an extensive test plan because, at this point, all new features have been tested and the focus is on widely rolling out the firmware through the network. If you are encountering problems with stable firmware,we recommend upgrading to the next release candidate to see if the problems continue. The release candidate will include many fixes that might resolve the problem.  

Stable Release Network(s) (Full Deployment)

Once a firmware is marked as stable, customers can roll out firmware to all the remaining networks either using the firmware upgrades tool or, optionally, using the automatic upgrade process to roll out firmware. If you have followed our firmware best practice for validating and testing the current Stable Release, you can deploy with confidence that it will work well in your unique environment.

 

In the scenario where you find the new beta or release candidate firmware is functioning as required and you would like to use this version on your entire deployment, go ahead and deploy this version across your entire deployment - we strive to deliver high quality firmware at all stages of our development process. If you do run into issues after the deployment, you can always easily roll back to the previous major stable firmware version.

Best Practice for MR

As our wireless portfolio grows, Meraki continues to focus on delivering the high performance and high availability network that modern deployments require. To achieve this goal we focus on minimizing downtime during an upgrade, maintaining scheduling flexibility, and preserving the accuracy of your upgrade maintenance window. Most Meraki access points (APs) will reboot in less than 1 minute after an update, ensuring minimal disruption to the end user even if they need to do a firmware upgrade during working hours.

 

Additionally, when you are running a Meraki Wireless network, it is important to keep a few things in mind to ensure you have a great Wi-Fi firmware deployment experience. First, make sure you keep all of your APs on a single firmware version. Many Wi-Fi features are depending on the same expected behavior among the access points. For example, if you are using L3 roaming, some different versions of firmware may not be compatible with each other for L3 roaming features in particular.

 

Second, when upgrading a wireless network, client devices with older drivers may have issues with new features. We test against over 100 unique client devices in our labs before shipping any wireless firmware, but it's a good idea to have a single test AP to validate clients that might be unique to your business environment. These may include a custom point of sale (POS) system or barcode scanner that is critical to your business. Again, testing these potentially unique clients on the latest Meraki Beta wireless firmware is a best practice as it ensures that Meraki can be notified of any potential interoperability issue early in the development cycle.

 

In addition to these basic best practices, Meraki APs also build features unique to the product line that make large scale firmware updates better.

Best Practice for Large Scale Wireless Networks

When running large scale campus wireless networks, traditionally, upgrading wireless firmware has been considered risky. Thanks to Meraki’s agile and cloud-based firmware development process, there are a few things you can do to make these deployments less risky. Even in the largest networks, the best practice with Meraki is to designate an isolated area of your network to test and validate the newest Meraki firmware. With a designated Meraki MR test area, you can get access to validate all Meraki wireless firmware in your physical environment. This allows you to engage with Meraki directly, earlier in the software development process, so you can help provide feedback on new features and identify any potential issue that may affect your deployment.

 

image2.png

 

In the example above we have designated the cafeteria and a large meeting area as our Meraki test area. This was done by moving the selected APs into their own dashboard network so they could be assigned a (beta) firmware version, separate from the main network(s). This also allows the APs to be rolled back to a stable version quickly, if needed, by simply moving the APs back to the main production dashboard network. We chose this area of our deployment that includes six Meraki access points as it provides us with a diverse group of client devices, as almost all employees frequent this area of the building, and at the same time, the impact of a potential wireless issue will be more manageable to the users in this environment.

 

Once you have validated and are comfortable with the current firmware in the test environment, you can confidently deploy the update to the rest of your network. It is important to note that, in this example, you may occasionally have some roaming issues as users navigate in and out of the designated test area, because the deployed firmware versions may be different, and roaming may not yet be seamless between the two versions.

Best Practice for MS

When you move farther up the networking stack to switching there are additional things you need to take into consideration. It’s always important to consider the topology of your switches as, when you drive closer to the network core and away from the access layer, the risk during a firmware upgrade increases. Because of this, in a larger switch-based network you should always start the upgrade closest to the access layer. Two unique aspects of managing Meraki switch firmware is that we support both:

 

  • Staged upgrades to allow you to upgrade in logical increments. For instance, starting from low-risk locations at the access layer and moving onto the higher risk core, and...

  • Automatic updates for switch stacks

 

When upgrading Meraki switches it is important that you allocate enough time in your upgrade window for each group or phase to ensure a smooth transition. Each upgrade cycle needs enough time to download the new version to the switches, perform the upgrade, allow the network to reconverge around protocols such as spanning tree and OSPF that may be configured in your network, and some extra time to potentially roll back if any issue is uncovered after the upgrade.

 

The high-level process for a switch upgrade involves the following:

  1. The switch downloads the new firmware (time varies depending on your connection)

  2. The switch starts a countdown of 20 minutes to allow any other switches downstream to finish their download

  3. The switch reboots with its new firmware (about a minute)

  4. Network protocols reconverge (varies depending on configuration)

Staged Upgrades

To make managing complex switched networks simpler, Meraki supports automatic staged firmware updates. This allows you to easily designate groups of switches into different upgrade stages. When you are scheduling your upgrades you can easily (as in the example below) mark multiple stages of upgrades. In this case, we started with the access layer switches in Stage 1 and gradually upgraded toward the core in Stage 3. Once you start the staged upgrade, the Stage 1 switches will complete the entire upgrade cycle before the Stage 2 upgrades start. This cycle will repeat until all the switches are upgraded in all three stages.

 

image1.png

Upgrading a Switch Stack

In addition to supporting staged upgrades, Meraki also simplifies managing a switch stack. As part of our upgrade toolset, we automatically handle the upgrade of the entire switch stack. This upgrade manages the upload of firmware to each switch and takes care of each reboot within the switch stack. If you are upgrading switch stacks within your staged upgrade, Meraki will automatically upgrade the switch stack as part the staged upgrades. The upgrade process for a stack follows the same high-level process outlined previously, with each stack member rebooting close to the same time and the stack then automatically re-forming as the members come online

Best Practice for MX

Unlike many other products offered by Meraki, MX appliances and Z-Series devices have a one-dashboard-network per-site model. This provides very granular control of how upgrades can be managed across the deployment.

 

All firmware upgrades will require that the MX appliance reboots, so it is important to ensure that an appropriate maintenance window has been put in place, as the MX upgrade process will take down the entire local network in most scenarios. Given the central/upstream nature of MX devices, it is also recommended to allow for sufficient time to monitor and test after the upgrade completes to ensure the maintenance window completes successfully.

 

When managing a deployment with many MXs, the following are useful best practices that can help make firmware transitions and management simpler.

Appliance Network with Two MXs in an HA Configuration

When MX appliances configured to operate in High Availability (HA) (either in NAT/routed mode or when operating as one-armed VPN concentrators), the dashboard will automatically take steps to minimize downtime when upgrades are performed to ensure a zero-downtime MX upgrade. This is achieved through the following automated process:

 

  1. The Primary MX downloads firmware

  2. The Primary MX stops advertising VRRP

  3. The Secondary MX becomes master

  4. The Primary MX reboots

  5. The Primary MX comes online again

  6. The Primary MX starts advertising VRRP again

  7. The Primary MX becomes Master Again

  8. The Secondary MX downloads firmware (approximately 15 minutes after the original upgrade is scheduled)

  9. The Secondary MX stops advertising VRRP

  10. The Secondary MX reboots and comes back online

Deployments using AutoVPN

When upgrading a VPN concentrator, it is important to plan for a maintenance window that allows for the upgrades to complete and for verifications to be performed that ensure connectivity is fully re-established and network systems are healthy.

 

It is highly recommended that customers plan for maintenance windows in accordance with the scale and complexity of the deployment where the upgrades are being performed. For example, more time should be allotted for upgrading a VPN concentrator supporting 1000 spoke sites and leveraging a dynamic routing connection between the concentrator and datacenter, than for a VPN concentrator with only 10 spoke sites.

 

If beta firmware is being tested on a VPN concentrator, it is best to plan for time in the maintenance window to allow for the upgrade to complete and validate the operational state after the upgrade has been completed. It is also recommended to allocate an additional window of time for rolling back to the previous build, in case you run into unmanageable issues.

 

When concentrators are configured in HA, they will follow the steps mentioned above. VPN tunnels will begin establishing to the spare appliance while the primary is upgrading. However, the primary appliances typically complete the upgrades fast enough that spoke sites have minimal interactions with the spare concentrator.

 

In general, even with equipment in HA, it is best to always be prepared for some amount of downtime and impact for spoke sites. In almost all cases these are simply a matter of seconds as spoke sites fail between concentrator pairs, but the impact can become more noticeable if there are WAN connectivity problems between the data center and spoke locations.

Meraki Firmware Development Lifecycle

One of the key advantages of being a cloud managed device company is that Meraki is able to leverage full internal automated testing, while also being able to utilize our cloud to monitor key device performance metrics across our entire installed user base. To ensure robust and reliable firmware development, Meraki follows a consistent software release process to validate and deploy consistent and reliable firmware. Meraki's firmware development process has four stages: alpha, beta, stable release candidate (RC), and stable. Every firmware version is created and released with the goal of graduating to stable. If a particular build fails to pass our key metrics at any stage of the development process, a new build is created and the process begins anew. The following sections go over each of the stages in more detail.

 

image6.png

Alpha Pre-Release

With all new Meraki firmware including both major and minor releases, we start out every new build by running it through our full alpha testing process. Before any release hits our users’ hands, we validate the release by running it through our ever-expanding testing suites, and check for regressions or new features that are not performing as expected. Each product line has automated and manual testing specific to the product, that are designed to ensure Meraki minimizes the chance of regressions as we continue to create and expand on our software feature set.

 

As part of our core philosophy, after a new build has successfully passed the testing phase, we deploy the new firmware release on our own personal and engineering networks. We believe it is important that we deploy and run our own firmware before any of our customers deploy our firmware. During this process we will run this firmware in our real world deployments for one or more weeks before we consider releasing the build as a new beta version.

 

image3.jpg

 

If a build successfully passes all of our release criteria, we will start to make the new build available to our customer base. If any issues are discovered that need to be resolved, we will start the process over once the issue has been addressed before moving the release forward. In some more rare cases, we will move forward with a build with a known regression, due to complexity or timing of the fix, and in this scenario we will note the regression in the release notes for that version.

Beta Release

Firmware is made available for production use at first under "Beta." Often customers will run beta firmware in their production network to take advantage of new features and bug fixes. Beta firmware has already gone through internal regression, stability, and performance testing to limit risks when applied to production networks. Customers that opt into beta firmware via the Try beta firmware configuration option on dashboard will be automatically notified and scheduled to upgrade to these versions as they are released.These upgrades can be canceled, modified, and reverted using the firmware upgrades tool in the dashboard. Customers can also manually upgrade their networks at any time to beta firmware by using the firmware upgrade tool. Beta firmware can be considered analogous to “Early Deployment” firmware seen in other products in the industry.

The latest beta firmware is fully supported by our Support and Engineering teams. Older betas are supported with best effort; an upgrade to the latest beta will ensure full support.

Stable Release Candidate

As a new firmware version matures from beta, it has the opportunity to graduate into a stable release candidate. A formal review of the beta firmware’s success is conducted by our software and product teams. Key performance indicators (KPIs) for quantifying firmware quality are analyzed including open support cases & engineering issues, firmware adoption, and stability metrics. After the formal review, a beta may be reclassified as a "Stable Release Candidate." At this point the firmware version will be indicated as such in the firmware upgrade tool. Once a new stable release candidate is available, Engineering will begin scheduling a limited set of customers for upgrade. These upgrades can be canceled, modified, or reverted using the firmware upgrade tool as well.

The latest stable release candidate firmware is fully supported by our Support and Engineering teams. Older stable release candidates are supported with best effort; an upgrade to the latest beta, stable release candidate, or stable will ensure full support.

Stable Release

A stable release candidate matures into a stable version over time as it is slowly rolled out to devices globally. When Meraki's install-base hits a specified threshold for a major version (roughly 20% of nodes), that firmware revision will be promoted to stable, pending a final formal review. For point releases, the determination will be made on a case-by-case basis.  

Again, the same KPIs are analyzed as used in the stable release candidate review. Upon completion of these processes the firmware can be promoted to "Stable." After promotion, stable versions can be applied by any customer via the firmware upgrade tool on dashboard. The latest stable version is also the version that is used for all newly created dashboard networks for a particular device.

Firmware Upgrade Tool

To make all of the best practices above simple to manage, you can use the Meraki firmware upgrade tool. We have built this tool to allow organizations to easily manage all Meraki firmware across the product portfolio in a single dashboard. As with all of our cloud features, we are continuing to build out more functionality in the firmware upgrade tool to increase usability and simplify firmware management.

 

image4.png

 

On the Overview tab, customers find a variety of information, such as a list of recent upgrades in the dashboard organization, pending upgrades that have been automatically or manually scheduled, the ability to cancel or reschedule these upgrades as well as a list of firmware versions that are available in beta, stable release candidate, or stable form for a given Meraki product.

Included with the available beta, stable release candidate, and stable firmware versions available in dashboard is a list of changelog notes. These notes allow customers to be fully aware of any new features, bug fixes, and existing known issues found between their existing firmware in use and the version planned for upgrade. Customers leveraging configuration templates may also enjoy the benefits of the firmware upgrade tool.

Requesting Specific Versions of Firmware

If there is a specific version of firmware that is needed for reasons of compatibility, then it can be requested from Meraki Support. Please note that any problems that are encountered while running versions of firmware that are not either stable or release candidate will not be supported and Meraki Support may need to recommend upgrading to the latest version of firmware in order to continue troubleshooting.

Locked Firmware

In general, it is discouraged to upgrade firmware on specific devices rather than upgrading the entire network. However, during the course of troubleshooting, Meraki Support may find it necessary to try a particular version of firmware on a specific device. For devices that have their firmware set manually by Meraki Support, you’ll see the message: Firmware version locked, please contact Support.

Removal of the locked firmware cannot be scheduled, please call Meraki Support to have the locks immediately removed. The device will then automatically downgrade/upgrade to match the network firmware.

Additional Documentation

In addition to this best practices document please reference our other documentation to help you best deploy your Meraki products:

 

Last modified

Tags

Classifications

This page has no classifications.

Explore the Product

Click to Learn More

Article ID

ID: 7405

Explore Meraki

You can find out more about Cisco Meraki on our main site, including information on products, contacting sales and finding a vendor.

Explore Meraki

Contact Support

Most questions can be answered by reviewing our documentation, but if you need more help, Cisco Meraki Support is ready to work with you.

Open a Case

Ask the Community

In the Meraki Community, you can keep track of the latest announcements, find answers provided by fellow Meraki users and ask questions of your own.

Visit the Community