Skip to main content

 

Cisco Meraki Documentation

vMX Setup for KVM [Outdated]

Overview

This document is meant for customers and engineers who want to run vMX in QEMU/KVM environment. Please, note this is currently in private beta and QEMU/KVM environment will be officially supported in the upcoming MX releases. Also, that KVM and QEMU are interchangeable in this document context. The vMX support for both are the same from the firmware POV.

Key Concepts

Before deploying a vMX, it is important to understand several key concepts:

Concentrator Mode

All MXs can be configured in either NAT or VPN concentrator mode. There are important considerations for both modes. For more detailed information, see the article on concentrator modes.

One-Armed Concentrator 

In this mode, the MX is configured with a single Ethernet connection to the upstream network. All traffic will be sent and received on this interface. This is the only supported configuration for MX appliances serving as VPN termination points into the AWS Cloud.

NAT Mode Concentrator 

In this mode, any traffic coming over auto-VPN or client VPN to the vMX will be NATed to the vMX's IP as it egresses the vMX. Other capabilities of the NAT mode including DHCP, HA or multiple ports (LAN and WAN) are not supported.  In each mode the vMX is still a one-armed appliance with one network interface. 

What is QEMU?

QEMU is a machine emulator that can run operating systems and programs for one machine on a different machine. Most often, QEMU is used as a virtualizer in collaboration with KVM components. The most popular QEMU emulators is libvirt . QEMU can also be deployed on any linux based operating system by using qemu-system-x86_64 process.

 

Meraki Dashboard Configuration 

Begin by creating a new Security Appliance network in your organization. If needed, please refer to the guide on creating a new network in the Meraki dashboard.

1. Add license(s) to the Meraki dashboard

To complete the vMX Meraki dashboard configuration, a vMX license must be available for use in your organization.

If your organization has already reached its vMX license limit, you will be unable to create new vMX networks until a vMX network is deleted or additional vMX licensing added.

If you do not have access to a vMX license or require additional vMX licenses, please reach out to your Meraki reseller or sales representative.

 

2. Create a "Security appliance" network type: 

Screen Shot 2020-07-20 at 1.15.19 PM.png

 

3. Assign vMX type to network

Once you have created the "Security appliance" network and added the appropriate license you will be able to deploy a new vMX to your network by clicking on the 'Add vMX' button.

add_vmx.png

 

4. Generate the authentication token

Before generating the token, please verify the firmware is configured for MX 15.37+. If the vMX network firmware is set to anything below that, the upgrade will not occur.

After you add the new vMX to your network, navigate to Security & SD-WAN > Monitor > Appliance status and select “Generate authentication token” to generate the token for the AWS user-data field.

auth_token.png

5. Copy the newly generated token and save it.

The newly generated token will be used in the "System Configurations" section when creating a new AWS Cloud instance.

The authentication token must be entered into the AWS Cloud instance within one hour of generating it, otherwise a new token must be generated. 

KVM Setup

Create a KVM Server (Optional and one-time setup)

  1. Deploy a linux instance (for example: ubunutu).

  2. Download an image and follow the steps mentioned on the official ubuntu/linux website

  3. Install libvirt packages
    1. https://ubuntu.com/server/docs/virtualization-libvirt - Follow instructions on this page. 
  4. Confirm libvirt is properly installed by running: virsh list and kvm-ok

Download the Meraki vMX .qcow2 image

Download the vMX image from the following URL

Screenshot 2024-04-24 at 22.35.42.png

Copy the image to preferred KVM server

Once the image is downloaded, copy it over to the desired KVM server that would be used for deployment. 

If using SCP, the scp cipher has to be aes128-ctr.

scp -c aes128-ctr -r vmx.qcow2 host:~/PATH_TO_IMAGE

 Create an XML definition

A sample xml definition can be found below. Please note the key definitions for the interfaces, paths to the image files.

<domain type='kvm'>
    <name>vmx</name>
    <memory unit='GB'>8</memory>
    <currentMemory unit='GB'>8</currentMemory>
    <vcpu placement='static'>4</vcpu>
    <os>
        <type arch='x86_64'>hvm</type>
        <boot dev='hd'/>
    </os>
    <cpu mode='host-passthrough'>
        <numa>
                    <cell id='0' cpus='0-3' memory='8' unit='GB'/>
                </numa>
    </cpu>
    <features>
            <acpi/>
            <apic/>
    </features>
    <clock offset='utc'>
        <timer name='rtc' tickpolicy='catchup'/>
        <timer name='pit' tickpolicy='delay'/>
        <timer name='hpet' present='no'/>
    </clock>
    <on_poweroff>destroy</on_poweroff>
    <on_reboot>restart</on_reboot>
    <on_crash>destroy</on_crash>
    <pm>
        <suspend-to-mem enabled='no'/>
        <suspend-to-disk enabled='no'/>
    </pm>
    <memoryBacking>
        <hugepages>
            <page size='2048' unit='KiB'/>
        </hugepages>
        </memoryBacking>
    <devices>
        <emulator>/usr/bin/qemu-system-x86_64</emulator>
        <disk type='file' device='disk'>
            <driver name='qemu' type='qcow2'/>
            <source file='/var/lib/libvirt/images/meraki.qcow2'/>
            <backingStore/>
            <target dev='hda' bus='ide'/>
        </disk>
        <disk type='file' device='cdrom'>
            <driver name='qemu' type='raw'/>
            <source file='/var/lib/libvirt/images/token.iso'/>
            <target dev='sda' bus='ide'/>
            <readonly/>
            <address type='drive' controller='0' bus='1' target='0' unit='1'/>
        </disk>
        <interface type='direct' trustGuestRxFilters='yes'>
            <source dev='ens35' mode='passthrough'/>
            <model type='virtio'/>
            <link state='up'/>
            <mac address='cc:03:d9:03:05:c3'/>
        </interface>
        <serial type='pty'>
                        <target type='isa-serial' port='0'>
                                <model name='isa-serial'/>
                        </target>
                </serial>
        <console type='pty'>
            <target type='serial' port='0'/>
        </console>
    </devices>
</domain>

Create ISO file with vmx_authentication_token

Copy the token generated from the dashboard and create a user-data file in your local machine

echo "token <token value" > user-data

Next, create ISO file with the user-data file. This file would be attached as a cd-rom on the vMX and mounted at run time. 

mkisofs -output <ISO file name>.iso -volid cidata -joliet -rock user-data

Start the VMX

virsh define vmx.xml #create the virsh defintion
virsh start vmx #starts the instance

Troubleshooting 

The most common problem people face when deploying a vMX is getting it provisioned and online in their Meraki dashboard in the first place. New troubleshooting/diagnosis messages have been added to the vMX console screen now so you can identify what went wrong during the provisioning process.

When the vMX boots it will execute the following steps during its initial provisioning process:

  1. Obtain user-data (vMX auth token)
  2. Authenticate with the dashboard (using auth token)
  3. Connect to dashboard

Obtain User Data and Authenticate to Dashboard 

Once a vMX has successfully connected to a network, it will then attempt to obtain its user data (vMX auth token). There are different user-data mechanisms in each platform that the vMX currently runs on to provide the token to the vMX. In AWS, Azure, GCP, and Alicloud there are user-data fields in the VM configuration where this can be provided. 

Token Expired 

vMX auth tokens have a lifetime of only one hour for security purposes. If you see the following message on your vMX console it means the token you provided is no longer valid.  

Token_Expired.png

Invalid Token 

If the token provided is incorrect in any way the "invalid token" message is displayed on the console.

Invalid_Token.png

Unable to Reach Meraki Dashboard 

If the vMX is unable to reach the dashboard on TCP port 7734 then the initial provisioning phase will fail and an "Unable to reach Meraki Dashboard" message will be displayed on the console (check the firewall information page for a list of all the firewall rules needed for the Cisco Meraki cloud communication). Please refer to the document on correct ports/IPs that need to be opened for Meraki dashboard communication.
Note: Please ensure that the VPC route table is able to get out to the internet.

Unable_to_Reach_Dash.png

  • Was this article helpful?