How to Deploy Custom Computer Vision on MV Cameras

Enable Custom CV using the Meraki dashboard 

To enable MV Sense Custom CV using the dashboard, navigate to the  Settings > Sense > Custom CV for the desired camera. If the Custom CV section is not visible, enable the Sense API. 

If you have not uploaded any Custom CV artifacts, complete the following steps:

1. Select Custom CV Enable button.

2. Agree to the terms and conditions

3. Select add custom model.

4. Upload an artifact: provide a model name and an artifact zip file

5. Once uploaded, close the modal window.

6. Select a model from the dropdown list.

7. Optionally, you can set a detection threshold that filters detections based on prediction confidence score. Defaults to 0.5 (50%).

8. Select Save on the top right corner when artifact is successfully uploaded.

If you have already uploaded an artifact:

1. Select Custom CV Enable button.

2. Agree to the terms and conditions.

3. Choose the model from the dropdown list.

4. (Optional) Set a detection threshold that filters detections based on prediction confidence score. Defaults to 0.5 (50%).

5. Select Save on the top right corner.

Enable Custom CV using the dashboard API 

To get started with API please refer to the API documentation which is accessible through Dashboard > Help > API docs Custom CV section.

To use the API, you need both the Dashboard API Key and the Organization ID.

Retrieve detection data via MQTT 

All MV cameras provide an MQTT client to retrieve detection outputs. If you are not familiar with MQTT, refer to the documentation to learn more about protocol and how to enable it for your organization.

To receive detection outputs from camera subscribe to the topic below from your MQTT broker. 

`/merakimv/<DEVICE_SERIAL>/custom_analytics`

Example MQTT message:

{
    "outputs":
        [{
            "class":0,
            "id":123,
            "location":[0.1,0.2,0.4,0.3],
            "score":0.7
         }],
     "timestamp":1646175500000
}

Outputs: An array of detected objects

Detections: Each detection represents a detected object that has its object class type, detection id, location coordinates [left, top, right, bottom] or [x0,y0,x1,y1] and probability score.

Timestamp: time & date when detections occur

Configure Custom CV webhooks

Custom CV webhooks let you receive real-time custom model detections and camera frames from your MV cameras. Custom CV webhooks extend the capabilities of Meraki cloud webhooks.

Key concepts

Before you configure Custom CV webhooks, review the following key concepts:

1. Webhook: An HTTP callback or POST request that is triggered by specific events or data changes in your Meraki network. When an event occurs, Meraki sends an HTTP request to a customer owned specified URL (webhook receiver) with relevant data, allowing you to respond to the event programmatically.

2. Custom payload template: Webhooks in Meraki require you to provide a custom payload template in the webhook server configuration. This template defines the structure of the data you will receive in your webhook message. While creating the template, you will use Liquid template tags and filters to format the data.

3. Liquid template: Liquid is a lightweight templating language used to generate dynamic content in web pages and, in this case, in webhook payloads. In Meraki, you can use Liquid tags and filters to access and manipulate data from events.

Configuration Steps

To configure Custom CV webhooks, complete the following steps:

Configure a Webhook Service: In the Meraki dashboard, navigate to Network-wide > Configure > Alerts > Webhooks. Configure the webhook service, define the events to monitor, and set the URL where Meraki sends webhook data.

Create a Custom Payload Template: Select Add or edit webhook templates to create a custom payload template, then select Save. This template determines how Meraki formats the data sent to your webhook receiver.

Using the Add or edit webhook templates option to create custom payload templates via the dashboard UI is only available if you opt in to Early API Access at Organization > Configure > Early Access. If you do not want to opt-in, create a custom payload template using a webhook API instead.

Custom payload template example:

{
    "device": "{{ deviceSerial }}",
    "data": {{ alertData | jsonify }}
}

Configure Custom CV Webhook: Select Add an Https Receiver, then provide all necessary information, for example a name, and select the custom template. (Optional): To ensure that your webhook is properly configured, you can use the "Send test webhook" feature to send a test event to your webhook receiver URL.

Configure Webhook on Camera: To enable webhooks on the device you need to use the Custom CV Webhook API.

{\"type\":\"object\"}

The following is an example command to set these parameters, you need to substitute the parameter values with your own:

curl -s -H "Content-Type: application/json" \
    -H "Accept: application/json" \
    -H "X-Cisco-Meraki-API-Key: $MERAKI_DASHBOARD_API_KEY" \
    -d '{"enabled":true,"artifactId":"$ARTIFACT_ID","parameters":[\
   {"name":"detection_threshold","value":$DETECTION_THRESHOLD},\
     {"name":"dev_output_webhook_server","value":"$WEBHOOK_ID"},\
     {"name":"dev_output_enable_snapshot_end_time","value":"$EXPIRATION_DATE"},\
     {"name":"dev_output_json_filter","value":"{\"type\":\"object\"}"}]}'\
    -X PUT "https://api.meraki.com/api/v1/devices/$DEVICE_SERIAL/camera/customAnalytics

import requests
import json

merakiHeaders = {"X-Cisco-Meraki-API-Key": "$MERAKI_DASHBOARD_API_KEY", "Content-Type": "application/json"}

ciBody = {
    "enabled": True,
    "artifactId": "$ARTIFACT_ID",
    "parameters": [
        {
            "name": "detection_threshold",
            "value": $DETECTION_THRESHOLD
        },
        {
            "name": "dev_output_webhook_server",
            "value": $WEBHOOK_ID
        },
        {
            "name": "dev_output_enable_snapshot_end_time",
            "value": $EXPIRATION_DATE
        },
        {
            "name": "dev_output_json_filter",
            "value": "{\"type\":\"object\"}"
        }
    ]
}
res = requests.put(url=f"https://api.meraki.com/api/v1/devices/$DEVICE_SERIAL/camera/customAnalytics", headers=merakiHeaders, data=json.dumps(ciBody))

{
  "serial": "XXXX-YYYY-ZZZZ",
  "mac": "00:00:00:00:00:00",
  "data": {
    "outputs": [
      {
        "class": 0,
        "id": 1,
        "location": [
          0.422,
          0.363,
          0.508,
          0.623
        ],
        "score": 0.344
      }
    ],
    "snapshot": "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAIBAQEBAQIBAQECAgICAgQDAgICAgUEBAMEBgUGBgYFBgYGBwkIBgcJBwYGCAsICQoKCgoKBggLDAsKDAkKCgr/2wBDAQICAgICAgUDAwUKBwYHCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgr/wAARCAIcA8ADASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/9k=",
    "timestamp": 1686669655302
  }
}

Custom CV artifacts uploads are failing

1. We allow only .zip files that contain only "model.tflite" file inside. Check .zip file for any other foreign files e.g. ".DS_store" and remove them. Try uploading again.

2. Review the error message that appears during the upload. Confirm that you built the model using an approved tflite version listed in the document above, and that the model input and outputs meet the custom artifact requirements.

Custom CV models are not available for selection

Confirm your custom artifact has been successfully uploaded to your artifact store. You can observe this by either using the Dashboard API to GET all custom artifacts or by viewing the "Add or delete custom models" on the Custom CV subsection of your camera settings' Sense subtab.

Before contacting support, verify the following cases:

1. The camera detects some objects but not all objects. Refer to the Custom CV Model Performance Issues section for more information.

2. The camera is connected to an MQTT broker, but the broker does not receive any detections. Misconfigured MQTT broker settings or network restrictions can cause this issue.

Custom CV model performance issues

If there is an issue with model performance (for example, object detections are inconsistent, detections are slow, object tracking is poor), Meraki Support cannot assist with these matters directly. Consult the documentation of your Custom CV developer or provider, or contact their support directly. If you are using a service from a Cisco SolutionsPlus provider, refer to the following section.

Support for Cisco SolutionsPlus Offerings for Custom CV

The following section lists contact details for Custom CV offerings available through Cisco SolutionsPlus vendors. For model performance issues, review the vendor's documentation or contact their support team. For Custom CV service issues, review the Troubleshooting section in this article and contact Meraki Support for further assistance.