MV Sense Custom Computer Vision

MQTT

All MV cameras provide MQTT client to retrieve detection outputs. If you are not familiar with MQTT follow this documentation to learn more about protocol and how to enable it for your organization.

To receive detection outputs from camera subscribe to the topic below from your MQTT broker. 

`/merakimv/<DEVICE_SERIAL>/custom_analytics`

Example MQTT message:

{
    "outputs":
        [{
            "class":0,
            "id":123,
            "location":[0.1,0.2,0.4,0.3],
            "score":0.7
         }],
     "timestamp":1646175500000
}

Outputs: An array of detected objects

Detections: Each detection represents a detected object that has its object class type, detection id, location coordinates [left, top, right, bottom] or [x0,y0,x1,y1] and probability score.

Timestamp: time & date when detections occur

Custom CV Webhooks

Custom CV Webhooks are a powerful feature that allow you to receive real-time custom model detections and/or frames from your Meraki MV cameras. Custom CV Webhooks extend Meraki Cloud webhooks capabilities.

Key Concepts

Before diving into the configuration details, let's cover some key concepts related to Meraki webhooks:

1. Webhook: A webhook is an HTTP callback or POST request that is triggered by specific events or data changes in your Meraki network. When an event occurs, Meraki sends an HTTP request to a customer owned specified URL (webhook receiver) with relevant data, allowing you to respond to the event programmatically.

2. Custom Payload Template: Webhooks in Meraki require you to provide a custom payload template in the webhook server configuration. This template defines the structure of the data you will receive in your webhook message. While creating the template, you will use Liquid template tags and filters to format the data.

3. Liquid Template: Liquid is a lightweight templating language used to generate dynamic content in web pages and, in this case, in webhook payloads. In Meraki, you can use Liquid tags and filters to access and manipulate data from events.

Configuration Steps

To get started with Custom CV webhooks, follow these steps:

Configure a Webhook Service: In your Meraki dashboard, go to the "Webhook" section under "Network-wide > Alerts." Here, you'll configure the webhook service, define the events you want to monitor, and set up the URL where Meraki should send webhook data. 

Create a Custom Payload Template: Click on "Add or edit webhook templates" to create a custom payload template.  Configure the template per your needs and hit "Save". This template will determine how the data is formatted when it's sent to your webhook receiver. 

Custom Payload Template Example:

{
    "device": "{{ deviceSerial }}",
    "data": {{ alertData | jsonify }}
}

Configure Custom CV Webhook: Click “Add an Https Receiver”. Provide all necessary information e.g. name and select the custom template. (Optional): To ensure that your webhook is properly configured, you can use the "Send test webhook" feature to send a test event to your webhook receiver URL.

Configure Webhook on Camera: To enable webhooks on the device you need to use Custom CV Webhook API [A LINK TO THE CUSTOM CV WEBHOOK API DOC].

{\"type\":\"object\"}

The following is an example command to set these parameters, note that you need to substitute the parameter values with your own:

cURL

curl -s -H "Content-Type: application/json" \
    -H "Accept: application/json" \
    -H "X-Cisco-Meraki-API-Key: $MERAKI_DASHBOARD_API_KEY" \
    -d '{"enabled":true,"artifactId":"$ARTIFACT_ID","parameters":[\
   {"name":"detection_threshold","value":$DETECTION_THRESHOLD},\
     {"name":"dev_output_webhook_server","value":"$WEBHOOK_ID"},\
     {"name":"dev_output_enable_snapshot_end_time","value":"$EXPIRATION_DATE"},\
     {"name":"dev_output_json_filter","value":"{\"type\":\"object\"}"}]}'\
    -X PUT "https://api.meraki.com/api/v1/devices/$DEVICE_SERIAL/camera/customAnalytics

Python Requests

import requests
import json

merakiHeaders = {"X-Cisco-Meraki-API-Key": "$MERAKI_DASHBOARD_API_KEY", "Content-Type": "application/json"}

ciBody = {
    "enabled": True,
    "artifactId": "$ARTIFACT_ID",
    "parameters": [
        {
            "name": "detection_threshold",
            "value": $DETECTION_THRESHOLD
        },
        {
            "name": "dev_output_webhook_server",
            "value": $WEBHOOK_ID
        },
        {
            "name": "dev_output_enable_snapshot_end_time",
            "value": $EXPIRATION_DATE
        },
        {
            "name": "dev_output_json_filter",
            "value": "{\"type\":\"object\"}"
        }
    ]
}
res = requests.put(url=f"https://api.meraki.com/api/v1/devices/$DEVICE_SERIAL/camera/customAnalytics", headers=merakiHeaders, data=json.dumps(ciBody))

Example of payload body received by the server (based on the template above)

{
  "serial": "XXXX-YYYY-ZZZZ",
  "mac": "00:00:00:00:00:00",
  "data": {
    "outputs": [
      {
        "class": 0,
        "id": 1,
        "location": [
          0.422,
          0.363,
          0.508,
          0.623
        ],
        "score": 0.344
      }
    ],
    "snapshot": "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAIBAQEBAQIBAQECAgICAgQDAgICAgUEBAMEBgUGBgYFBgYGBwkIBgcJBwYGCAsICQoKCgoKBggLDAsKDAkKCgr/2wBDAQICAgICAgUDAwUKBwYHCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgr/wAARCAIcA8ADASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/9k=",
    "timestamp": 1686669655302
  }
}