MV Sense Custom Computer Vision

Before starting

The term artifact is used to describe a zip archive file that contains model.tflite file and an optional configuration file.

Custom Artifact Requirements

• The artifact should be a zip archive which extracted size is no more than 40 MB

• The artifact should contain a tflite model file which is named exactly "model.tflite"

• The tflite model should comply with the following interface:

Inputs

Outputs

If you are not sure how to get started, try the example tflite object detection model

Artifact Packaging

• Starter model: ssd-mobilenet-v1 - a tflite model trained on COCO dataset

• Feel free to use any tflite model complying with the above requirements which is less than 40MB

• Rename your tflite file as "model.tflite". This is a necessary step. A model file with other names will not be accepted.

• Compress your model to a .zip archive, make sure the file is at the root of the archive, i.e., do not put it under any directories when zipping.

Get Started

There are two ways to get started with MV Sense Custom CV: the Meraki Dashboard UI or Custom CV APIs.

Dashboard UI

To enable MV Sense Custom CV using Meraki Dashboard, go to the desired camera's Settings --> Sense -> Custom CV section. If you don’t see the Custom CV section make sure that the Sense API is enabled.

If you haven’t uploaded any Custom CV artifacts yet:

1. Click Custom CV “Enable” button

2. Agree to the terms and conditions

3. Click add custom model

4. Upload an artifact: provide a model name and an artifact zip file

5. When upload is finished, close the modal window

6. Pick a model from the dropdown list

7. Optionally, you can set a detection threshold that filters detections based on prediction confidence score. Defaults to 0.5 (50%).

8. Click button “save” on the top right corner when artifact is successfully uploaded

If you have already uploaded an artifact:

1. Click Custom CV “Enable” button

2. Agree to the terms and conditions

3. Choose the model from the dropdown list

4. Optionally, you can set a detection threshold that filters detections based on prediction confidence score. Defaults to 0.5 (50%).

5. Click button “save” on the top right corner

Dashboard API

To get started with API please refer to the API documentation which is accessible through Dashboard -> Help -> API docs Custom CV section

Before API usage you need to have both the Dashboard API Key and Organization ID.

Getting the data

`/merakimv/<DEVICE_SERIAL>/custom_analytics`

{
    "outputs":
        [{
            "class":0,
            "id":123,
            "location":[0.1,0.2,0.4,0.3],
            "score":0.7
         }],
     "timestamp":1646175500000
}

{
    "device": "{{ deviceSerial }}",
    "data": {{ alertData | jsonify }}
}

{\"type\":\"object\"}

curl -s -H "Content-Type: application/json" \
    -H "Accept: application/json" \
    -H "X-Cisco-Meraki-API-Key: $MERAKI_DASHBOARD_API_KEY" \
    -d '{"enabled":true,"artifactId":"$ARTIFACT_ID","parameters":[\
   {"name":"detection_threshold","value":$DETECTION_THRESHOLD},\
     {"name":"dev_output_webhook_server","value":"$WEBHOOK_ID"},\
     {"name":"dev_output_enable_snapshot_end_time","value":"$EXPIRATION_DATE"},\
     {"name":"dev_output_json_filter","value":"{\"type\":\"object\"}"}]}'\
    -X PUT "https://api.meraki.com/api/v1/devices/$DEVICE_SERIAL/camera/customAnalytics

import requests
import json

merakiHeaders = {"X-Cisco-Meraki-API-Key": "$MERAKI_DASHBOARD_API_KEY", "Content-Type": "application/json"}

ciBody = {
    "enabled": True,
    "artifactId": "$ARTIFACT_ID",
    "parameters": [
        {
            "name": "detection_threshold",
            "value": $DETECTION_THRESHOLD
        },
        {
            "name": "dev_output_webhook_server",
            "value": $WEBHOOK_ID
        },
        {
            "name": "dev_output_enable_snapshot_end_time",
            "value": $EXPIRATION_DATE
        },
        {
            "name": "dev_output_json_filter",
            "value": "{\"type\":\"object\"}"
        }
    ]
}
res = requests.put(url=f"https://api.meraki.com/api/v1/devices/$DEVICE_SERIAL/camera/customAnalytics", headers=merakiHeaders, data=json.dumps(ciBody))

{
  "serial": "XXXX-YYYY-ZZZZ",
  "mac": "00:00:00:00:00:00",
  "data": {
    "outputs": [
      {
        "class": 0,
        "id": 1,
        "location": [
          0.422,
          0.363,
          0.508,
          0.623
        ],
        "score": 0.344
      }
    ],
    "snapshot": "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAIBAQEBAQIBAQECAgICAgQDAgICAgUEBAMEBgUGBgYFBgYGBwkIBgcJBwYGCAsICQoKCgoKBggLDAsKDAkKCgr/2wBDAQICAgICAgUDAwUKBwYHCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgr/wAARCAIcA8ADASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/9k=",
    "timestamp": 1686669655302
  }
}

Custom CV artifacts uploads are failing

1. We allow only .zip files that contain only "model.tflite" file inside. Check .zip file for any other foreign files e.g. ".DS_store" and remove them. Try uploading again.

2. Check the error message when upload happens. Make sure that model is built using the approved tflite version listed in the document above, and model input and outputs satisfy the custom artifact requirements.

Custom CV models aren't available for selection

Confirm your custom artifact has been successfully uploaded to your artifact store. You can observe this by either using the Dashboard API to GET all custom artifacts or by viewing the "Add or delete custom models" on the Custom CV subsection of your camera settings' Sense subtab.

Custom CV artifact and configuration is saved, but the camera isn't detecting objects

Before contacting support, please verify following cases:

1. Camera can detect objects but not all - please see "Custom CV model perfomance issues" section below.

2. The camera is connected to MQTT broker, but MQTT broker does not receive any detections. A potential issue may lie in mqtt broker config settings or in network restrictions.

Custom CV model performance issues

If there is an issue with model performance (e.g. object detections are inconsistent, detections are slow, object tracking is poor), Meraki Support cannot assist with these matters directly. Please consult the documentation of your Custom CV developer/provider or contact their support directly. If you are using a service from a Cisco SolutionsPlus provider, please see the following section.

Cogniac

Cogniac provides a simple, no-code platform for customers to easily define and deploy their own custom object detection with as few as 50 images to start. To learn more, please see their product page here.

Regions supported: America, EMEA, APJC, LATAM

·   Support Email Address: Support@cogniac.co

    160 W Santa Clara St, San Jose, CA 95113

·   Support Phone Number:

     +1 (888) 634-7483

     +1 (805) 392 9976

·   Support Hours:

     24 X 7 (round the clock support)