Skip to main content
Cisco Meraki

MV Sense Custom Computer Vision

Documentation for Custom Computer Vision (Custom CV) that allows to deploy and run custom Machine Learning model directly on MV camera to receive objects detections beyond people & vehicle and motion detections. Here you will find hints and tips on how to get prepared and started!

Custom Computer Vision

Custom Computer Vision (Custom CV) allows users to deploy and run custom Machine Learning models directly on MV cameras to perform object detections that are tailored to their unique requirements.

This functionality is available on all second generation hardware. Custom CV allows you to run one custom model at a time and needs to be enabled per camera. It is available under MV Sense License, and 10 free MV Sense Licenses are included in every MV organization.

When Custom Computer Vision is enabled, the default Meraki analytics are disabled. This includes: motion detection, people & vehicle detection, and audio analytics.

Custom CV is available in networks running MV 4.17+.

Before starting


The term artifact is used to describe a zip archive file that contains model.tflite file and an optional configuration file.

Custom Artifact Requirements
  • The artifact should be a zip archive which extracted size is no more than 40 MB

  • The artifact should contain a tflite model file which is named exactly "model.tflite"

  • The tflite model should comply with the following interface:

Inputs

Index 

Type 

Description

0

float32

A tensor representing an RGB image in NHWC order, i.e. taking the shape of [1, height, width, 3]. The tensor should represent a normalized image, where each value should be between -1 and 1.

Outputs

Index

Type

Description

0

float32

Locations. Multidimensional array of [N][4] floating point values between 0 and 1, the inner arrays representing bounding boxes in the form [top, left, bottom, right].

1

float32

Classes. Array of N integers (output as floating point values) each indicating the index of a class label

2

float32

Scores. Array of N floating point values between 0 and 1 representing probability that a class was detected.

3

float32

Number of detections. Value of N.

 

Note that the model outputs need to be in the same order as specified. If you use tflite converter to produce the tflite model from a saved tensorflow model, note that tflite converter might produce models with unexpected output ordering in some cases and you will not receive MQTT messages. If you encounter such problems, please try a lower version (<=2.5) of tflite converter.

If you are not sure how to get started, try the example tflite object detection model

Artifact Packaging
  • Starter model: ssd-mobilenet-v1 - a tflite model trained on COCO dataset

  • Feel free to use any tflite model complying with the above requirements which is less than 40MB

  • Rename your tflite file as "model.tflite". This is a necessary step. A model file with other names will not be accepted.

  • Compress your model to a .zip archive, make sure the file is at the root of the archive, i.e., do not put it under any directories when zipping.

Get Started

There are two ways to get started with MV Sense Custom CV: the Meraki Dashboard UI or our open APIs.

Dashboard UI

To enable MV Sense Custom CV using Meraki Dashboard, go to the desired camera's Settings --> Sense -> Custom CV section. If you don’t see the Custom CV section make sure that the Sense API is enabled.

clipboard_ef78ac5b2534a5f6f86aa583c8c827729.png

If you haven’t uploaded any Custom CV artifacts yet:

1. Click Custom CV “Enable” button

2. Agree to the terms and conditions

enable_custom_cv.gif

3. Click add custom model

4. Upload an artifact: provide a model name and an artifact zip file

add_custom_cv_model.gif

5. When upload is finished, close the modal window

6. Pick a model from the dropdown list

7. Optionally, you can set a detection threshold that filters detections based on prediction confidence score. Defaults to 0.5 (50%).

8. Click button “save” on the top right corner when artifact is successfully uploaded

select_model_and_save.gif

 

If you have already uploaded an artifact:

1. Click Custom CV “Enable” button

2. Agree to the terms and conditions

3. Choose the model from the dropdown list

4. Optionally, you can set a detection threshold that filters detections based on prediction confidence score. Defaults to 0.5 (50%).

5. Click button “save” on the top right corner

enable_select_and_save.gif

 

Dashboard API

To get started with API please refer to the API documentation which is accessible through Dashboard -> Help -> API docs Custom CV section

Before API usage you need to have both the Dashboard API Key and Organization ID.

To receive detection outputs from camera subscribe to the topic below from your MQTT broker

`/merakimv/<DEVICE_SERIAL>/custom_analytics`

Example MQTT message:

{
    "outputs":
        [{
            "class":0,
            "id":123,
            "location":[0.1,0.2,0.4,0.3],
            "score":0.7
         }],
     "timestamp":1646175500000
}

Outputs: An array of detected objects

Detections: Each detection represents a detected object that has its object class type, detection id, location coordinates [left, top, right, bottom] or [x0,y0,x1,y1] and probability score.

Timestamp: time & date when detections occur

  • Was this article helpful?