MV Sense Custom Computer Vision
Custom Computer Vision
Custom Computer Vision (Custom CV) allows users to deploy and run custom Machine Learning models directly on MV cameras to perform object detections that are tailored to their unique requirements.
This functionality is available on all second and third-generation hardware. Custom CV allows to run one custom model at a time and needs to be enabled per camera. It is available under MV Sense License, and 10 free MV Sense Licenses are included in every MV organization.
When Custom Computer Vision is enabled, the default Meraki analytics are disabled. This includes motion detection, people & vehicle detection, and audio analytics.
Custom CV is available in networks running MV 4.17+.
Uploading of new Custom CV artifacts requires Organization Administrator access.
High-Level Overview
A customer or a partner develops a custom computer vision model to detect objects tailored to the business use case, making sure that the model format is compliant with the Custom Artifact Requirements described in this document. Then, enables MV Sense license for the camera of their interest and uses Custom CV APIs or Dashboard UI interface to enable Custom CV and upload a model to the camera. Optionally sets additional parameters such as detection threshold. Once a model is successfully uploaded to the camera, if all Custom Artifact Requirements requirements are satisfied, the chosen camera starts model inference. To retrieve the detection output results from the camera MQTT protocol is used for communication between the device and the customer's, or partner's application. Follow the documentation to learn more about MQTT and how to get started.
Before starting
The term artifact is used to describe a zip archive file that contains model.tflite file and an optional configuration file.
Custom Artifact Requirements
-
The artifact should be a zip archive which extracted size is no more than 40 MB
-
The artifact should contain a tflite model file which is named exactly "model.tflite"
-
The tflite model should comply with the following interface:
Inputs
|
Index |
Type |
Description |
|
0 |
float32 |
A tensor representing an RGB image in NHWC order, i.e. taking the shape of [1, height, width, 3]. The tensor should represent a normalized image, where each value should be between -1 and 1. |
Outputs
|
Index |
Type |
Description |
|
0 |
float32 |
Locations. Multidimensional array of [N][4] floating point values between 0 and 1, the inner arrays representing bounding boxes in the form [top, left, bottom, right]. |
|
1 |
float32 |
Classes. Array of N integers (output as floating point values) each indicating the index of a class label |
|
2 |
float32 |
Scores. Array of N floating point values between 0 and 1 representing probability that a class was detected. |
|
3 |
float32 |
Number of detections. Value of N. |
Note that the model outputs need to be in the same order as specified. If you use tflite converter to produce the tflite model from a saved tensorflow model, note that tflite converter might produce models with unexpected output ordering in some cases and you will not receive MQTT messages. If you encounter such problems, please try a lower version (<=2.5) of tflite converter.
If you are not sure how to get started, try the example tflite object detection model
Artifact Packaging
-
Starter model: ssd-mobilenet-v1 - a tflite model trained on COCO dataset
-
Feel free to use any tflite model complying with the above requirements which is less than 40MB
-
Rename your tflite file as "model.tflite". This is a necessary step. A model file with other names will not be accepted.
-
Compress your model to a .zip archive, make sure the file is at the root of the archive, i.e., do not put it under any directories when zipping.
Get Started
There are two ways to get started with MV Sense Custom CV: the Meraki Dashboard UI or Custom CV APIs.
Dashboard UI
To enable MV Sense Custom CV using Meraki Dashboard, go to the desired camera's Settings --> Sense -> Custom CV section. If you don’t see the Custom CV section make sure that the Sense API is enabled.
If you haven’t uploaded any Custom CV artifacts yet:
1. Click Custom CV “Enable” button
2. Agree to the terms and conditions
3. Click add custom model
4. Upload an artifact: provide a model name and an artifact zip file
5. When upload is finished, close the modal window
6. Pick a model from the dropdown list
7. Optionally, you can set a detection threshold that filters detections based on prediction confidence score. Defaults to 0.5 (50%).
8. Click button “save” on the top right corner when artifact is successfully uploaded
If you have already uploaded an artifact:
1. Click Custom CV “Enable” button
2. Agree to the terms and conditions
3. Choose the model from the dropdown list
4. Optionally, you can set a detection threshold that filters detections based on prediction confidence score. Defaults to 0.5 (50%).
5. Click button “save” on the top right corner
Dashboard API
To get started with API please refer to the API documentation which is accessible through Dashboard -> Help -> API docs Custom CV section
Before API usage you need to have both the Dashboard API Key and Organization ID.
Getting the data
All MV cameras provide MQTT client to retrieve detection outputs. If you are not familiar with MQTT follow this documentation to learn more about protocol and how to enable it for your organization.
To receive detection outputs from camera subscribe to the topic below from your MQTT broker.
`/merakimv/<DEVICE_SERIAL>/custom_analytics`
Example MQTT message:
{
"outputs":
[{
"class":0,
"id":123,
"location":[0.1,0.2,0.4,0.3],
"score":0.7
}],
"timestamp":1646175500000
}
Outputs: An array of detected objects
Detections: Each detection represents a detected object that has its object class type, detection id, location coordinates [left, top, right, bottom] or [x0,y0,x1,y1] and probability score.
Timestamp: time & date when detections occur
Troubleshooting Custom CV
Custom CV artifacts uploads are failing
1. We allow only .zip files that contain only "model.tflite" file inside. Check .zip file for any other foreign files e.g. ".DS_store" and remove them. Try uploading again.
2. Check the error message when upload happens. Make sure that model is built using the approved tflite version listed in the document above, and model input and outputs satisfy the custom artifact requirements.
Custom CV models aren't available for selection
Confirm your custom artifact has been successfully uploaded to your artifact store. You can observe this by either using the Dashboard API to GET all custom artifacts or by viewing the "Add or delete custom models" on the Custom CV subsection of your camera settings' Sense subtab.
Custom CV artifact and configuration is saved, but the camera isn't detecting objects
Before contacting support, please verify following cases:
1. Camera can detect objects but not all - please see "Custom CV model perfomance issues" section below.
2. The camera is connected to MQTT broker, but MQTT broker does not receive any detections. A potential issue may lie in mqtt broker config settings or in network restrictions.
Custom CV model performance issues
If there is an issue with model performance (e.g. object detections are inconsistent, detections are slow, object tracking is poor), Meraki Support cannot assist with these matters directly. Please consult the documentation of your Custom CV developer/provider or contact their support directly. If you are using a service from a Cisco SolutionsPlus provider, please see the following section.
Support for Cisco SolutionsPlus Offerings for Custom CV
Below are the contact details for Custom CV offerings available through Cisco SolutionsPlus vendors. Please use review their documentation or contact their support if there are performance issues with the model. If there is an issue with the Custom CV service, please review the troubleshooting guidelines above and contact Meraki Support for further assistance.
Cogniac
Cogniac provides a simple, no-code platform for customers to easily define and deploy their own custom object detection with as few as 50 images to start. To learn more, please see their product page here.
Regions supported: America, EMEA, APJC, LATAM
· Support Email Address: Support@cogniac.co
160 W Santa Clara St, San Jose, CA 95113
· Support Phone Number:
+1 (888) 634-7483
+1 (805) 392 9976
· Support Hours:
24 X 7 (round the clock support)