Windows ML ROS Node
The AI platform in Windows 10 enables developers to use pre-trained machine learning models in their Apps on Windows devices. This offers developers a number of benefits:
- Low latency, real-time results. Windows can perform AI evaluation tasks using the local processing capabilities of the PC, enabling real-time analysis of large local data such as images and video. Results can be delivered quickly and efficiently for use in performance intensive workloads like game engines, or background tasks such as indexing for search.
- Reduced operational costs. Together with Microsoft’s Cloud AI platform, developers can build affordable, end-to-end AI solutions that combine training models in Azure with deployment to Windows devices for evaluation. Significant savings can be realized by reducing or eliminating costs associated with bandwidth due to ingestion of large data sets, such as camera footage or sensor telemetry. Complex workloads can be processed in real-time on the edge, with minimal sample data sent to the cloud for improved training on observations.
- Flexibility. Developers can choose to perform AI tasks on device or in the cloud based on what their customers & scenarios need. AI processing can happen on the device if it becomes disconnected, or in scenarios where data cannot be sent to the cloud due to cost, size, policy or customer preference.
- Install Visual Studio 2019 with UWP development
- ROS melodic for Windows
The WinML ROS Node is distrubted as source. To consume it in your robot, clone the winml_tracker sources into your workspace.
mkdir c:\workspace\winml_demo\src cd c:\workspace\winml_demo\src catkin_init_workspace git clone https://github.com/ms-iot/winml_tracker
Find or construct a suitible onnx model. For example, you can download TinyYOLO from the Onnx Model Zoo.
choco upgrade curl curl https://onnxzoo.blob.core.windows.net/models/opset_8/tiny_yolov2/tiny_yolov2.tar.gz --output tiny_yolov2.tar.gz tar -xf tiny_yolov2.tar.gz
Create a Launch file, which references the model.onnx file:
<launch> <arg name="onnx_model_path_arg" default="$(find winml_tracker)/testdata/model.onnx"/> <node pkg="winml_tracker" type="winml_tracker_node" name="winml_tracker" output="screen"> <param name="onnx_model_path" value="$(arg onnx_model_path_arg)"/> <param name="confidence" value="0.5"/> <param name="tensor_width" value="416"/> <param name="tensor_height" value="416"/> <param name="tracker_type" value="yolo"/> <param name="image_processing" value="resize"/> <param name="debug" value="true"/> <param name="image_topic" value="/cv_camera/image_raw" /> </node> <!-- NOTE: The image properties need to be valid for the camera, or the node will auto select the closest values --> <node pkg="cv_camera" type="cv_camera_node" name="cv_camera" output="screen"> <param name="rate" type="double" value="5.0"/> <param name="image_width" type="double" value="640"/> <param name="image_height" type="double" value="480"/> </node> <node pkg="tf" type="static_transform_publisher" name="winml_link" args="0 -0.02 0 0 0 0 map base_link 100" /> </launch>
While 'Pose' processing is enabled, the service required to generate the model has not been published as of October 2020
|onnx_model_path||Path to the model.onnx file|
|confidence||Minimum confidence before publishing an event. 0 to 1|
|tensor_width||The Width of the input to the model.|
|tensor_height||The Height of the input to the model.|
|tracker_type||Currently enabled -
|image_topic||The image topic to subscribe to|
|label||used to filter the found object to a specific label|
|mesh_rotation||The orientation of the mesh when debug rendering pose|
|mesh_scale||The scale of the mesh when debug rendering pose|
|mesh_resource||The mesh used for debug rendering pose|
|model_bounds||9 coordinates used to perform the point in perspective caluclation for pose|
|calibration||Path to the OpenCV calibration file for point in persective|
WinML subscribes to the topic listed in the
image_topic property, or
The WinML Publishes the following topics:
Outputs an image with highlighing if the debug property is set
An array of
visualization_msgs::Marker for found objects
A single instance of the DetectedObjectPose message, which is output when tracker_type is set to pose.