DOCS-4341: Update run inference

Author: npentrel

assets/tutorials/data-management/tensor-output.png

@@ -15,120 +15,129 @@ aliases:
   - /ml/vision/segmentation/
   - /ml/vision/
   - /get-started/quickstarts/detect-people/
-description: "Run inference on a model with a vision service or an SDK."
+description: "Run machine learning inference locally on your robot or remotely in the cloud using vision services, ML model services, or SDKs."
+date: "2025-09-12"
 ---
 
 Inference is the process of generating output from a machine learning (ML) model.
-With Viam, you can run inference to generate the following kinds of output:
 
-- object detection (using bounding boxes)
-- classification (using tags)
-
-You can run inference locally on a Viam machine, or remotely in the Viam cloud.
+You can run inference [locally on a Viam machine](#machine-inference), or [remotely in the Viam cloud](#cloud-inference).
 
 ## Machine inference
 
-You can use `viam-server` to deploy and run ML models directly on your machines.
-
-You can run inference on your machine in the following ways:
-
-- with a vision service
-- manually in application logic with an SDK
+When you have [deployed an ML model](/data-ai/ai/deploy/) on your machine, you can run inference on your machine [directly with the ML model service](#using-an-ml-model-service-directly) or [using a vision service](#using-a-vision-service) that interprets the inferences.
 
 Entry-level devices such as the Raspberry Pi 4 can run small ML models, such as TensorFlow Lite (TFLite).
-More powerful hardware, including the Jetson Xavier or Raspberry Pi 5 with an AI HAT+, can process larger AI models, including TensorFlow and ONNX.
+More powerful hardware, including the Jetson Xavier or Raspberry Pi 5 with an AI HAT+, can process larger models, including TensorFlow and ONNX.
+If your hardware does not support the model you want to run, see [Cloud inference](#cloud-inference).
+
+### Using an ML model service directly
 
 {{< tabs >}}
-{{% tab name="Vision service" %}}
+{{% tab name="Web UI" %}}
 
-Vision services apply an ML model to a stream of images from a camera to generate bounding boxes or classifications.
+1. Visit your machine's **CONFIGURE** or **CONTROL** page.
+1. Expand the **TEST** area of the ML model service panel to view the tensor output.
 
-{{<resources_svc api="rdk:service:vision" type="vision">}}
+{{< imgproc src="/tutorials/data-management/tensor-output.png" alt="Example tensor output" resize="x1000" class="shadow imgzoom fill" >}}
+
+{{% /tab %}}
+{{% tab name="Python" %}}
+
+The following code passes an image to an ML model service, and uses the [`Infer`](/dev/reference/apis/services/ml/#infer) method to make inferences:
+
+{{< read-code-snippet file="/static/include/examples-generated/run-inference.snippet.run-inference.py" lang="py" class="line-numbers linkable-line-numbers" data-line="82-85" >}}
+
+{{% /tab %}}
+{{% tab name="Go" %}}
+
+The following code passes an image to an ML model service, and uses the [`Infer`](/dev/reference/apis/services/ml/#infer) method to make inferences:
+
+{{< read-code-snippet file="/static/include/examples-generated/run-inference.snippet.run-inference.go" lang="go" class="line-numbers linkable-line-numbers" data-line="166-171" >}}
+
+{{% /tab %}}
+{{< /tabs >}}
+
+### Using a vision service
+
+Vision services apply an ML model to a stream of images from a camera to:
 
-{{% alert title="Tip" color="tip" %}}
-Some vision services include their own ML models, and thus do not require a deployed ML model.
-If your vision service does not include an ML model, you must [deploy an ML model to your machine](/data-ai/ai/deploy/) to use that service.
-{{% /alert %}}
+- detect objects (using bounding boxes)
+- classify (using tags)
 
 To use a vision service:
 
-1. Visit your machine's **CONFIGURE** page.
+1. Navigate to your machine's **CONFIGURE** page.
 1. Click the **+** icon next to your main machine part and select **Component or service**.
-1. Type in the name of the service and select a vision service.
-1. If your vision service does not include an ML model, [deploy an ML model to your machine](/data-ai/ai/deploy/) to use that service.
-1. Configure the service based on your use case.
-1. To view the deployed vision service, use the live detection feed.
-   The feed shows an overlay of detected objects or classifications on top of a live camera feed.
-   On the **CONFIGURE** or **CONTROL** pages for your machine, expand the **Test** area of the service panel to view the feed.
+1. Select a vision service.
+1. Configure your vision service.
+   If your vision service does not include an ML model, you need to [deploy an ML model to your machine](/data-ai/ai/deploy/) and select it when configuring your vision service.
 
-   {{< imgproc src="/tutorials/data-management/blue-star.png" alt="Detected blue star" resize="x200" class="shadow" >}}
-   {{< imgproc src="/tutorials/filtered-camera-module/viam-figure-preview.png" alt="Detection of a viam figure with a confidence score of 0.97" resize="x200" class="shadow" >}}
+{{% expand "Click to search available vision services" %}}
 
-For instance, you could use [`viam:vision:mlmodel`](/operate/reference/services/vision/mlmodel/) with the `EfficientDet-COCO` ML model to detect a variety of objects, including people, bicycles, and apples, in a camera feed.
-
-Alternatively, you could use [`viam-soleng:vision:openalpr`](https://app.viam.com/module/viam-soleng/viamalpr) to detect license plates in images.
-Since this service includes its own ML model, there is no need to configure a separate ML model.
+{{<resources_svc api="rdk:service:vision" type="vision">}}
 
-After adding a vision service, you can use a vision service API method with a classifier or a detector to get inferences programmatically.
-For more information, see the APIs for [ML Model](/dev/reference/apis/services/ml/) and [Vision](/dev/reference/apis/services/vision/).
+{{% /expand%}}
 
-{{% /tab %}}
-{{% tab name="SDK" %}}
+{{% expand "Click to view example vision services" %}}
 
-With the Viam SDK, you can pass image data to an ML model service, read the output annotations, and react to output in your own code.
-Use the [`Infer`](/dev/reference/apis/services/ml/#infer) method of the ML Model API to make inferences.
+<!-- prettier-ignore -->
+| Example | Description |
+| ------- | ----------- |
+| Detect a variety of objects | Use the [`viam:vision:mlmodel`](/operate/reference/services/vision/mlmodel/) vision service with the `EfficientDet-COCO` ML model to detect a variety of objects, including people, bicycles, and apples, in a camera feed. |
+| Detect license plates | Use the [`viam-soleng:vision:openalpr`](https://app.viam.com/module/viam-soleng/viamalpr) vision service to detect license plates in images. This service includes its own ML model. |
 
-For example:
+{{% /expand%}}
 
 {{< tabs >}}
-{{% tab name="Python" %}}
+{{% tab name="Web UI" %}}
 
-```python {class="line-numbers linkable-line-numbers"}
-import numpy as np
+1. Visit your machine's **CONFIGURE** or **CONTROL** page.
+1. Expand the **TEST** area of the vision service panel.
 
-my_mlmodel = MLModelClient.from_robot(robot=machine, name="my_mlmodel_service")
+   The feed shows an overlay of detected objects or classifications on top of a live camera feed.
+
+   {{< imgproc src="/tutorials/data-management/blue-star.png" alt="Detected blue star" resize="x200" class="shadow" >}}
+   {{< imgproc src="/tutorials/filtered-camera-module/viam-figure-preview.png" alt="Detection of a viam figure with a confidence score of 0.97" resize="x200" class="shadow" >}}
 
-image_data = np.zeros((1, 384, 384, 3), dtype=np.uint8)
+{{% /tab %}}
+{{% tab name="Python" %}}
 
-# Create the input tensors dictionary
-input_tensors = {
-    "image": image_data
-}
+The following code passes an image from a camera to a vision service and uses the [`GetClassifications`](/dev/reference/apis/services/vision/#GetClassifications) method:
 
-output_tensors = await my_mlmodel.infer(input_tensors)
-```
+{{< read-code-snippet file="/static/include/examples-generated/run-vision-service.snippet.run-vision-service.py" lang="py" class="line-numbers linkable-line-numbers" data-line="36-37" >}}
 
 {{% /tab %}}
 {{% tab name="Go" %}}
 
-```go {class="line-numbers linkable-line-numbers"}
-input_tensors := ml.Tensors{"0": tensor.New(tensor.WithShape(1, 2, 3), tensor.WithBacking([]int{1, 2, 3, 4, 5, 6}))}
+The following code passes an image from a camera to a vision service and uses the [`GetClassifications`](/dev/reference/apis/services/vision/#GetClassifications) method:
 
-output_tensors, err := myMLModel.Infer(context.Background(), input_tensors)
-```
+{{< read-code-snippet file="/static/include/examples-generated/run-vision-service.snippet.run-vision-service.go" lang="go" class="line-numbers linkable-line-numbers" data-line="66-69" >}}
 
 {{% /tab %}}
-{{< /tabs >}}
+{{% tab name="TypeScript" %}}
+
+The following code passes an image from a camera to a vision service and uses the [`GetClassifications`](/dev/reference/apis/services/vision/#GetClassifications) method:
+
+{{< read-code-snippet file="/static/include/examples-generated/run-vision-service.snippet.run-vision-service.ts" lang="ts" class="line-numbers linkable-line-numbers" data-line="32-38" >}}
 
 {{% /tab %}}
 {{< /tabs >}}
 
 ## Cloud inference
 
 Cloud inference enables you to run machine learning models in the Viam cloud, instead of on a local machine.
-Cloud inference often provides more computing power than edge devices, so you can benefit from:
+Cloud inference provides more computing power than edge devices, enabling you to run more computationally-intensive models or achieve faster inference times.
 
-- larger, more accurate models
-- faster inference times
+You can run cloud inference using any TensorFlow and TensorFlow Lite model in the Viam registry, including unlisted models owned by or shared with you.
 
-You can run cloud inference using any TensorFlow and TensorFlow Lite model in the Viam registry, including private models owned by or shared with your organization.
-
-To run cloud inference, you must pass
+To run cloud inference, you must pass the following:
 
 - the binary data ID and organization of the data you want to run inference on
 - the name, version, and organization of the model you want to use for inference
 
-The [`viam infer`](/dev/tools/cli/#infer) CLI command runs inference in the cloud on a piece of data using the specified ML model:
+You can obtain the binary data ID from the [**DATA** tab](https://app.viam.com/data/view) and the organization ID by running the CLI command `viam org list`.
+You can find the model information on the [**MODELS** tab](https://app.viam.com/models).
 
 ```sh {class="command-line" data-prompt="$" data-output="2-18"}
 viam infer --binary-data-id <binary-data-id> --model-name <model-name> --model-org-id <org-id-that-owns-model> --model-version "2025-04-14T16-38-25" --org-id <org-id-that-executes-inference>
@@ -151,5 +160,7 @@ Bounding Box Format: [x_min, y_min, x_max, y_max]
   No annotations.
 ```
 
-`infer` returns a list of detected classes or bounding boxes depending on the output of the ML model you specified, as well as a list of confidence values for those classes or boxes.
-This method returns bounding box output using proportional coordinates between 0 and 1, with the origin `(0, 0)` in the top left of the image and `(1, 1)` in the bottom right.
+The command returns a list of detected classes or bounding boxes depending on the output of the ML model you specified, as well as a list of confidence values for those classes or boxes.
+The bounding box output uses proportional coordinates between 0 and 1, with the origin `(0, 0)` in the top left of the image and `(1, 1)` in the bottom right.
+
+For more information, see [`viam infer`](/dev/tools/cli/#infer).

docs/data-ai/ai/run-inference.md

@@ -31,24 +31,24 @@ Triggers support the following attributes:
 The following template demonstrates the form of a JSON configuration for a trigger that alerts when data syncs to the Viam cloud:
 
 ```json {class="line-numbers linkable-line-numbers"}
-  "triggers": [
-    {
-      "name": "<trigger name>",
-      "event": {
-        "type": "part_data_ingested",
-        "data_ingested": {
-          "data_types": ["binary", "tabular", "file", "unspecified"]
-        }
-      },
-      "notifications": [
-        {
-          "type": "<webhook|email>",
-          "value": "<webhook URL or email address>",
-          "seconds_between_notifications": <number of seconds>
-        }
-      ]
-    }
-  ]
+"triggers": [
+  {
+    "name": "<trigger name>",
+    "event": {
+      "type": "part_data_ingested",
+      "data_ingested": {
+        "data_types": ["binary", "tabular", "file", "unspecified"]
+      }
+    },
+    "notifications": [
+      {
+        "type": "<webhook|email>",
+        "value": "<webhook URL or email address>",
+        "seconds_between_notifications": <number of seconds>
+      }
+    ]
+  }
+]
 ```
 
 ## Conditional attributes