[Walkthrough] Run the Relay Observability template

[kic635]

Goal

This walkthrough shows how to run the Relay Observability template with AgentSeek Desktop, open the application, and inspect the resulting traces in Trace Center.

What is NVIDIA NeMo Relay?

Before starting, see the official documentation:

• NeMo Relay Overview
• NeMo Relay GitHub repository
• NeMo Relay Observability documentation
• OpenInference output
• OpenTelemetry output

NeMo Relay is not the trace UI and it is not the database. It is an agent runtime and instrumentation layer. It observes the lifecycle of an agent run, including LLM calls, tool calls, middleware, subagents, errors, and nested scopes. It then converts the captured lifecycle events into standard or Relay-specific observability formats and exports them to downstream systems.

In this template, the roles are:

Agent application
        ↓
NeMo Relay: capture scopes, lifecycle events, and tool/LLM activity
        ↓
ATOF / ATIF / OpenInference / OpenTelemetry
        ↓
Phoenix Trace Center
        ↓
OceanBase SeekDB persistence

Relay is therefore the instrumentation and export layer. Phoenix is the trace inspection layer, and SeekDB is the persistence layer.

Step 1: Install AgentSeek Desktop

Download and install https://github.com/agentseek-ai/agentseekd/releases/tag/v0.1.0

Step 2: Select SeekDB

After installation, select the SeekDB option.

Step 3: Select the template

Open the template list and select the Relay Observability template.

If the template is not listed, click the pencil icon to edit the template source.

Set the template source to:

https://github.com/agentseek-ai/agentseek-templates/tree/main

Step 4: Install the template

Click Install.

Configure the following environment variables as shown in the screenshot:

• BUB_API_KEY
• BUB_API_BASE
• TAVILY_API_KEY

Step 5: Wait for installation

Wait for the installation to complete. You can open the Logs Center to monitor its progress.

Step 6: Start the instance

After installation completes, open the instance list and click the action button.

Step 7: Open the application and inspect traces

After the instance starts, open its details page.

Click Open Application to open the frontend and start a conversation.

After the conversation, return to the instance details page and click the Trace ID entry shown in the screenshot to open Trace Center and inspect the generated traces.

Understanding ATOF and ATIF

The Trace Center view is backed by data captured and exported by NeMo Relay. The following two formats are important:

• ATOF event format
• ATOF observability documentation
• ATIF observability documentation

ATOF: Agent Trajectory Observability Format

ATOF is the raw, canonical event stream produced from the agent lifecycle. It records what actually happened during execution: scopes, turns, LLM calls, tool calls, subagent activity, errors, and lifecycle transitions. In the template, ATOF can be retained as a JSONL audit/debug stream.

ATOF is useful for:

• verifying whether Relay captured the real execution;
• debugging missing or malformed events;
• keeping a raw audit record before further projection or export;
• diagnosing whether a problem happened in the application, Relay, or Phoenix path.

ATIF: Agent Trajectory Interchange Format

ATIF is a normalized agent trajectory representation for analysis and evaluation workflows. It organizes an agent run into a portable trajectory that can be consumed by downstream tools, evaluation pipelines, and inspection workflows.

The distinction is:

Agent execution
      ↓
ATOF: raw canonical observability events
      ↓
ATIF: normalized agent trajectory for analysis/evaluation
      ↓
OpenInference / OpenTelemetry: interoperable trace projections
      ↓
Phoenix and other observability backends

What Relay does with ATOF and ATIF

Relay is the component that captures the runtime lifecycle and produces these outputs. It does not replace ATOF or ATIF; it is the runtime that generates and projects the captured events into those formats.

Summary: the role of NeMo Relay

NeMo Relay is the execution-observability layer between the agent application and downstream systems. It captures the run once, preserves parent-child scope and lifecycle context, and can project the same captured execution into ATOF, ATIF, OpenInference, and OpenTelemetry outputs.

In this walkthrough:

• Relay captures the Agent/LLM/Tool execution;
• ATOF provides the raw audit and debugging event stream;
• ATIF provides a normalized trajectory for analysis and evaluation;
• OpenInference/OpenTelemetry provide interoperable trace exports;
• Phoenix provides the Trace Center UI;
• SeekDB persists the Phoenix trace data.

The key point is that Phoenix is only the inspection and persistence surface. NeMo Relay is what observes and structures the actual agent execution before it reaches Phoenix.