Governing AI Agents in dbt - materialization, new node type, or something else?

[poojacrahen]

Following Grace's feedback on PR #15371, I am opening this discussion to hash out the right approach before implementation.

The problem

Snowflake Cortex Agents can be created and modified directly in the Snowsight UI with no audit trail. There is currently no way to version-control, peer-review, or CI/CD-deploy an AI agent through dbt. The same governance guarantees dbt provides for tables, views, and semantic views do not extend to agents today.

The proposal

Managing AI agent definitions as dbt artifacts, versioned in git, deployed through CI/CD, visible in lineage. The key capability is ref() inside tool_resources, so agents appear downstream of their semantic views in the DAG and semantic view references are environment-aware.

Open questions

1. Materialization or new node type?

PR #15371 implemented this as a model materialization. Grace correctly notes that unlike table, view, and dynamic_table, a Cortex Agent does not create a ref-able relation in the warehouse. The precedent for non-data materializations is limited. A new node type (similar to how sources and exposures are distinct from models) might be more architecturally correct, but would require deeper changes to the DAG layer.

2. How would it inject into the DAG?

If it is a new node type rather than a materialization, how does dbt know to include it in dbt run? Does it get its own command (dbt deploy-agents)? Or does it participate in the standard node execution graph?

3. Snowflake-specific or multi-adapter?

Snowflake Cortex Agents are Snowflake-native. But the broader problem of governing AI agents as code through dbt likely applies to other platforms as AI agent objects emerge. Should the spec be designed with multi-adapter extensibility in mind from the start?

What we have seen work in production

We are exploring running this pattern as a local macro override with CREATE OR REPLACE AGENT on every dbt run to bring agents closer to development and CI/CD and treat them as first-class citizens of the DAG lineage. The governance story works in practice. The open question is where this capability belongs in the dbt architecture.

[graciegoheen]

The key capability is ref() inside tool_resources, so agents appear downstream of their semantic views in the DAG and semantic view references are environment-aware.

Can you help me understand why ref-ing agents would be useful / how this would work?

[poojacrahen]

Great question. The ref() capability matters for two reasons:

1. Environment promotion

Without ref(), you hardcode the semantic view name in the agent spec:

tool_resources:
  Analyst1:
    semantic_view: "PROD_DB.MARTS.sem_orders"

That breaks in dev or staging where the database name is different. With ref():

tool_resources:
  Analyst1:
    semantic_view: "{{ ref('sem_orders') }}"

dbt resolves this to the correct fully-qualified name for whatever target environment you are deploying to. The same agent model works in dev, staging, and prod without any changes.

2. DAG lineage and dependency enforcement

When the agent spec contains {{ ref('sem_orders') }}, dbt knows the agent depends on that semantic view. This means:

• If sem_orders fails to build, dbt will not attempt to deploy the agent
• The agent appears downstream of sem_orders in the lineage graph, so you can see the full chain from raw data through semantic views to deployed agents
• dbt run --select sem_orders+ would include the agent in the run

Without ref(), the agent is invisible to dbt's dependency graph. You could drop or rename the semantic view and the agent definition would silently break with no warning.

[graciegoheen]

Ok, so this is just about ref-ing the semantic views. But you would never actually ref the agent itself? In that case, I don't think it makes sense for this to be a model materialization.

[poojacrahen]

@graciegoheen That is a fair conclusion for today's single-agent case. You are right that nothing would currently ref() the agent itself.

The broader argument for dbt ownership is about deployment lifecycle rather than ref-ability. A Cortex Agent is a first-class Snowflake object with DDL (CREATE, ALTER, DROP), it lives in a schema, it has grants and an owner. The question is whether dbt should own the deployment of Snowflake objects that sit at the end of the DAG, even if nothing depends on them downstream.

That said, if the right answer is a new node type rather than a materialization, I get it. Happy to follow wherever the discussion leads on the right abstraction.

One thing worth keeping in mind: if Snowflake evolves their strategy from single agents to multi-agent orchestration where one agent calls another, that changes the ref-ability picture and the right abstraction here matters more.

[jeremychia]

would there be a possibility to define this as an expansion on exposures?

[poojacrahen]

That is an interesting angle. Exposures already capture "this downstream thing consumes dbt models" and auto-exposures for supported integrations like Tableau go further by generating them automatically from the BI tool's metadata. That is closer to what we need.
​
The key distinction is that a Cortex Agent is a first-class Snowflake object, just like a table, view, or dynamic table. It has DDL (CREATE, ALTER, DROP), it lives in a schema, it has an owner and grants, and it can be queried via SHOW AGENTS and GET_DDL. In that sense it is more similar to what dbt already manages as materializations than to what exposures document.
​
Exposures, including auto-exposures, are documentation and lineage artifacts. dbt does not issue DDL to create or update the downstream object based on them. For a Cortex Agent, we want dbt to own the actual deployment, meaning CREATE OR REPLACE AGENT runs as part of dbt run and the agent cannot diverge from what is in git.
​
So the question becomes: should agents be a new exposure type that also has deployment semantics, or a new node type that participates in the execution graph?

[KarthikRajashekaran]

Great thread — we’ve been wrestling with the same question.

We didn’t wait for a dbt-native agent construct because our agents sit in a different layer than our data models. dbt governs the semantic/data plane (semantic views, silver LLM outputs, tests, lineage). Cortex Agents are Snowflake config objects — orchestration instructions, tool wiring, skills, RBAC — that reference dbt-built objects but aren’t transformations themselves.

We built a package as a separate CLI and deploy agents through the same platform stack we use for everything else in Snowflake:

dbt → semantic views, LLM-backed silver/gold models, tests
Flyway → stored procedures, UDFs, and other DDL agents call as custom tools
Terraform → roles, warehouses, skill stages, service accounts
package → CREATE AGENT / MCP specs, env promotion, get-perms, deploy/fetch
That split let us ship without blocking on a new dbt node type. Agents live in our repo next to Flyway and Terraform, not inside dbt projects — one agent per user journey, wired to the smallest sufficient dbt semantic surface.

So for us it’s not “dbt vs agents” — it’s dbt for governed data, Flyway/Terraform/Package for platform + agent lifecycle

[poojacrahen]

Interesting! And the layering makes a lot of sense.

One thing I am curious about is how do you handle the dependency between dbt and the agent layer? For example if a semantic view changes in a breaking way, does your package coordinate the agent redeploy, or do those happen independently? Trying to understand whether the lineage between the two layers is explicit somewhere or managed through convention.

The reason I wanted agents inside dbt was to get that dependency enforcement through ref() so if the upstream semantic view fails, the agent deploy is blocked. But curious whether your approach solves that differently or whether it is a tradeoff you accepted.

Your point that agents are config objects rather than transformations is exactly what Grace raised about whether materialization is the right abstraction and I can see why.

[KarthikRajashekaran]

yes — tradeoff accepted, with guardrails

How CI/CD keeps prod stable

We also rely on environment promotion dev → prod (fixed branch pipeline). If a semantic view change breaks, it should fail in dev/CI first — dbt run/tests (and agent smoke tests where we have them) — before prod ever sees it. Prod stays on the last known-good dbt + agent combination until the full stack passes the promotion chain.

That’s not the same as ref() enforcement, but it’s our main production safety net while we iterate in lower envs.

One other reason we kept agents out of dbt: Cortex Agents aren’t Analyst-only. Some agents are built entirely on custom tools — our one of the Agent uses 10 Flyway-deployed stored procedures with no semantic model. Others mix Analyst + Search + procs That’s platform/agent config across Flyway + Terraform + sfmgr, not a dbt transformation.

Bottom line: dbt should feed agents (semantic views, marts, tests), not be the agent deploy system.

[marcelobour]

This discussion seems very interesting and complex. The suggestions and alternatives make a lot of sense. For Pooja's use case, having a specific dbt object seems pretty useful, and having the agent version-controlled feels healthy and almost mandatory.

But as Karthik stated, there are many other cases that do not depend on semantic models, and there could be other scenarios that wouldn't fit or wouldn't be useful to process agents in dbt.

In the same way, while the fact that the agent is a DDL is an argument in favor of having a node type in dbt, it is also composed of YAML, which makes it especially complex. This hybrid nature could actually be an argument either for or against creating a custom materialization for this specific case.

Another thing that worries me is that Snowflake's definition, format, and behavior might change, and we would have to constantly chase those changes. In a way, I think it's not general or consolidated enough yet to add it to dbt.

I hadn't read the "Expectations for dbt contributors" before, and reading this part:

"We also believe dbt as a framework should be extensible enough to "make the easy things easy, and the hard things possible". To that end, we don't believe it's appropriate for dbt to have an out-of-the-box solution for every niche problem. Users have the flexibility to achieve many custom behaviors by defining their own macros, materializations, hooks, and more. We view it as our responsibility as maintainers to decide when something should be "possible" — via macros, packages, etc. — and when something should be "easy" — built into the dbt standard."

So, instead of just sharing my personal feeling, I ask myself if this should be "easy". And I think not, especially under the argument that it is not a consolidated technology yet, none of us would be surprised if it takes a different shape next year. Also, it doesn't cover a full range of use cases and it would be mainly for Snowflake out of the box tools, which could label it as a niche problem.

After looking at the whole conversation, I feel like my comment might seem unhelpful and like a negative vote. Initially, I really engaged with the idea and felt the impulse to praise Pooja for the solution. But after paying attention to every comment and many angles of this, I'm just being honest and trying to be the most sensible.

Reading this, I also remembered the Open Semantic Interchange OSI, but looking into it a bit, it doesn't cover Agent definitions yet, if that changes, this discussion would definitely gain a lot of traction.

By no means do I think this idea should be shelved, the root concept is very interesting: dbt brought software development best practices to the data world, and Pooja's idea has the same spirit of extending that philosophy to the world of AI agents.

[ian-andriot]

I think this could be a new node type, and broader than just agents. Ultimately, these are ML/AI artifacts. They have training, testing, and inference steps, along with their own materialized state and versions.

For something downstream, whether that’s another ML model or an agent, it needs to resolve the correct version and run inference against it. That seems like a different lifecycle from a normal dbt model or materialization.