Why this is not working?

[sundargautam18]

Title: Tool calling not working with TanStack AI + Ollama (LLaMA 3) — tool never triggers

Category: Q&A

---

Problem

I’m using TanStack AI + Ollama + Server Streaming with a custom tool (get_post).
The UI streams messages, but the tool never gets called.

Instead of calling the tool, the model didn't send any thing but i remove it responds

---

My Setup

Server (route.ts)

const stream = chat({
  adapter,
  messages,
  model: "llama3",
  conversationId,
  tools: [
    getPostServer, // ← I think this might be wrong?
  ],
});

Client (chat.tsx)

const { messages, sendMessage } = useChat({
  connection: fetchServerSentEvents("/api/chat"),
});

Tool Definition

export const getPostDef = toolDefinition({
  name: "get_post",
  inputSchema: z.object({
    id: z.number(),
  }),
});

Tool Server Handler

export const getPostServer = getPostDef.server(async ({ id }) => {
  const res = await fetch(`https://jsonplaceholder.typicode.com/posts/${id}`);
  return await res.json();
});

---

What I Expect

When I type:

get post 1

The model should produce:

• a tool-call part
• TanStack should call get_post
• Then return tool result

---

What Actually Happens

• The model never calls the tool
• It just answers with a normal text response like:

Questions

1. Is Ollama LLaMA 3 compatible with TanStack's automatic tool calling?

2. Is there an example of a working setup with:

   • @tanstack/ai
   • @tanstack/ai-react
   • Ollama
   • SSE streaming
   • Tools enabled

---

Additional Notes

• Streaming works fine.
• Messages appear normally.
• getPostServer is never invoked (no log output).
• I’m using Next.js API route.

---

Thanks!

Any help or examples would be appreciated.

[AlemTuzlak]

llama 3 doesn't support tools, maybe try using 3.1 and it should work

[harry-whorlow]

As @AlemTuzlak has mentioned llama 3 doesn’t support tools, I have an open pr that should add more thorough Type safety (as well a bunch more models) that will warn you if you try to pass options that the model doesn’t support.

This is something I'm still trying to figure out, but from what I can tell the Ollama api swallows configuration that the models do not consume. So it's taken me some time to figure it out 😅