Native json output support for agents

[rashadism]

Description

I've been playing around with fantasy recently and really loving it!

Is there currently a way to get structured output from agents similar to object.Generate[T] for models? I couldn't find one in the docs or code.

I got it working with a tool + HasToolCall stop condition, but couldn't figure out how to get json mode working with agents.

If it is not supported yet, I'd suggest something like follows:

Current interfaces

  type LanguageModel interface {
      Generate(context.Context, Call) (*Response, error)
      Stream(context.Context, Call) (StreamResponse, error)
      GenerateObject(context.Context, ObjectCall) (*ObjectResponse, error)
      StreamObject(context.Context, ObjectCall) (ObjectStreamResponse, error)
      Provider() string
      Model() string
  }

  type Agent interface {
      Generate(context.Context, AgentCall) (*AgentResult, error)
      Stream(context.Context, AgentStreamCall) (*AgentResult, error)
  }

Suggestion: Add ObjectGenerator interface

  type ObjectGenerator interface {
      GenerateObject(context.Context, ObjectCall) (*ObjectResponse, error)
      StreamObject(context.Context, ObjectCall) (ObjectStreamResponse, error)
  }

Have Agent implement this interface. The agent would resolve the mode from its underlying model's provider(Or should this be configurable for both agents and models?)

This would allow object.Generate[T] to work seamlessly with both agents and models

  // Works with model
  result, _ := object.Generate[Analysis](ctx, model, fantasy.ObjectCall{...})

  // Same API works with agent  
  result, _ := object.Generate[Analysis](ctx, agent, fantasy.ObjectCall{...})

Would love to hear on this, and will be happy to send a PR if this looks good and aligns with the project direction

[crazybolillo]

Taking a look at #64, it looks like the next step is for ObjectCall to support tools, similar to how Call supports Tools.

[crazybolillo]

Got this working. I added Tools and ToolChoice to object.Call. Also added the following method to agent interface.

GenerateObject(context.Context, schema.Schema, AgentCall) (*AgentResult, error)

Providers will need to support the added tools and tool choice in their GenerateObject methods. So far I have only tested it with OpenAI and works fine. Interface is a bit ugly because AgentResult has no typed Object any in its response to be able to pass the parsed object, but the text output should always be valid JSON that conforms to the schema.

Not sure if this is the right approach and probably not the best interface. I will publish a PR so it can be polished to meet project standards and hopefully get it merged.