Skip to content

Conversation

@mikehostetler
Copy link
Contributor

Problem

Users need to apply Req plugins (logging, caching, middleware) without dropping to low-level API.

Solution

Added :req_plugins option that accepts a list of (Req.Request.t() -> Req.Request.t()) functions.

Supported Functions

  • ReqLLM.generate_text/3
  • ReqLLM.generate_text!/3
  • ReqLLM.generate_object/4
  • ReqLLM.generate_object!/4
  • ReqLLM.embed/3
  • ReqLLM.Images.generate_image/3

Note: Streaming functions (stream_text/3, stream_object/4) use Finch directly and do not support :req_plugins.

Usage

# Add custom header
ReqLLM.generate_text(model, prompt,
  req_plugins: [
    fn req -> Req.Request.put_header(req, "X-Custom", "value") end
  ]
)

# Chain multiple plugins (executed in order)
ReqLLM.generate_text(model, prompt,
  req_plugins: [
    &MyApp.ReqPlugins.add_logging/1,
    &MyApp.ReqPlugins.add_tracing/1
  ]
)

Testing

  • Added 3 tests verifying plugins are applied and order is preserved

Closes #359

Allows users to apply Req plugins to requests without using the low-level API.
Plugins are functions that transform the Req.Request before execution.

Supported in:
- ReqLLM.Generation.generate_text/3
- ReqLLM.Generation.generate_object/4
- ReqLLM.Embedding.embed/3
- ReqLLM.Images.generate_image/3

Note: Streaming functions (stream_text/3, stream_object/4) use Finch directly
and do not support :req_plugins.

Closes #359

Amp-Thread-ID: https://ampcode.com/threads/T-019c117b-7aa8-72ba-9556-23dc1118034f
Co-authored-by: Amp <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Allow setting up Req Attachments without the need of using low level API

2 participants