Skip to content
This repository was archived by the owner on Apr 29, 2025. It is now read-only.
This repository was archived by the owner on Apr 29, 2025. It is now read-only.

Allow to opt out of caching for certain requests #25

@mattiapomelli

Description

@mattiapomelli

Caching is awesome to save on tokens, but there are few scenarios where someone might want to opt out of it.

E.g. allowing to re-generate a response if the user doesn't like it. In this case the prompt is the same as a previous request (so the response would be taken from the cache) but we actually want to obtain a fresh new result.

I would then suggest to allow to opt out of caching on a request basis. Maybe using a specific header for this?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions