Ability to attach metadata to user feedback so feedback can be more specific

[dividor]

Feature request

I would like to be able to attach metadata to user feedback, such as the retrieval result and the text excerpt for that result, the user was commenting on, but also environmental fields related to the application.

from langsmith import Client
client = Client()
client.create_feedback(run_id, feedback_type_str, score=score, comment=comment)

Looking at the code, there doesn't seem to be an option to attach metadata to user (API) feedback.

Motivation

Having metadata on feedback means it can be filtered more easily when queried. For example, empowering users to provide feedback on individual retrieval results is absolutely key to developing a solid evaluation test set. This is only possible if we attach some indication of which result and, for convenience, the excerpt they are rating.

Though some features exist in the Langsmith UI, it is way too technical for many non-technical business users - the people we need granular feedback from to refine any LLM system - so being able to simplify this by providing feedback options on the front-end app is very powerful. Without metadata much of this isn't possible and all we can do is capture feedback on the run, which isn't specific enough.

[oegeshipster]

you can pass a parameter correction that can be used for field specific corrections I think. maybe that could help?