VAL: evaluating the chances of a tool being picked based on description and query cosine similarity

[heaven00]

This is an experimental testing strategy to use descriptions of the tools and curated queries for each tool, what this will allow us to measure is:

• An indicator of whether there the tool is likely to be picked up for a certain kind of query
• detect possible overlaps in descriptions that can reduce the effectiveness of the right tool being used.

There are possibly better ways like using https://sbert.net/examples/sentence_transformer/applications/semantic-search/README.html and improving our tool descriptions to contain examples etc. but this is meant to showcase the strategy.

It's also functional in its current form too for us to improve upon and have a discussion on :)

[zilto]

I really like the direction of this! I think it's a great occasion for dog-fooding dlt-hub/dlt. I'll open a separate repository to build the eval pipelines :)