Revert "Revert "Add composite scores"" 😬 #581
Conversation
This reverts commit 55a3d48.
There was a problem hiding this comment.
Pull Request Overview
This PR reverts a previous revert and restores the composite evaluators documentation for LangSmith. It adds back the ability to combine multiple evaluator scores into a single composite score using both SDK and UI methods.
- Restores complete documentation for composite evaluators functionality
- Adds guidance for both programmatic (SDK) and UI-based composite evaluator creation
- Includes comprehensive examples with weighted averaging and sum aggregation methods
Reviewed Changes
Copilot reviewed 2 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| src/langsmith/composite-evaluators.mdx | Complete documentation file for composite evaluators with SDK examples and UI workflow |
| src/docs.json | Navigation entry to include the composite evaluators page in the LangSmith evaluators section |
|
Preview ID generated: preview-revert-1758211550-bac30a7 |
Co-authored-by: Copilot <[email protected]>
| load_dotenv() | ||
|
|
||
| # Access environment variables | ||
| openai_api_key = os.getenv('OPENAI_API_KEY') |
There was a problem hiding this comment.
The variable oai_client is used throughout the code but is never initialized. You need to add oai_client = OpenAI(api_key=openai_api_key) after retrieving the API key.
| openai_api_key = os.getenv('OPENAI_API_KEY') | |
| openai_api_key = os.getenv('OPENAI_API_KEY') | |
| oai_client = OpenAI(api_key=openai_api_key) |
Copilot uses AI. Check for mistakes.
| WEIGHTED_KEY = "weighted_summary" | ||
|
|
||
| # Pull experiment results | ||
| EXPERIMENT_ID = list(client.list_projects(reference_dataset_name=DATASET_NAME, limit=1))[0].id |
There was a problem hiding this comment.
The variable DATASET_NAME is not defined. It should reference the dataset name, likely dataset.name from the earlier dataset creation.
Copilot uses AI. Check for mistakes.
|
Preview ID generated: preview-revert-1758552995-531bd21 |
|
Preview ID generated: preview-revert-1758643688-56cd113 |
|
Preview ID generated: preview-revert-1758643897-127f7e6 |
Reverts #580
MERGE ON TUESDAY