Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow input csv and output csv in experiment run #103

Merged
merged 7 commits into from
Sep 24, 2024
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
update running experiments notebook
rchan26 committed Sep 24, 2024
commit 35019cd8b7210a8273044d164744c169a18190ce
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
24-09-2024, 09:14: Error (i=1, id=9): NotImplementedError - API unknown-api not recognised or implemented
24-09-2024, 09:14: Error (i=2, id=10): NotImplementedError - API unknown-api not recognised or implemented
24-09-2024, 09:14: Error (i=3, id=11): NotImplementedError - API unknown-api not recognised or implemented
24-09-2024, 09:14: Completed experiment: test.jsonl! Experiment processing time: 3.703 seconds, Average time per query: 1.234 seconds

This file was deleted.

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
{"id": 9, "prompt": ["Hello", "My name is Bob and I'm 6 years old", "How old am I next year?"], "api": "test", "model_name": "test", "parameters": {"candidate_count": 1, "max_output_tokens": 64, "temperature": 1, "top_k": 40}, "response": "This is a test response"}
{"id": 10, "prompt": ["Can you give me a random number between 1-10?", "What is +5 of that number?", "What is half of that number?"], "api": "test", "model_name": "test", "parameters": {"candidate_count": 1, "max_output_tokens": 128, "temperature": 0.5, "top_k": 40}, "response": "This is a test response"}
{"id": 11, "prompt": "How many theaters are there in London's South End?", "api": "test", "model_name": "test", "response": "This is a test response"}
{"id": 9, "prompt": ["Hello", "My name is Bob and I'm 6 years old", "How old am I next year?"], "api": "test", "model_name": "test", "parameters": {"candidate_count": 1, "max_output_tokens": 64, "temperature": 1, "top_k": 40}, "timestamp_sent": "24-09-2024-09-14-39", "response": "This is a test response"}
{"id": 10, "prompt": ["Can you give me a random number between 1-10?", "What is +5 of that number?", "What is half of that number?"], "api": "test", "model_name": "test", "parameters": {"candidate_count": 1, "max_output_tokens": 128, "temperature": 0.5, "top_k": 40}, "timestamp_sent": "24-09-2024-09-14-40", "response": "This is a test response"}
{"id": 11, "prompt": "How many theaters are there in London's South End?", "api": "test", "model_name": "test", "timestamp_sent": "24-09-2024-09-14-41", "response": "ValueError - This is a test error which we should handle and return"}
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
24-09-2024, 09:14: Error (i=3, id=11): ValueError - This is a test error which we should handle and return
24-09-2024, 09:14: Completed experiment: test2.jsonl! Experiment processing time: 3.615 seconds, Average time per query: 1.205 seconds

This file was deleted.

This file was deleted.

155 changes: 129 additions & 26 deletions examples/notebooks/running_experiments.ipynb

Large diffs are not rendered by default.