Skip to content

Mc record category labeling #143

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
maxachis merged 21 commits into from
Jan 29, 2025
Merged

Mc record category labeling #143

maxachis merged 21 commits into from
Jan 29, 2025

Conversation

@maxachis
Copy link
Collaborator

@maxachis maxachis commented Jan 29, 2025 •
edited
Loading

#138

Several enhancements:

  • Adds logic for labeling record types, using OpenAI
  • Rename "Cycler" logic to "Task Operator". Add information clarifying nomenclature.
  • Add /task API points for getting information on specific tasks.
  • Remove unused files
  • Miscellaneous other changes

maxachis added 15 commits January 24, 2025 15:34
@maxachis
Add url_record_type_labeling directory and add init files to hugging_…
c743cdd 
…face directory and subdirectories
@maxachis
Add url_record_type_labeling directory to Dockerfile
076fbbb
@maxachis
Remove unused json files
8f81089
@maxachis
Begin draft
4c9f03f
@maxachis
Create first draft of DeepSeek record classifier and test
5cca756
@maxachis
Change "Cycle" term to "Task" for clarity. Add README description.
e1dce2f
@maxachis
Add precise test data
f861bd6
@maxachis
Draft work
593b219
@maxachis
Refine DeepSeekRecordClassifier
990cf3b 
* Add relevant environment variable to ENV.md
* Fix bug in manual DeepSeekRecordClassifier test
* Update DeepSeekRecordClassifier.py to pass test
* Refine URLHTMLContentInfo to include list of strings
@maxachis
Remove unused files
6a30a43
@maxachis
Implement draft of Record Type Classifier:
1e502a3 
* Refine Task Operators to follow unified logic, including logging
* Add tables related to tracking tasks and errors/urls within tasks
* Add new tests, update and refine existing tests
* Refactor some database client logic
@maxachis
Create /task route
698902e
@maxachis
Rename directory
fbc9bcb
@maxachis
Change name of url_error_info foreign key constraint
b90a7bd
@maxachis
Add logic for automatically assigning values to record types via OpenAI.
2cecaf5
@maxachis maxachis merged commit f017c1b into dev Jan 29, 2025
4 checks passed
@maxachis maxachis deleted the mc_record_category_labeling branch April 17, 2025 19:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@josh-chamberlain josh-chamberlain Awaiting requested review from josh-chamberlain josh-chamberlain is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@maxachis