arXiv Classifier Annotator

This repo contains code for the annotation tool at arxiv-classifier-annotator.streamlit.app/. The tool is made publicly available using Streamlit Community Cloud and uses Firebase as the data backend.

TODO:

• Create a config.yaml file to store these variables

Setup

To install dependencies, run:

conda create -n arxiv_website
conda install python=3.12 conda-forge::streamlit conda-forge::bs4
pip install firebase-admin

Setting up firestore:

1. Create a new private key: https://console.firebase.google.com/u/0/project/arxiv-website/settings/serviceaccounts/adminsdk
2. Download key locally to a destination outside the repo and create a softlink to API_KEYS

# in the repo root
mkdir API_KEYS
ln -s <absolute path to download destination> API_KEYS/certificate.json

Troubleshooting info from Johann

From Johann:

PermissionDenied: 403 Cloud Firestore API has not been used in project arxiv-website before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/firestore.googleapis.com/overview?project=arxiv-website then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry. [links { description: "Google developers console API activation" url: "https://console.developers.google.com/apis/api/firestore.googleapis.com/overview?project=arxiv-website" } , reason: "SERVICE_DISABLED" domain: "googleapis.com" metadata { key: "service" value: "firestore.googleapis.com" } metadata { key: "consumer" value: "projects/arxiv-website" } ] https://console.cloud.google.com/apis/api/firestore.googleapis.com/metrics?project=arxiv-website https://console.firebase.google.com/u/0/project/arxiv-website/settings/serviceaccounts/adminsdk https://console.cloud.google.com/firestore/databases/-default-/data/panel/mod_queues/0?authuser=0&hl=en&project=arxiv-website

Deploy

1. Push the moderator queues to Firestore using the following command:

python push_mod_queues.py -dp DATA_PATH --mod_queue_collection MODERATOR_QUEUE_COLLECTION

Note

Here, DATA_PATH is the path to the JSON file generated by our evaluation protocol. Examples of Firestore collections can be found in utils.py (see MODERATOR_QUEUE_COLLECTION, PAPER_INFO_COLLECTION, MODERATOR_RESULTS_COLLECTION).

2. Push the paper info to Firestore using the following command:

python push_paper_info.py -dp DATA_PATH --mod_queue_collection PAPER_INFO_COLLECTION

3. To run (locally):

streamlit run arxiv-classifier-app.py