api_prompt_routing.json

{
    "method": "api_prompt_routing",
    "task": "agieval_tiny",
    "task_type": "multiple_choice",
    "gpu_ids": [0,1],
    "model_names": [
        "chtmp223/Qwen2.5-7B-CLIPPER",
        "chengq9/ToolRL-Qwen2.5-3B",
        "AgentFlow/agentflow-planner-7b",
        "bunsenfeng/parti_3_full"
    ],
    "hyperparameters": {
        "max_response_length": 512,
        "temperature": 0.7,
        "top_p": 0.9,
        "batch_size": 32,
        "model_descriptions": [
            "We fine-tuned models for the task of narrative claim verification, where the model has to verify whether a claim about a book is True or False given the full book text. ",
            "The model is trained through RL especially for tool use task, focusing on tool learning, parameter filling, etc.",
            "The framework directly optimizes planner agent within the system in an online fashion using Flow-based Group Refined Policy Optimization (Flow-GRPO)",
            "Finetuned from llama3.1 to test an efficient language model architecture design. Finetuning enables us to test the architecture on a relatively large model while avoiding expensive train-from-scratch; this is very helpful for architecture research that doesn't deviate too much from the standard Transformer."
        ]
    }
}