Long Competition 2025: Spyfall

This is the repository for the 2025 UT Austin MLDS Long Competition where teams will be competing in a game of Spyfall. Teams will submit agents that can use LLMs and embedding models to ask/answer questions and determine the spy/location. The goal of this Long Competition is to show your LLM prompting/natural language processing skills.

Game Rules

Setup: Each game is played with 4-12 players. All players are given the location except for the spy.

Spy Objective: The spy must figure out the location without revealing their identity.

Non-Spy Objective: Players must figure out who the spy is.

Gameplay: Each game consists of a fixed number of rounds. In each round, the following happens:

1. Questioning: A random player starts by asking another player a question about the location. The player who answers the question will be the one to ask the question in the next round. You can ask a question to any player except the player who asked you a question the previous round.

   Ex: A --> B --> A is not allowed.

2. Questioning Analysis: All players are given time to analyze the question/answer.

3. Guessing: The spy may guess the location. This will end the game.

4. Accusation: Players may accuse another player of being the spy. Successfully indicting a player will end the game. For a player to be indicted, the following conditions must be met:

   • A majority of the players must accuse any player.
   • One player must get a plurality of the votes. If a tie occurs, nothing happens.

   Ex: If 2 players accuse player A, 1 player accuses player B, 1 player accuses player C, and 3 players do not vote, player A is successfully indicted.

5. Voting Analysis: Players can see who voted for who and are given time to analyze the votes.

The game ends when:

• The spy guesses the location.
• A player is indicted of being the spy.
• All rounds are completed.

Scoring:

• Spy Victory: The spy earns 2 points if no one is indicted of being the spy, 4 points if a non-spy player is indicted of being the spy, and 4 points if the spy stops the game and successfully guesses the location.
• Non-Spy Victory: Each non-spy player earns 1 point.

Getting Started

Setting up the Code:

To run this project locally with conda, run the following commands. This may take a while.

conda create -n long_comp python==3.10.13
conda activate long_comp
pip install -r requirements.txt

If you do not have conda installed, you can install it here. Additionally, if you use VSCode, the VSCode Extension Python Environment Manager by Don Jayamanne is nice for managing and selecting default conda environments.

If you use additional packages, please add them to requirements.txt.

Note: this repository is thoroughly type-annotated. To catch type errors, you can install the VSCode Extension Mypy Type Checker by Microsoft.

Setting up Your LLM API Key:

We will be using together.ai which offers a $5 credit (~50M tokens) for new users, no credit card required.

To get your API key, click the link above to create an account. A pop-up will appear with your API key.

Next, create a file named .env in the root directory with the following text:

TOGETHER_API_KEY = <your_together_api_key>

Running Games/Simulations:

See main.py for an example of how to run games and simulations.

Note: by default, a dummy llm and embedding model are selected. You can change this at the top of main.py.

Submitting Your Agent:

Write your agent in submission.py and use @register_agent(<team name here>) to register your agent under your team name. Also, please put your team member's name/emails/EIDs in a comment at the top. You can name your class anything you want. Commit and push submission.py and any other files you added to the GitHub Classroom repository.

Rules

• Besides submission.py, requirements.txt, and any added files, do not modify any other files in the repository. For instance, do not change data.py. If you need to store data, use a separate file.
• Prompt injections are allowed.
• You may not use any NLP models outside what is provided.