T-Pot Payload Server

The T-Pot Payload Server is a lightweight, stateless FastAPI microservice designed to extract, inspect, and serve payload attack binaries captured by T-Pot honeypots. Developed openly as part of the GreedyBear ecosystem, it provides secure O(1) payload lookups, hash/metadata generation, and streaming downloads for automated ingestion by GreedyBear or external security research platforms.

---

Architecture

The payload server runs as a containerized sidecar alongside T-Pot on the host machine. It mounts host honeypot capture directories as read-only volumes and exposes a secure REST API.

flowchart TD
    subgraph Host ["T-Pot Host System"]
        subgraph Honeypots ["T-Pot Honeypots"]
            Dionaea["Dionaea"]
            Cowrie["Cowrie"]
            Honeytrap["Honeytrap"]
            ADBHoney["ADBHoney"]
        end

        DataDir["T-Pot Data Volume Directory\n(e.g., /home/user/tpotce/data)"]
        Dionaea -->|Captures binaries| DataDir
        Cowrie -->|Captures binaries| DataDir
        Honeytrap -->|Captures binaries| DataDir
        ADBHoney -->|Captures binaries| DataDir

        subgraph Container ["tpot-payload-server (Docker)"]
            FastAPI["FastAPI Application"]
            Auth["X-API-Key Guard"]
            Scanner["Metadata Scanner & Hasher"]
        end

        DataDir -.->|"Read-Only Mount (:ro)"| Container
    end

    GreedyBear["GreedyBear / Consumers"] -->|"GET /api/v1/payloads/recent"| Auth
    GreedyBear -->|"GET /api/v1/payloads/download/{locator}"| Auth
    Auth --> FastAPI
    FastAPI --> Scanner

Loading

---

Deployment

Prerequisites

• Docker 20.10+ and Docker Compose v2.0+
• A running instance of T-Pot CE (or existing honeypot data directories on host, currently supported by version T-Pot 24.04.1)

Automated Installation (Recommended)

Run the included install.sh script to automatically detect your T-Pot installation (if at the default path, you can use the --tpot-dir flag to specify a custom path), generate a secure API key, configure an HTTPS reverse proxy, and deploy the container stack.

git clone https://github.com/GreedyBear-Project/tpot-payload-server.git
cd tpot-payload-server
sudo ./install.sh

Note: The deployment runs independently and safely survives T-Pot updates (git reset --hard).

Manual Installation

1. Clone the repository:

   git clone https://github.com/GreedyBear-Project/tpot-payload-server.git
   cd tpot-payload-server

2. Configure environment variables: Copy the example environment configuration into docker/.env:

   cp docker/.env.example docker/.env

   Open docker/.env and configure TPOT_DATA_PATH with the absolute path to T-Pot's data directory on your host:

   TPOT_DATA_PATH=/home/user/tpotce/data

3. Start the container:

   docker compose -f docker/docker-compose.yml up -d

4. Verify container health:

   # If using the default HTTPS proxy:
   curl -k https://localhost:64445/health
   
   # If you disabled the proxy or are testing the API directly:
   curl http://localhost:64444/health
   
   # Expected response: {"status":"ok"}

Configuration Reference

All settings can be configured via environment variables in docker/.env:

Environment Variable	Required	Default	Description
TPOT_DATA_PATH	Yes	(None)	Absolute path to T-Pot's data directory on the host system (e.g. /home/user/tpotce/data or /data).
API_KEY	No	(Empty)	Secret key for authenticating API requests via the X-API-Key header. When empty/unset, authentication is disabled.
API_PORT	No	64444	Host port mapped to the API service container.
PROXY_PORT	No	64445	Host port for the NGINX HTTPS reverse proxy.
HONEYPOT_DIRS	No	dionaea/binaries,cowrie/downloads,honeytrap/downloads,adbhoney/downloads	Comma-separated list of relative honeypot subdirectories to scan for payloads.

---

API Reference

The service exposes the following endpoints:

GET /health

Returns the status of the payload server.

• Response: 200 OK

  {
    "status": "ok"
  }

---

GET /api/v1/payloads/recent

Scans configured honeypot directories and returns metadata for payload files modified within the specified Unix timestamp window.

• Query Parameters:
  • start_ts (float, required): Start of modification time window (Unix timestamp, inclusive).
  • end_ts (float, required): End of modification time window (Unix timestamp, inclusive).
• Headers: X-API-Key (string, required if API_KEY is configured).
• Response: 200 OK — List of PayloadMetadata objects.

  [
    {
      "locator": "dionaea/binaries/0123456789abcdef0123456789abcdef",
      "mime_type": "application/x-dosexec",
      "md5": "e10adc3949ba59abbe56e057f20f883e",
      "sha1": "cdfbe90179257628a7e0a16a49591410884ef47a",
      "sha256": "f2ca1bb6c7e907d06dafe4687e579fce76b37e4e93b7605022da52e6ccc26fd2",
      "mtime": 1723000000.0,
      "size": 1048576,
      "source_honeypot": "dionaea"
    }
  ]

• Error Responses:
  • 403 Forbidden: Missing or invalid X-API-Key header.
  • 422 Unprocessable Content: Invalid timestamps (e.g., start_ts > end_ts).

---

GET /api/v1/payloads/download/{locator}

Streams the raw binary payload file identified by its relative locator (obtained from /recent).

• Path Parameters:
  • locator (string, required): Relative file locator path (e.g. dionaea/binaries/sample.bin).
• Headers: X-API-Key (string, required if API_KEY is configured).
• Response: 200 OK — application/octet-stream binary file stream.
• Error Responses:
  • 403 Forbidden: Missing or invalid X-API-Key header.
  • 404 Not Found: Payload file does not exist at locator path.
  • 422 Unprocessable Content: Disallowed locator path (e.g., path traversal attempts).

---

Interactive Documentation

FastAPI automatically generates interactive OpenAPI documentation:

• Swagger UI: Available at http://<host>:<port>/docs
• ReDoc: Available at http://<host>:<port>/redoc
• OpenAPI Schema (JSON): Available at http://<host>:<port>/openapi.json

---

Authentication

Authentication uses header-based API key validation:

• Header Name: X-API-Key
• Behavior:
  • If API_KEY environment variable is set: All protected endpoints require a matching X-API-Key header.
  • If API_KEY environment variable is empty or unset: Authentication is disabled (designed for isolated internal network operation).

---

Security Model

The server incorporates defense-in-depth mechanisms for safe handling of untrusted malware samples:

1. Read-Only Volume Mounts: Honeypot capture directories on the host are mounted into the container as read-only (:ro), preventing any file modification or deletion.
2. Read-Only Container Filesystem: Container execution specifies read_only: true, preventing write operations to root filesystems.
3. Privilege Isolation: Container runs with no-new-privileges:true and as a non-root user.
4. Path Traversal Guards:
   • Locator path components are strictly sanitized against HONEYPOT_DIRS.
   • Filenames are regex-validated against safe alphanumeric patterns (^[A-Za-z0-9._-]+$).
   • Path resolution verifies that target files reside inside BASE_DATA_DIR using is_relative_to, guarding against symlink traversal.
5. Non-Executing Inspection: File analysis calculates cryptographic hashes and MIME types via streaming read blocks without executing or loading sample code.

---

Development

Setup

We use uv for dependency management and ruff for linting and formatting.

1. Install dependencies:

   uv sync --all-groups

2. Configure pre-commit hooks:

   uv run pre-commit install -c .github/.pre-commit-config.yaml

3. Run the test suite:

   uv run pytest

4. Lint and format code:

   uv run ruff check
   uv run ruff format

---

Sponsors and Acknowledgements

The Honeynet Project

The Honeynet Project is an international non-profit security research organization dedicated to investigating cyber attacks and developing open-source security tools.

Google Summer of Code

This project was developed during the Google Summer of Code (GSoC) program!

---

Maintainers and Contributors

Special thanks to:

• Tim Leonhard for mentoring and guiding the project architecture.
• opbot-xd for all the contributions.

---

How to Contribute

Head over to our CONTRIBUTING guide for details on submitting issues and pull requests.