DataEvolver

Self-Evolving Multi-Agent Data Construction for Text-Rich Image Generation

Siyu Yan^1,2,*   Yizhen Gao^1,*   Yilin Wang¹   Dongxing Mao¹   Alex Jinpeng Wang^1,†

¹Central South University    ²The Hong Kong University of Science and Technology
^*Equal contribution    ^†Corresponding author

The Idea · Architecture · Agents · Results · Quick Start · Citation

---

The Idea

Rejected samples are not waste. They are the feedback signal that should shape the next construction round.

DataEvolver is a self-evolving multi-agent data construction framework for text-rich image generation. Instead of following a static crawl → filter → freeze pipeline, DataEvolver treats rejected samples as feedback signals and uses them to improve subsequent construction rounds.

The key idea is simple: rejected samples should not be discarded silently; they should guide the next round of retrieval, filtering, and generation. Construction-time failures, including OCR errors, semantic mismatches, duplicates, and topic coverage gaps, are converted into actionable feedback for policy revision.

01 · Policy evolution Turn rejection patterns into reusable guidance.

02 · Four agents Retriever, Verifier, Critic, and Generator close the loop.

03 · Training-free Evolve the construction policy in natural-language space.

04 · Strong transfer Improve multiple text-rich image generators.

Closed-Loop Architecture

DataEvolver replaces a static data pipeline with an adaptive loop: retrieve or generate candidates, verify them, diagnose rejection causes, update the policy, and begin the next round with better queries and prompts.

_{DataEvolver pipeline. Feedback drives query updates, experience updates, and prompt refinement across construction rounds.}

Four Cooperative Agents

Agent	Role	Backend
🔍 Retriever	Discovers candidate images with optimized search queries and updates query strategies using feedback from previous rounds.	llm.model via Ollama, default mistral:latest
✅ Verifier	Filters candidates through OCR, image quality, watermark, deduplication, CLIP semantic relevance, and text-consistency checks.	PaddleOCR/Tesseract + CLIP + Sentence-Transformers + pHash
📊 Critic	Summarizes rejection patterns into semantic feedback, maintains an experience library, and revises construction policies.	llm.cir_model via Ollama, default qwen3.5:4b
🎨 Generator	Uses CountAnalyse and PromptPlanner to identify coverage gaps, write prompts, synthesize text-aware images, and re-filter generated samples.	mistral:latest + qwen3.5:4b agents, Qwen/Qwen-Image T2I, optional qwen3-vl:latest annotation

The entire loop is traceable: accepted samples retain query, caption, OCR, quality, semantic, and filtering metadata, while rejected samples preserve the evidence needed for policy revision.

Results

On PixArt-α at the 0.75M data scale, DataEvolver improves OCR-F1 over the strongest matched-budget baseline.

_{Overview and scaling trend. DataEvolver converts discarded samples into iterative policy updates and improves OCR-F1 under matched data budgets.}

Evaluation Set	Relative OCR-F1 Improvement
TextScenesHQ	+85.3%
LongTextBench	+35.3%

Ablation studies show that both the Critic and the Generator contribute to the final performance, indicating that feedback-based policy revision and targeted completion are both necessary for effective text-rich data construction.

Quick Start

1. Install

git clone https://github.com/CSU-JPG/DataEvolver.git
cd DataEvolver

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

2. Run the pipeline

For an existing environment, run the main entrypoint directly:

python main.py --config config.yaml --shard-id host-a --keep-rejects

--shard-id is required and should be unique for each concurrent machine or shard. Use --keep-rejects if you want rejected samples retained for diagnosis.

Full CUDA and Ollama bootstrap

For a full CUDA/Ollama bootstrap, use:

bash run.sh

The script creates a Python 3.9 Conda environment, installs CUDA-oriented dependencies, starts eight local Ollama servers on ports 11437-11444, pulls the default Ollama models, and launches main.py.

Configuration

All major settings are controlled by config.yaml.

Paths — data, logs, and output directories

paths:
  images_crawled: "/path/to/crawled/images"
  images_generated: "/path/to/generated/images"
  ann_dir: "/path/to/annotations"
  log_dir: "/path/to/logs"
  accepted_dir: "/path/to/accepted"
  gen_dir: "/path/to/generated/raw"
  gen_accepted_dir: "/path/to/gen/accepted"

LLM & Agent Models — Ollama-compatible local models

llm:
  model: "mistral:latest"          # Retriever / QueryGenerator / QueryPlannerAgent / CountAnalyse
  cir_model: "qwen3.5:4b"          # Critic agents / PromptPlanner / prompt rewriting

All LLM agents are served through local Ollama instances. src/agents/local_agents.py round-robins requests over ports 11437-11444.

Pipeline Control — crawl/generate phases and feedback loops

dataset_num: 1000000

generation:
  enabled: true
  max_regeneration_rounds: 3
  prompt_critic_enabled: true

critic:
  enabled: true
  strategy:
    enabled: true
    queries_enabled: true
    warmup_rounds: 3
  experience:
    use_agent: true

dynamic_query:
  enabled: true
  low_watermark: 30

feedback_query:
  enabled: true
  min_samples: 20

The main run first performs asynchronous retrieval and filtering, then enters the optional generation phase when generation.enabled is true.

Retrieval, OCR, and Filtering — Bing, OCR, quality, semantic, and dedup checks

bing:
  per_subtopic_images: 1500
  mkt: "auto"
  safesearch: "moderate"

ocr:
  engine: "paddle"
  lang: "ch"
  use_multi_gpu: true
  use_multiprocess: true

quality:
  min_ocr_coverage: 0.25
  min_legibility: 0.5
  min_side: 384
  max_side: 4096

semantic:
  enabled: true
  model_name: "openai/clip-vit-base-patch32"

text_consistency:
  enabled: true
  model_name: "sentence-transformers/all-MiniLM-L6-v2"

hash_deduplication:
  enabled: true
  max_distance: 3
  scope: "global"

The Verifier combines OCR-derived quality signals, watermark detection, pHash deduplication, CLIP similarity, and sentence-transformer text consistency before writing accepted samples.

Generation and Annotation Models — Qwen-Image and Qwen-VL

qwen:
  true_cfg_scale: 4.5
  num_inference_steps: 25
  seed: 42

annotation:
  qwen_vl:
    enabled: true
    model: "qwen3-vl:latest"
    verbose: false

Image synthesis uses the Diffusers pipeline Qwen/Qwen-Image by default. Override it with the QWEN_IMAGE_MODEL environment variable if needed. The Qwen-VL setting is used for optional vision-language caption/annotation, not for text-to-image generation.

Topics & Seed Queries — target domains for data construction

topics:
  - name: Store_Signs_and_Shopfronts
    seed_queries:
      - "store signboard shop front"
      - "restaurant sign food street sign"
      - "Chinese shop plaque traditional signage"

  - name: Book_Covers
    seed_queries:
      - "book cover title author text"
      - "novel cover book jacket design"

Seed queries define the initial retrieval space. During construction, the Retriever and Critic refine these queries based on acceptance and rejection patterns.

Quick Customization

1. Edit paths in config.yaml.
2. Define target topics and seed queries.
3. Choose Ollama-compatible LLM backends.
4. Start Ollama instances and make sure mistral:latest, qwen3.5:4b, and qwen3-vl:latest are available, or replace them in config.yaml.
5. Adjust OCR, semantic, text-consistency, deduplication, and quality thresholds.
6. Enable or disable generation with generation.enabled.
7. Run python main.py --config config.yaml --shard-id <unique-shard-id>.

Citation

If you find DataEvolver useful, please cite:

@article{yan2026dataevolver,
  title   = {DataEvolver: Self-Evolving Multi-Agent Data Construction for Text-Rich Image Generation},
  author  = {Yan, Siyu and Gao, Yizhen and Wang, Yilin and Mao, Dongxing and Wang, Alex Jinpeng},
  journal = {arXiv preprint},
  year    = {2026}
}

Acknowledgments

This project builds on several open-source tools, including PaddleOCR, Tesseract OCR, Ollama, Hugging Face Diffusers, Qwen-Image, Qwen-VL, CLIP, and Sentence-Transformers.

---

Reject. Reflect. Revise. Evolve.