bibtidy

A fast, zero-dependency BibTeX linter, deduplicator, and manuscript citation checker for people who write papers in LaTeX or Pandoc Markdown.

$ python3 -m bibtidy lint refs.bib
refs.bib:25: error E002: [Smith2020] near-duplicate of smith2020analysis (run `bibtidy dedupe` to merge)
refs.bib:46: error E003: [draft42] missing required field 'author' for @article
refs.bib:51: warning W002: [nodoi2023] missing DOI on @article from 2023
bibtidy: 47 entries; 2 errors, 1 warnings

---

Why bibtidy?

You merged a .bib from a co-author. Zotero added entries with capitalized keys. Google Scholar exported some with no DOI. You'd like a flake8 for your bibliography. None of the existing tools fit:

Tool	Problem
JabRef	GUI, hard to script, can't drop into CI
bibtool	Powerful but cryptic DSL syntax
biber	Tied to the LaTeX compile cycle, no smart dedup
bibtex-tidy	JS-only, focuses on whitespace not semantics
bibmanager	Forces you into its own SQLite database
Caltech bibmerge	Web-only — paste your bib into a browser
bibtidy	One Python file tree, stdlib only, runs in <200 ms on 5K entries

Install

git clone https://github.com/asech109/bibtidy.git
cd bibtidy
python3 -m bibtidy --version    # bibtidy 0.1.0

Requirements: Python 3.9+. No pip install, no virtualenv, no network.

If you'd rather treat it as a binary, symlink it into your $PATH:

ln -s "$(pwd)/bibtidy" ~/.local/lib/bibtidy
echo '#!/usr/bin/env bash\nexec python3 -m bibtidy "$@"' > ~/.local/bin/bibtidy
chmod +x ~/.local/bin/bibtidy

Three commands

1. bibtidy lint — find problems

python3 -m bibtidy lint refs.bib

Exit code = number of error-severity findings, so it composes with CI and pre-commit hooks:

python3 -m bibtidy lint refs.bib && echo "bib is clean"

For machines, use --json:

python3 -m bibtidy lint refs.bib --json | jq '.diagnostics[] | select(.code=="E002")'

2. bibtidy dedupe — collapse near-duplicates

python3 -m bibtidy dedupe refs.bib -o refs.clean.bib

Feed it as many .bib files as you like; they'll be merged together:

python3 -m bibtidy dedupe mine.bib coauthor.bib zotero-export.bib -o combined.bib

The merge is deterministic:

• The entry with the most non-empty fields becomes the base.
• Every field across the cluster is filled with the longest non-empty value.
• The lexicographically smallest key wins.
• The other keys are recorded as aliases, so older \cite{Smith2020} references in your .tex are still recognized by bibtidy check.

3. bibtidy check — manuscript ↔ bib

python3 -m bibtidy check paper.md refs.bib

Reports broken citations (referenced in paper.md but missing from refs.bib) with line numbers, and orphan entries (in the bib but never cited). Exit code = number of broken citations.

Supported Pandoc citation forms:

Form	Recognized
@key	yes
[@key]	yes
[@key, p. 33] (with locator)	yes
[@key1; @key2; @key3] (multi)	yes
[-@key] (suppress-author)	yes
user@example.com (email)	ignored
Inside fenced code or inline	ignored

Diagnostic codes

Code	Severity	Meaning
E001	error	Duplicate citation key
E002	error	Near-duplicate entry (suggest bibtidy dedupe)
E003	error	Missing required field for this entry type
E004	error	Malformed entry (could not be parsed)
W001	warning	Title is entirely lowercase
W002	warning	Missing DOI on a recent @article / @inproceedings
W003	warning	Same author + year + similar title, but below merge threshold

Pre-commit hook

.git/hooks/pre-commit:

#!/usr/bin/env bash
set -e
for bib in $(git diff --cached --name-only --diff-filter=ACM | grep '\.bib$'); do
    python3 -m bibtidy lint "$bib"
done

Or as a pre-commit framework entry in .pre-commit-config.yaml:

- repo: local
  hooks:
    - id: bibtidy
      name: bibtidy lint
      entry: python3 -m bibtidy lint
      language: system
      files: \.bib$

How dedup works

Two signals feed one union-find over the entire entry list:

1. DOI exact match (after normalization: lowercase, strip https://doi.org/, trim trailing punctuation).
2. (year, normalized-first-author-lastname) bucket + title similarity ≥ 0.92 using difflib.SequenceMatcher on accent-stripped, alphanumeric-only titles.

Year mismatch disqualifies the title-similarity signal; missing year + missing DOI means we refuse to merge (deliberate false-positive guardrail — preprint and erratum often share a title).

Architecture

bibtidy/
├── parser.py      tolerant BibTeX tokenizer + serializer (~316 LOC)
├── normalize.py   DOI / title / author normalization (~64 LOC)
├── dedupe.py      union-find over DOI + title-similarity (~166 LOC)
├── lint.py        rule engine producing Diagnostic records (~155 LOC)
├── manuscript.py  Pandoc-citation regex scanner (~97 LOC)
└── cli.py         argparse subcommands + JSON output (~190 LOC)

Every module is pure (no I/O). The CLI is the only thing that touches the filesystem. No file exceeds 500 lines.

Tests

python3 -m unittest discover -s tests -v
# Ran 28 tests in 0.005s
# OK

Performance

On a synthetic 5,000-entry .bib (~5% duplicates):

Command	Wall time
lint	176 ms
dedupe	164 ms

(Apple Silicon, Python 3.12. See VALIDATION.md for the full benchmark.)

Limitations

• No crossref-field expansion (rarely used in modern .bib files).
• Title similarity is tuned for Latin-script titles; CJK works but you may want to lower TITLE_RATIO_THRESHOLD in dedupe.py.
• No external lookup (Crossref / OpenAlex) — offline by design.
• Aliases recorded by dedup live in memory only; we do not write them back into the serialized .bib (no standard format for it).

Roadmap

• --fix flag on lint for low-risk auto-fixes (DOI URL stripping, title brace protection).
• bibtidy rekey refs.bib to enforce a consistent key scheme (firstauthor_year_titlestem).
• LaTeX \cite{...} / \textcite{...} scanner (currently Pandoc only).
• Optional Crossref lookup behind --online, with a small disk cache.

License

MIT — see LICENSE.