Semantic Drift in Long-Form Financial Disclosures in Portuguese

This repository contains the analysis code, configuration, and reference result tables for a study of semantic drift over time in long-form financial disclosures written in Portuguese. The pipeline measures how the meaning of successive disclosures from the same issuer changes across time, compares neural sentence-embedding models against lexical baselines, and validates drift spikes against external market and event signals.

The Journal of Finance and Data Science accepted the article on June 26, 2026 (manuscript JFDS-D-26-00085R1). Elsevier made the journal pre-proof available online on July 2, 2026 as article 100199.

Article DOI:

https://doi.org/10.1016/j.jfds.2026.100199

Dataset DOI:

https://doi.org/10.5281/zenodo.21263919

NILC repository:

https://github.com/nilc-nlp/semantic-drift-disclosures

Folder: CODE

A config-driven (Hydra + OmegaConf) Python pipeline.

• CODE/src/semantic_drift/ — the package (data, model, analysis, trainer modules).
• CODE/run/conf/ — Hydra configuration groups (datasets, models, aggregations, tasks).
• CODE/run/pipeline/main.py — the pipeline entrypoint.
• CODE/run/analysis/scripts/ — post-hoc analysis (baselines, event studies, figures).
• CODE/tests/ — unit and integration tests plus toy fixtures.
• CODE/RESULTS/ — path-sanitized result tables (drift series, report summaries, event registries) backing the paper.

Folder: DATA

We keep the raw corpus (disclosure PDFs) out of git. DATA/ contains the folder layout, a manifest.yaml inventory, and a fetch helper (DATA/scripts/fetch_data.py). Datasets: Vale, Itaú, EDP, Conab (grãos), and SLC Agrícola (507 documents, ~2 GB).

The corpus uses public regulatory filings archived on Zenodo:

https://doi.org/10.5281/zenodo.21263919

Download and verify all datasets into DATA/raw/ (URLs and SHA-256 checksums are in manifest.yaml):

python DATA/scripts/fetch_data.py            # all datasets, with checksum verification
python DATA/scripts/fetch_data.py --list     # show datasets and local status
python DATA/scripts/fetch_data.py --dataset vale   # a single dataset

For manual download, fetch the five zips from Zenodo and unpack each into DATA/raw/<dataset>/. The Hydra dataset configs point at ../DATA/raw/<folder> (relative to CODE/). Use dataset=toy to run the pipeline without the full corpus.

How to install

Requires Python 3.12 and uv.

cd CODE
uv sync --group dev

How to run

All commands run from the CODE/ directory.

# Offline smoke test (uses bundled toy fixtures, no real data):
uv run python run/pipeline/main.py dataset=toy model=dummy

# Full pipeline for one configuration (after DATA/raw/ is populated):
uv run python run/pipeline/main.py dataset=vale model=bertimbau aggregation=mean_pooling

# Lexical baselines (document-level):
uv run python run/pipeline/main.py dataset=vale model=tfidf
uv run python run/pipeline/main.py dataset=vale model=jaccard

# Validations that require network data (stock / macro series):
uv run python run/pipeline/main.py dataset=vale model=bertimbau validation=with_network

# Post-hoc analysis:
uv run python run/analysis/scripts/baseline_comparison.py
uv run python run/analysis/scripts/article_drift_csv.py

The pipeline writes generated artifacts under CODE/run/outputs/ (gitignored). CODE/RESULTS/ contains the reference tables used in the paper.

Models and API keys

• BERTimbau and EuroBERT run locally via sentence-transformers; weights download on first use.

• Qwen3-Embedding-8B uses OpenRouter and needs an API key:

  cd CODE
  cp .env.example .env
  # set OPENROUTER_API_KEY in .env (never commit this file)
  uv run python run/pipeline/main.py dataset=vale model=qwen aggregation=mean_pooling

.gitignore excludes .env; do not commit API keys.

License

MIT covers the code and configuration. Source terms govern the raw disclosure PDFs.

BibTeX Entry

@article{correa2026semanticdrift,
  author  = {Correa, Joao Victor M. and Bravo, Jorge Miguel and
             da Silveira, Rodrigo Lanna Franco and Cruz Junior, Jose Cesar and
             Batista, Fernando and Silva, Renato Moraes},
  title   = {Semantic Drift in Long-Form Financial Disclosures in Portuguese},
  journal = {The Journal of Finance and Data Science},
  year    = {2026},
  pages   = {100199},
  doi     = {10.1016/j.jfds.2026.100199}
}