Skip to content

(documentation) Near Entity Recognition: add aggregation_strategy docs and integration test script#561

Open
the-david-oy wants to merge 1 commit into
mainfrom
fde/bei-bert-ner-integration-docs
Open

(documentation) Near Entity Recognition: add aggregation_strategy docs and integration test script#561
the-david-oy wants to merge 1 commit into
mainfrom
fde/bei-bert-ner-integration-docs

Conversation

@the-david-oy

Copy link
Copy Markdown

Summary

  • Updated BEI-Bert-ner-bert-base-ner-uncased/README.md: Adds aggregation_strategy="max" request/response examples alongside the existing raw token-level API. Consecutive token predictions for multi-token entities (e.g. "j.d. vance", "New York City") are merged into a single span with the max-pooled label score — matching transformers.pipeline("ner", aggregation_strategy="max") behavior. Adds cURL + Performance Client examples.

  • Updated templating/generate_templates.py NER client_usage: Same addition so future generate_templates.py runs regenerate the README with aggregation_strategy docs included.

  • New BEI-Bert-ner-bert-base-ner-uncased/test_ner_integration.py: Validation script for BEI-Bert NER deployments. Run it against a live endpoint to verify:

    1. Token-level label predictions are 100% identical to HuggingFace Transformers
    2. aggregation_strategy="max" returns expected entity spans
    3. Latency benchmark (mean + p99 after warmup)
    4. Throughput benchmark (256 sentences, 32 concurrent requests)

Usage

# Deploy
cd 11-embeddings-reranker-classification-tensorrt/BEI-Bert-ner-bert-base-ner-uncased
truss push --publish

# Validate
pip install torch transformers baseten-performance-client
python 11-embeddings-reranker-classification-tensorrt/BEI-Bert-ner-bert-base-ner-uncased/test_ner_integration.py \
  --base-url https://model-xxxxxx.api.baseten.co/environments/production/sync

- Update NER/bert-base-ner-uncased README with aggregation_strategy="max"
  examples that match transformers.pipeline("ner", aggregation_strategy="max")
  behavior. Adds cURL, performance client, and per-token examples side-by-side.
- Update generate_templates.py NER client_usage to include the same
  aggregation_strategy documentation so future regenerations include it.
- Add test_ner_integration.py: validates BEI NER predictions match HuggingFace
  ground truth (token-level + span-level), runs latency/throughput benchmarks.
@the-david-oy the-david-oy added the documentation Improvements or additions to documentation label Mar 31, 2026
@the-david-oy the-david-oy self-assigned this Mar 31, 2026
@the-david-oy the-david-oy changed the title NER: add aggregation_strategy docs and integration test script (documentation) Near Entity Recognition: add aggregation_strategy docs and integration test script Mar 31, 2026
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants