Skip to content

fix(pii): cap each batched NER pass at one window of characters - #10099

Merged
waleedlatif1 merged 1 commit into
stagingfrom
fix/pii-batch-char-bound
Nov 16, 2026
Merged

waleedlatif1 merged 1 commit into
stagingfrom
fix/pii-batch-char-bound

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

  • _analyze_many sent every text under NER_WINDOW_CHARS to analyze_iterator in one call, so many mid-size texts shared one spaCy pass whose memory grows with their total length (ten 99k-char texts cost as much as one 990k-char text)
  • Short texts are now packed in order into batches of at most NER_WINDOW_CHARS characters in total, one analyze_iterator call per batch, with _NER_SLOT taken per batch so other requests interleave, the same as _analyze_long's windows
  • No NER pass now sees more than one window of characters, whether it covers a short text, a window, or a batch

Type of Change

  • Bug fix

Testing

  • Measured peak RSS of _analyze_many with the en model under MALLOC_ARENA_MAX=2, old vs new server code, on a Linux box:
    • ten 99k-char texts: +3.76GB → +0.43GB, same duration
    • 128 × 2k-char texts (the most a single app request can carry): +896MB → +405MB, slightly faster
    • 2000 × 128-char texts: unchanged (+136MB → +133MB)
  • Compared per-index results old vs new across a mixed batch (texts of 0, 3, 50, 2k, 45k, 60k, 99k, 100k, 100,001 and 150k chars plus 600 short ones), an empty list, all-long texts and empty strings, with and without an entity list: identical
  • py_compile on apps/pii/server.py

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing (new tests pass the test-audit authoring gate)
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

A batch of many texts each under NER_WINDOW_CHARS still went through one
nlp.pipe pass, and that pass costs memory for every character in it, like one
long text. Pack the short texts into batches of at most NER_WINDOW_CHARS in
total and take the NER slot per batch.
@vercel

vercel Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Nov 16, 2026 9:41pm UTC

Request Review

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 1 file

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Turn on auto-fix | Re-trigger cubic

@greptile-apps

greptile-apps Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 4/5

[Medium impact] The change affects memory use in the PII batch analysis and redaction paths.

The PR appears safe to merge, with a non-blocking gap in automated coverage for the new batch limit.

Findings

  1. P2 Batch limit has no test ▶

Summary

_analyze_many now splits short texts into batches totaling at most NER_WINDOW_CHARS.

  • Batch PII analysis keeps each NER pass under 100,000 characters.

Reviews (1) · Last reviewed commit: "fix(pii): cap each batched NER pass at o..." · Reviewed by Greptile

Comment thread apps/pii/server.py Resolved
@waleedlatif1
waleedlatif1 merged commit b525e27 into staging Nov 16, 2026
42 checks passed
@waleedlatif1
waleedlatif1 deleted the fix/pii-batch-char-bound branch November 16, 2026 22:50

This branch was previously deployed

1 inactive deployment
Preview — 0dacda3f Deployed Nov 16, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant