Repository navigation
feat(mcp): maxBodyChars mines only the start of each body in event_patterns and emerging_signals - #3742
feat(mcp): maxBodyChars mines only the start of each body in event_patterns and emerging_signals#3742bcanvural wants to merge 2 commits into
Conversation
…tterns and emerging_signals Refs ClickHouse#3650
🦋 Changeset detectedLatest commit: 80279b2 The changes in this PR will be included in the next version bump. This PR includes changesets to release 3 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
@bcanvural is attempting to deploy a commit to the HyperDX Team on Vercel. A member of the Team first needs to authorize it. |
| `Template is longer than ${MAX_PREDICATE_TEMPLATE_LENGTH} characters, so ` + | ||
| 'whereSnippet is empty. Do not search with it; it would match every row.', | ||
| 'whereSnippet is empty. Do not search with it; it would match every row. ' + | ||
| 'Pass maxBodyChars to mine a shorter prefix of each body.', |
There was a problem hiding this comment.
🟡 P2 — New drilldownUnavailable hint tells callers to pass maxBodyChars even on paths that cannot accept it
The hint is now added to every long template, but runEventPatterns is also called for EventPatterns dashboard tiles in packages/api/src/mcp/tools/query/helpers.ts:547. That call never forwards maxBodyChars, and the tile tools have no such parameter, so an agent there is told to do something it can't. The same hint also appears when the caller already passed a maxBodyChars above 4000. Fix: only add the sentence when maxBodyChars is accepted on that path, e.g. a canTruncate option set by the eventPatterns tool. When it was already set, say to lower it below MAX_PREDICATE_TEMPLATE_LENGTH instead.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
There was a problem hiding this comment.
Fixed in 80279b2: I removed the sentence. Dashboard tiles also reach this code and can't pass maxBodyChars, and the descriptions of clickstack_event_patterns and clickstack_emerging_signals already point to it.
PR Review
2 finding(s): 🔴 0 P0/P1 · 🟡 0 P2 · 🔵 2 P3 No inline comments. 2 P3 folded at the bottom. 🔵 2 P3 — no action needed before merge
Priority comes from a separate grading pass over the finished review. P3 is folded here instead of posted inline. No finding blocks merge automatically; a maintainer decides. |
Deep Review✅ No critical issues found. This is a small, well-contained change. 🔵 P3 nitpicks (2)
Reviewers (9): correctness, security, testing, maintainability, project-standards, performance, api-contract, kieran-typescript, previous-comments. Testing gaps:
No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond |
Dashboard tiles also reach runEventPatterns and cannot pass maxBodyChars; the tool descriptions already point to it.
| } | ||
| // Truncate in the query, so long bodies are not transferred, and in the | ||
| // drill-down predicate, so it matches the prefix the template came from. | ||
| const maxBodyChars = options?.maxBodyChars; |
There was a problem hiding this comment.
I see
maxBodyChars: z
.number()
.int()
.min(1)
.max(1_000_000)
.optional()
What is the scenario where we need this additional check? is this something we can enforce via typescript and downstream zod schemas as opposed to a runtime check?
Summary
Closes #3650. Adds a
maxBodyCharsoption toclickstack_event_patternsandclickstack_emerging_signals, as agreed in the issue.Long error logs (multi-KB stack traces, request dumps) are slow to mine: every sampled row's full body is read from ClickHouse, Drain is slow on them, and templates over 4,000 characters get no drill-down
whereSnippet. Often the most frequent error pattern is the one that can't be drilled into.With
maxBodyChars: N, the body column becomesleftUTF8(<body>, N):whereSnippetmatches events by the prefix the template came from;summary.bodyColumnshows the expression used.It works on top of
bodyExpression(a column, a map reference orJSONExtractString). The value must be an integer from 1 to 1,000,000;mineWindowPatternschecks it again before it goes into SQL. ThedrilldownUnavailablehint for long templates now points tomaxBodyChars.How to reproduce
Starting point:
make dev-int.On
main:queryTool.int.test.ts,emergingSignalsTool.int.test.ts).make dev-int FILE="mcp/__tests__/(queryTool|emergingSignalsTool) --testNamePattern=maxBodyChars": both fail, because the option is ignored and the full body is mined.On this branch:
make dev-int FILE="mcp/__tests__/(queryTool|emergingSignalsTool)"passes (126 tests).How to test on Vercel preview
N/A — non-UI change.
References
bodyExpression: allow truncating the body (substring/left) for pattern mining #3650bodyExpression)