Skip to content

ci: add report-only Trivy, govulncheck and yarn npm audit scans - #3735

Open
jordan-simonovski wants to merge 2 commits into
mainfrom
jordansimonovski/vuln-scanning
Open

jordan-simonovski wants to merge 2 commits into
mainfrom
jordansimonovski/vuln-scanning

Conversation

@jordan-simonovski

Copy link
Copy Markdown
Contributor

Summary

CI now reports known vulnerabilities in our Docker images, the collector's Go code and our npm dependencies, using free tools only. Findings appear in each run's job summary, and runs on main and the daily schedule also send them to the Security tab. Nothing fails a build on findings yet, because main already has some to work through.

  • Trivy reports fixable HIGH and CRITICAL CVEs in the collector, app and all-in-one images whenever docker-build.yml builds them.
  • yarn npm audit covers production dependencies in every workspace, plus all of @hyperdx/cli, which bundles its devDependencies into the release.
  • govulncheck reports only the vulnerabilities the migrate tool's code actually reaches. It checks against the Go release the Dockerfile builds with, not go.mod's minimum version.
  • The dependency scans run on PRs that touch dependency files, on every push to main, and daily, so new advisories against dependencies that haven't changed still show up.
  • No breaking changes. This only adds CI steps, none of which can fail a build on findings.

Key decisions

  • Report-only for now. The baseline on main is 6 high npm advisories in production deps, 2 reachable Go vulnerabilities in the migrate tool, and HIGH findings in the collector and all-in-one images. The app image is clean. Gating today would turn every PR red.
  • PR runs write to the job summary but don't upload SARIF. An upload adds a code scanning check that fails on any alert missing from main's last analysis, so a Trivy DB refresh alone could turn an unrelated PR red.
  • The all-in-one scan doesn't use .vex/openssl-mongodb.vex.json. Its statements name the Docker Scout product, which Trivy can't match on a locally built image, so it suppressed nothing.
  • tj-actions/changed-files in docker-build.yml is now pinned by SHA, because those jobs now hold security-events: write. The new trivy-action and upload-sarif steps are pinned the same way.

How to reproduce

On this branch:

  1. Open the "Build OTel Collector Image" job on this PR (the scan action is in its path filter) and check the job summary for the Trivy table.
  2. Open the "Dependency Scan" workflow run and check its summary for both audits and the govulncheck report.
Implementation detail
  • The Trivy steps live in .github/actions/trivy-scan and run with continue-on-error and a 10 minute timeout, so a Trivy DB outage can't fail or stall an image build.
  • load: true is now set on the collector and app builds so Trivy can read the image from the local daemon.
  • govulncheck covers cmd/migrate only. Trivy covers the OCB-built collector binary by matching the Go modules embedded in it.
  • I checked each command locally against main and the 2-nightly images, and ran actionlint and zizmor on the workflows.

Image, Go and npm vulnerability findings now show up in each run's job summary and, from main and the daily schedule, in the GitHub Security tab. Nothing fails a build on findings yet, because main already has some.
@jordan-simonovski
jordan-simonovski requested a review from a team as a code owner November 19, 2026 23:36
@vercel

vercel Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

2 Skipped Deployments
Project Deployment Actions Updated
hyperdx-oss Ignored Ignored Preview Nov 20, 2026 12:03am UTC
hyperdx-storybook Ignored Ignored Preview Nov 20, 2026 12:03am UTC

Request Review

@changeset-bot

changeset-bot Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 0416abe

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@github-actions github-actions Bot added the review/tier-2 Low risk — AI review + quick human skim label Nov 19, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🔵 Tier 2 — Low Risk

Small, isolated change with no API route or data model modifications.

Why this tier:

  • Standard feature/fix — introduces new logic or modifies core functionality

Review process: AI review + quick human skim (target: 5–15 min). Reviewer validates AI assessment and checks for domain-specific concerns.
SLA: Resolve within 4 business hours.

Stats
  • Production files changed: 1
  • Production lines changed: 45
  • Branch: jordansimonovski/vuln-scanning
  • Author: jordan-simonovski

To override this classification, remove the review/tier-2 label and apply a different review/tier-* label. Manual overrides are preserved on subsequent pushes.

@github-actions

Copy link
Copy Markdown
Contributor

Duplicate code analysis

No new duplicate code introduced by this PR.


jscpd compared this branch against origin/main
across app, api and common-utils. Only new clones are listed; the codebase
currently holds 108 in total (1.13%).
Run yarn dupes locally to reproduce, or yarn dupes:all for the full picture.

security-events: write
defaults:
run:
working-directory: packages/otel-collector

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 P2 — Bumping Go in the Dockerfile doesn't trigger govulncheck on the PR

govulncheck takes its Go version from docker/otel-collector/Dockerfile, but that file isn't in the pull_request path filter. A PR that changes only the golang base image (a toolchain bump that changes stdlib advisories) gets no Dependency Scan run. The change only shows up after merge or on the daily schedule. Add docker/otel-collector/Dockerfile to paths.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 0416abe: docker/otel-collector/Dockerfile is now in the path filter, and the version step fails with an error if it can't parse the Go version.

@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

PR Review

✅ No issues found.

@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Deep Review

✅ No critical issues found. This is a report-only CI change (composite Trivy action + new dependency-scan workflow + scan steps wired into docker-build.yml). The security model is sound: the trigger is pull_request (not pull_request_target), every SARIF upload is gated if: github.event_name != 'pull_request', and no untrusted input reaches a shell sink, so the new security-events: write grants are not exploitable by fork PRs under default settings. The prior review feedback (Dockerfile in the path filter; fail-loud Go-version parse) is confirmed resolved. The findings below are all robustness and observability gaps in the new scanning paths — none block a build.

🟡 P2 — recommended

  • .github/workflows/dependency-scan.yml:110 — The govulncheck -format sarif ./... > raw.sarif line is unguarded under bash -eo pipefail and the upload has no non-empty check, so an operational scanner error (module-load or proxy outage) aborts the step before jq runs, leaving govulncheck.sarif absent while the continue-on-error upload refreshes nothing and the job stays green.
    • Fix: Suppress the exit with || true on the SARIF generation and gate the upload on a test -s packages/otel-collector/govulncheck.sarif check that emits a ::warning when the file is missing.
    • correctness, reliability, adversarial, testing
  • .github/actions/trivy-scan/action.yml:36 — The composite converts and uploads trivy.sarif with no existence or non-empty assertion, so a Trivy DB/registry outage fails trivy convert mid-summary, leaving a green job and a truncated summary block with no warning that the image was never scanned.
    • Fix: Assert trivy.sarif is non-empty before the upload step and emit a ::warning when the scan produced no output, so lost coverage is visible even though the build intentionally stays green.
    • adversarial, reliability, testing
  • .github/workflows/dependency-scan.yml:23 — The concurrency group ${{ github.workflow }}-${{ github.ref }} with cancel-in-progress: true collides the daily scheduled run and push-to-main runs (both resolve to refs/heads/main), so a push can cancel the in-flight daily scan and silently skip the advisory sweep it exists to provide.
    • Fix: Key the group on github.event_name as well, or set cancel-in-progress: ${{ github.event_name == 'pull_request' }}, mirroring docker-build.yml.
    • reliability, adversarial, maintainability
  • .github/workflows/docker-build.yml:117 — Only the otel-collector-image job's changed-files filter watches .github/actions/trivy-scan/**; the app-image and all-in-one-image filters omit it, so a PR that edits only the shared scan action rebuilds and scans one of three images and leaves app/all-in-one scan regressions unexercised until after merge.
    • Fix: Add .github/actions/trivy-scan/** to the changed-files path filters of the app-image and all-in-one-image jobs.
    • adversarial, maintainability
🔵 P3 nitpicks (3)
  • .github/workflows/dependency-scan.yml:90 — The Setup Go and Install govulncheck steps lack continue-on-error, so a transient Go proxy or toolchain-download outage fails the whole report-only job red instead of degrading to a warning.
    • Fix: Add continue-on-error: true to both steps; the scan step already captures a missing binary via || status=$?.
  • .github/workflows/dependency-scan.yml:80 — The Go-version sed is coupled to the exact FROM golang:<ver>-... AS migrate-builder format; benign Dockerfile edits (a trailing comment, a hyphen-less or build-arg tag) trip the exit 1 guard and fail the report-only job.
    • Fix: Loosen the regex to tolerate optional suffixes and comments, or source the version from a single shared variable rather than re-parsing the Dockerfile.
  • .github/actions/trivy-scan/action.yml:42 — The pinned github/codeql-action/upload-sarif SHA is duplicated here and in dependency-scan.yml, so a one-sided bump drifts the two upload steps onto different versions.
    • Fix: Note the shared pin in a comment referencing the other call site, or centralize the upload behind the composite action.

Reviewers (8): correctness, security, reliability, adversarial, maintainability, testing, project-standards, previous-comments.

Testing gaps: No fixture exercises the jq SARIF path-rewrite against a representative govulncheck payload, so schema drift would silently mislocate code-scanning alerts. No check asserts generated SARIF files exist and are non-empty before upload. No verification that a scanner hard-failure (not just findings) leaves the dependent builds green.

No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond

@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

E2E Test Results

✅ All tests passed • 470 passed • 1 skipped • 1117s

Status Count
✅ Passed 470
❌ Failed 0
⚠️ Flaky 12
⏭️ Skipped 1

Tests ran across 4 shards in parallel.

View full report →

Its Go version now comes from the collector Dockerfile, so a PR that bumps it re-runs the scan, and a FROM line the parse can't read fails with a clear error instead of scanning against the runner's Go.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review/tier-2 Low risk — AI review + quick human skim

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants