Repository navigation
feat(docker): run the all-in-one image on DocumentDB instead of MongoDB 4.0.5 - #3734
jordan-simonovski wants to merge 1 commit into
Conversation
…DB 4.0.5 MongoDB 4.0.5 is EOL, SSPL licensed, and comes from Alpine 3.9 packages that carry OpenSSL 1.1.1k critical CVEs. The all-in-one image now runs DocumentDB v1.2-0 (MIT, Postgres 18) from the same Dockerfile stage the dev, integration and e2e stacks use. Metadata stays on the /data/db mount under /data/db/pgdata; the changeset documents moving old MongoDB data.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
🦋 Changeset detectedLatest commit: 6ff89f8 The changes in this PR will be included in the next version bump. This PR includes changesets to release 3 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
🔴 Tier 4 — CriticalTouches authentication, tenancy data models, the public API or shipped database config — or substantially changes the query rendering engine, background tasks, the OTel pipeline, image build, or release CI. Why this tier:
Review process: Deep review from a domain expert. Synchronous walkthrough may be required. Stats
|
Duplicate code analysisNo new duplicate code introduced by this PR. jscpd compared this branch against |
| FROM alpine:3.24 AS documentdb | ||
| COPY --from=documentdb-builder /out/ / | ||
| RUN apk add --no-cache bash openssl su-exec $(cat /etc/documentdb/apk-packages) \ | ||
| && addgroup -S -g 70 postgres && adduser -S -u 70 -G postgres -h /var/lib/postgresql postgres \ |
There was a problem hiding this comment.
🔴 P1 — Pinning postgres to UID/GID 70 breaks existing dev DocumentDB volumes
The old docker/documentdb/Dockerfile ran addgroup -S postgres && adduser -S ..., which on alpine:3.24 picks the first free system ID (100 or above), not 70. docker-compose.dev.yml:21 keeps /data/docdb on a host bind mount (.volumes/docdb_dev_<slot>), so a dev who already ran #3375 has a PGDATA full of files owned by the old UID. docker/documentdb/entrypoint.sh:9 (install -d -o postgres ...) only re-owns the top directory. The .documentdb-initialised marker skips initdb, so pg_ctl running as UID 70 gets permission denied on PG_VERSION/global/* and the db container never becomes healthy. Fix: in the root branch of the entrypoint, re-own the contents when the owner differs, e.g. [ "$(stat -c %u "$PGDATA")" = "$(id -u postgres)" ] || chown -R postgres:postgres "$PGDATA", run before the install -d.
Nothing blocks merge automatically, but a maintainer will expect this fixed if it is a real defect in code this PR changes. If it is about surrounding code, reply and say so instead of patching. Do not widen the PR. How to respond
| fi | ||
|
|
||
| # Start DocumentDB (Postgres + Mongo wire-protocol gateway on :10260) | ||
| DOCDB_USER=hyperdx DOCDB_PASSWORD=hyperdx PGDATA=/data/db/pgdata \ |
There was a problem hiding this comment.
🟡 P2 — In the all-in-one image the DocumentDB entrypoint's clean-shutdown trap never fires, so Postgres is SIGKILLed on every docker stop
docker/documentdb/entrypoint.sh:51-52 relies on receiving SIGTERM to run pg_ctl stop ("so persistent volumes skip crash recovery"). In the all-in-one image, PID 1 is sh /etc/local/entry.sh, which sets no trap and does not forward signals. docker stop therefore waits out its timeout and SIGKILLs everything, and Postgres runs crash recovery on each boot. Fix: add a trap in entry.local.base.sh that forwards TERM/INT to the background DocumentDB PID (and the others) and waits for them, or run the services under a small init such as tini.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
PR Review
4 finding(s): 🔴 1 P0/P1 · 🟡 1 P2 · 🔵 2 P3 2 posted as inline comment(s) on the changed lines. 2 P3 folded at the bottom. 🔵 2 P3 — no action needed before merge
Priority comes from a separate grading pass over the finished review. P3 is folded here instead of posted inline. No finding blocks merge automatically; a maintainer decides. |
Deep Review✅ No critical issues found. This change swaps bundled MongoDB 4.0.5 for DocumentDB (Postgres 18 + Mongo wire gateway) in the all-in-one image. The high-risk paths hold up: the first-boot init deletes only 🟡 P2 -- recommended
🔵 P3 nitpicks (3)
Reviewers (10): correctness, security, adversarial, reliability, data-migrations, maintainability, testing, project-standards, learnings-researcher, previous-comments. Testing gaps:
No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond |
E2E Test Results✅ All tests passed • 474 passed • 1 skipped • 1107s
Tests ran across 4 shards in parallel. |
Summary
The all-in-one image now runs DocumentDB (MIT, Postgres 18) in place of MongoDB 4.0.5. That removes the SSPL licence and the four critical OpenSSL 1.1.1k CVEs that came with the Alpine 3.9 MongoDB packages. It uses the same DocumentDB build that local dev, integration and e2e have run on since #3375.
/data/dbmount the docs tell people to use, now under/data/db/pgdata.hyperdx:hyperdxcredentials are fixed. The old mongod was also localhost-only.Breaking changes
/data/dband are not read. The container logs a warning on first boot when it finds them.mongodumpwith the 4.0 tools, thenmongorestoreinto the new container. They were tested from an image built onmainto this branch, and every document and index came across.Implementation detail
docker/documentdb/Dockerfileinto stages ofdocker/hyperdx/Dockerfile. The composedbservice builds thedocumentdbtarget and the all-in-one copies from the same builder, so the two can't drift.apk audit --systemis clean in both images.lddover Postgres, the gateway and the extensions, and fails if the base image's Alpine release is missing a library.scripts/ci/smoke-all-in-one.shpasses, and the migration was run end to end. amd64 and the auth image were not built locally.How to reproduce
Starting point: a checkout of this branch with Docker.
On
main:docker run --rm --entrypoint mongod docker.clickhouse.com/clickhouse/clickstack-all-in-one:2 --versionprints MongoDB 4.0.5.On this branch:
docker buildx build . -f docker/hyperdx/Dockerfile --target all-in-one-noauth --build-context clickhouse=./docker/clickhouse --build-context otel-collector=./docker/otel-collector --build-context hyperdx=./docker/hyperdx --build-context api=./packages/api --build-context app=./packages/app -t aio:local --loadscripts/ci/smoke-all-in-one.sh aio:local. It should printcanary ingested end-to-end.mainimage with-v vol:/data/db, then follow the steps in.changeset/aio-documentdb.md.References