Skip to content

fix(gitlab): scale group discovery and permission-aware search - #10138

Merged
waleedlatif1 merged 3 commits into
stagingfrom
codex/gitlab-large-groups
Nov 17, 2026
Merged

waleedlatif1 merged 3 commits into
stagingfrom
codex/gitlab-large-groups

Conversation

@waleedlatif1

@waleedlatif1 waleedlatif1 commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

GitLab group sources stopped at 1,000 projects, and searching a large group required walking projects individually. This raises discovery to 50,000 projects and uses native group search for CSV sources, with existing per-project authorization and bounded fallback for unsupported scopes. Administrator-token sources retain their existing project traversal.

CSV uploads and the downloadable exporter now support up to 500,000 data rows and 8 MiB per file, within the existing 10 MiB encoded request limit. The form blocks combined CSV payloads that exceed the encoded request limit before saving. Discovery keeps only bounded project IDs/paths; summary reads avoid loading the full manifest; the exporter writes incrementally and publishes both files only after success. Existing project sources, CSV headers, source IDs, upload endpoints, and revision checks remain compatible.

Product update

Group connections can cover larger namespaces without adding projects individually. GitLab setup and CI documentation describes the limits, search capabilities, continuation behavior, and CI timing. GitLab's own offset/rate limits and the permission payload limits still apply.

Type of Change

  • Bug fix
  • Documentation

Testing

  • Real PostgreSQL and HTTP provider integration tests exercise 50,000 projects, 150,000 grants, broad search reaching the final project, legacy cursors, access denial, confidentiality, scope fallback, and failed-update rollback.
  • HTTPS exporter tests exercise a group above 1,000 projects, missing pagination totals, files above 4 MiB, row/byte caps, and atomic publication after late failures.
  • A form regression test exercises individually valid files whose JSON encoding exceeds the total request limit, then verifies recovery after replacing one file.
  • Focused GitLab tests, repository lint, type-check, generated artifacts, docs compatibility, and audits.
  • Full repository suites run in CI. The scale tests use deterministic provider fixtures; they do not claim a live large-instance GitLab run.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Nov 17, 2026 6:57pm UTC

Request Review

@greptile-apps

greptile-apps Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no actionable defects were established.

Summary

Raises GitLab group discovery to 50,000 projects and adds group-wide search for CSV-backed sources.

  • GitLab group connections can include up to 50,000 projects.
  • CSV-permission group searches use GitLab’s group search when the scope supports it.
  • GitLab permission CSVs can hold more rows and the exporter writes them safely.
  • GitLab permission summaries are read without loading the CSV rows.

No actionable new issues were found. No previous review threads were supplied. The previous-review merge-base check failed, so this review covered the full PR. Tests were inspected but not run.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[CSV-backed group search] --> B[GitLab group search]
  B --> C[Filter saved projects and reader grants]
  B -->|Unsupported scope| D[Bounded project search]
  D --> E[Verify each candidate]
  C --> E
  E --> F[Check current project identity and content rules]
  F --> G[Return permitted results and continuation]
Loading

Reviews (3) · Last reviewed commit: "fix(gitlab): clarify byte limits and use..." · Reviewed by Greptile

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 19 files

This PR changes authentication, authorization, or input validation. Ultrareviews find 2.4x more serious bugs than standard reviews. Comment @cubic-dev-ai ultrareview to run one.

Turn on auto-fix | Re-trigger cubic

Comment thread apps/docs/openapi-v2-resources.json Resolved
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 27 files

Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic

Comment thread apps/docs/openapi-v2-resources.json Resolved
@waleedlatif1
waleedlatif1 force-pushed the codex/gitlab-large-groups branch from 1b94ceb to 1830196 Compare November 17, 2026 18:50
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 27 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Turn on auto-fix | Re-trigger cubic

@waleedlatif1
waleedlatif1 merged commit 41b11cd into staging Nov 17, 2026
63 of 64 checks passed
@waleedlatif1
waleedlatif1 deleted the codex/gitlab-large-groups branch November 17, 2026 18:56

This branch was successfully deployed

1 active deployment
Preview — 1830196e Deployed Nov 17, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant