Skip to content

feat(modal): add Modal Labs integration - #6896

Merged
icecrasher321 merged 2 commits into
stagingfrom
feat/modal-integration
Aug 20, 2026
Merged

icecrasher321 merged 2 commits into
stagingfrom
feat/modal-integration

Conversation

@icecrasher321

Copy link
Copy Markdown
Contributor

Adds a Modal block with three operations: Call Function, Chat Completion, and List Models.

Why only three operations

Modal has no public REST control plane. The Python, JS, and Go SDKs all speak gRPC to api.modal.com, and their complete docs (llms-full.txt) contain no REST API section — so Sandboxes, Apps, Volumes, and the rest of Modal's primitives simply aren't reachable from an HTTP-only tool. What is reachable is deployed Web Functions/Servers, and the OpenAI-compatible Endpoints API. That's what this covers.

Operation Endpoint
Call Function your deployed Web Function / Server URL
Chat Completion {endpoint}/v1/chat/completions
List Models {endpoint}/v1/models (defaults to the Shared Endpoint host)

Notes for review

Auth sends the proxy-token pair as Modal-Key / Modal-Secret rather than the equivalent combined Authorization: Bearer wk-….ws-…. That leaves Authorization free for a Web Function that validates its own bearer token. Required for Endpoints (always authenticated), optional for Web Functions (unauthenticated by default).

Both URL fields require https. Modal terminates TLS for .modal.run, .modal.direct, and custom domains alike, so a cleartext URL is always a misconfiguration — and one that would put the proxy token on the wire in the clear.

request.modelInput is on Chat Completion only. The system prompt and user message provably reach a model, so they project to canonical placeholders before egress. Call Function deliberately has no provenance hook: a Web Function runs arbitrary user code, and nothing proves its body reaches a model. Adding one there would be inference, not evidence.

One thing I could not verify. Modal's docs show the /v1/models curl and say it lists "model IDs", but never print the response body. data[].id is directly supported by that wording; object, created, and owned_by are inferred from OpenAI compatibility, so they're marked optional. Happy to tighten them against a live payload if someone has a token.

Behaviour worth knowing

  • A body typed for POST and then switched to GET is dropped at the tool layer, so a stale value can't ride along.
  • A function that labels its body application/json but returns something else surfaces as raw text instead of failing the call with a bare SyntaxError — that's the user's bug to see, not a reason to lose the response.
  • All three response reads are capped at 10 MiB.

Testing

38 tests across the block mapping, the URL/auth/body helpers, and call_function's response handling. The two tests covering the behaviours above were each reverted once and watched go red.

Full audit suite (32 checks), lint, and type-check all pass; generated artifacts regenerated and their diffs reviewed — every hunk is additive.

🤖 Generated with Claude Code

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant