Repository navigation
feat(modal): add Modal Labs integration - #6896
Merged
Merged
Conversation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a Modal block with three operations: Call Function, Chat Completion, and List Models.
Why only three operations
Modal has no public REST control plane. The Python, JS, and Go SDKs all speak gRPC to
api.modal.com, and their complete docs (llms-full.txt) contain no REST API section — so Sandboxes, Apps, Volumes, and the rest of Modal's primitives simply aren't reachable from an HTTP-only tool. What is reachable is deployed Web Functions/Servers, and the OpenAI-compatible Endpoints API. That's what this covers.{endpoint}/v1/chat/completions{endpoint}/v1/models(defaults to the Shared Endpoint host)Notes for review
Auth sends the proxy-token pair as
Modal-Key/Modal-Secretrather than the equivalent combinedAuthorization: Bearer wk-….ws-…. That leavesAuthorizationfree for a Web Function that validates its own bearer token. Required for Endpoints (always authenticated), optional for Web Functions (unauthenticated by default).Both URL fields require https. Modal terminates TLS for
.modal.run,.modal.direct, and custom domains alike, so a cleartext URL is always a misconfiguration — and one that would put the proxy token on the wire in the clear.request.modelInputis on Chat Completion only. The system prompt and user message provably reach a model, so they project to canonical placeholders before egress. Call Function deliberately has no provenance hook: a Web Function runs arbitrary user code, and nothing proves its body reaches a model. Adding one there would be inference, not evidence.One thing I could not verify. Modal's docs show the
/v1/modelscurl and say it lists "model IDs", but never print the response body.data[].idis directly supported by that wording;object,created, andowned_byare inferred from OpenAI compatibility, so they're markedoptional. Happy to tighten them against a live payload if someone has a token.Behaviour worth knowing
application/jsonbut returns something else surfaces as raw text instead of failing the call with a bareSyntaxError— that's the user's bug to see, not a reason to lose the response.Testing
38 tests across the block mapping, the URL/auth/body helpers, and
call_function's response handling. The two tests covering the behaviours above were each reverted once and watched go red.Full audit suite (32 checks), lint, and type-check all pass; generated artifacts regenerated and their diffs reviewed — every hunk is additive.
🤖 Generated with Claude Code