Repository navigation
Conversation
The kernel backend (use_kernel=True) ignored the Thrift-style _port and _connection_uri overrides. Resolve both connector-side onto the kernel's single host and http_path, with the Thrift rules: - _port (1..65535; 0 or None mean the default) folds into server_hostname as host:port. The default 443 adds nothing, IPv6 literals are bracketed, and an embedded port equal to _port is accepted. An empty server_hostname with _port raises ValueError. - _connection_uri (http(s)://host[:port]/path?query) replaces host, port, path and query, and wins over _port. Scheme-less, non-http(s), path-less and credential-bearing URIs raise ValueError. URI-only use works. - With OAuth (U2M, Azure service principal, JWT key file, or M2M client id and secret), a _connection_uri whose origin differs from https://server_hostname raises before a kernel session opens. PAT is not affected. A non-default _port with OAuth is not guarded: the token exchange goes to the same host:port, which is documented. - The SPOG x-databricks-org-id header follows the resolved path. Tests cover the resolver tables, the OAuth origin guard, the SPOG re-derivation, and a real-wheel loopback dial proving the kernel reaches the requested port over TLS for _port and for _connection_uri. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
The helper returned Optional[str] for the resolved path, but KernelDatabricksClient takes a str, so mypy fails in check-types on every supported Python. Session.__init__ already types http_path as str, so this is an annotation-only change with no runtime effect. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
There was a problem hiding this comment.
🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).
Verdict: 2 Low
Looks good: two low concerns. Resolver, OAuth origin guard and SPOG re-derivation match Thrift's precedence and have thorough table tests. Nit: a few new lines (the Conflicting-ports f-string, the _split_host and _uri_host_and_path docstrings) exceed the 100-char limit in CONTRIBUTING.md.
There was a problem hiding this comment.
🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).
Verdict: 2 Low
Looks good, with 1 low concern. The resolver, the OAuth origin guard and the SPOG re-derivation match the auth bridge's routing and are well covered by tests. Nit: a few new lines go past the 100-char limit (the Conflicting ports f-string and the _split_host/_uri_host_and_path docstrings), and the AC-P4 ticket reference in the docstring.
This overview is Claude Opus 5.5's initial take; GPT-6.1 Sol added findings it doesn't mention — see the findings below.
A kernel connection without _port or _connection_uri again passes server_hostname through unchanged, so odd hosts no longer raise on the default path. The OAuth origin guard now skips connections the kernel auth bridge routes to PAT: an access_token with no JWT or client-secret M2M credentials, even when auth_type is databricks-oauth. azure-sp-m2m and M2M/JWT stay guarded. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
There was a problem hiding this comment.
🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).
Verdict: 1 Low
Looks good, with 1 low concern. The resolver, the OAuth predicate (which matches the auth bridge's routing) and the SPOG re-derivation all check out. Nits: session.py:135 and the _split_host/_uri_host_and_path docstrings go past the 100-char limit, and test_session.py imports databricks.sql twice.
Wraps the origin-guard condition to the repo's black line length (check-linting). The guard no longer treats a custom credentials_provider as OAuth: the kernel auth bridge rejects that provider with NotSupportedError, so no OAuth origin is involved. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
…ernel port rules Check the host for a path, query or fragment before the port is parsed, so 'h.io/path:8443' with _port cannot read as port 8443. Only explicit _port is affected; a host without _port passes through unchanged, as on main. Add Thrift tests pinning that host and _port reach ThriftDatabricksClient unchanged. Update CONNECTION_PARAMETERS.md: _port and _connection_uri are now kernel options, with the embedded-port and falsy rules and the OAuth origin rule. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
Replace the _HOSTPORT_RE regex, the _port regex and the manual bracketing in the kernel host folding with urllib.parse.urlsplit, int() and an ipaddress pre-check for bare IPv6 literals. The original host string still passes through unchanged unless a port is folded in; folded IPv6 hosts are bracketed.
_check_oauth_origin treats an access_token that decodes as a JWT whose issuer is not server_hostname as OAuth. The kernel wraps every provider with the connection host, so such a token would be exchanged at the _connection_uri origin, while Thrift exchanges it at server_hostname. A mismatched origin now raises before any request. PATs and same-host issuers are unaffected.
The port-conflict warning is skipped when _port is the default 443, matching Node. Non-default ports still warn on a mismatch.
Behavior-table tests in test_session.py pass unchanged. One warning test changed: test_embedded_port_wins_over_differing_explicit_port lost its ('h.io:8443', 443) row, which asserted the 443 warning. That case moved to test_embedded_port_with_default_443_does_not_warn under the new rule.
Known differences from bc7c09e: the OAuth origin check now compares the parsed hostname, so a server_hostname such as 'h.io/x' or 'user@h' with a matching _connection_uri origin passes where it was previously refused. The error text for 'h:70000', 'h:abc' and '[::1' is now one generic malformed-or-out-of-range message. 'user:pass@h' with _port is still refused.
Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
_is_bare_ipv6 now returns early unless the text has at least two colons and no leading bracket. ip_address raised a ValueError for every host name and host:port string, which made the folding path about 2x slower than bc7c09e. A bare IPv6 literal always has two or more colons, so behavior is unchanged; the session tests and the realkernel loopback tests pass. Microbenchmark, median us per call (20000 calls x 7 repeats): - bare host, _port 8443: 1.305 base, 1.261 new - host:port, _port 8443: 1.364 base, 1.393 new (+2%) - IPv6 bracketed, _port 8443: 1.340 base, 1.355 new (+1%) - IPv6 bare, _port 8443: 1.413 base, 2.642 new (+87%, the successful ip_address parse) Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
Remove _is_bare_ipv6 and the ipaddress import. Bare IPv6 hosts such as ::1 never worked on the kernel path: the kernel prepends https:// and its URL parser rejects an unbracketed IPv6 literal. IPv6 literals must be bracketed ([::1]).
With _port set, urlsplit reads everything after the first colon of a bare IPv6 literal as the port. That text always contains another colon, so the integer parse fails and the existing "malformed IPv6 literal or port" ValueError is raised before any request. This includes _port=443. Without _port the host still passes through unchanged, as on main.
Tests: the ("::1", 8443) -> "[::1]:8443" row is removed. ::1 with _port 8443 or 443, and 2001:db8::1 with _port 8443, are now in the reject table.
Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
There was a problem hiding this comment.
🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).
✅ No issues identified.
🔀 1 finding considered and dismissed by the judge
- 🟡 Medium
src/databricks/sql/session.py:240— An access_token combined with identity_federation_client_id bypasses the origin guard when its issuer matches server_hostname, although the bridge forwards that
Judge's evidence:if routed_to_pat or not oauth:
mypy (check-types) rejected passing the Optional access_token to _is_foreign_jwt, because bool(access_token) does not narrow its type. Use `access_token and _is_foreign_jwt(...)` inside bool(), which narrows it. The behavior is the same; the session tests pass. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
There was a problem hiding this comment.
🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).
Verdict: 1 Low
Looks good, no blocking issues. The host/port/URI resolution, the OAuth-origin guard (which follows the auth_bridge.py routing order) and the SPOG re-derivation all check out. Nit: no test covers a caller-set x-databricks-org-id combined with _connection_uri, and test_resolved_kernel_host_and_path_reach_kernel_client never closes its connection.
This overview is Claude Opus 5.5's initial take; GPT-6.1 Sol added findings it doesn't mention — see the findings below.
_redact kept everything before the first "://" as the scheme. In a scheme-less _connection_uri such as "h.io/sql/x?token=secret&redirect=https://example.com", that prefix is the path and query, so the error message echoed token=secret. The prefix is now kept only when it is http or https; otherwise it is shown as <redacted>. Two regression rows in test_invalid_connection_uri_message_omits_credentials fail without this change. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
There was a problem hiding this comment.
🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).
Verdict: 1 Medium · 1 Nit
Looks good, no blocking issues. I checked the host/port/URI resolution, redaction, the SPOG header re-derivation, and the OAuth origin guard's PAT-routing logic against auth_bridge.kernel_auth_kwargs (azure-sp-m2m, then JWT, then M2M, then PAT, then U2M), and they are consistent. Thrift passthrough is unchanged and covered by tests.
This overview is Claude Opus 5.5's initial take; GPT-6.1 Sol added findings it doesn't mention — see the findings below.
Other findings
- 🟡 Medium — An access_token paired with identity_federation_client_id bypasses the origin guard as PAT, although the bridge forwards that selector for mandatory token exchange, allowing _connection_uri to redirect that exchange to another origin. Treat an explicit federation client ID as exchange-bearing auth, exclude it from the PAT exemption, and add a mismatched-origin regression test. (Found by GPT-6.1 Sol · the judge couldn't confirm it)
- ⚪ Nit — The test opens a kernel connection and never closes it, unlike test_spog_org_id_follows_resolved_path. Append .close() to the connect() call. (Found by Claude Opus 5.5 · confirmed by the judge)
…is set With identity_federation_client_id, the kernel bridge forwards the access_token for mandatory workload-identity token exchange at the connection host. _check_oauth_origin routed any access_token that wasn't a foreign-issuer JWT to the PAT exemption, so a mismatched _connection_uri origin could receive that exchange, where Thrift exchanges at server_hostname. Such a token now counts as OAuth for the origin check, so a mismatched origin raises before any request. A plain PAT without a federation client id is unaffected. Also close the connection in test_resolved_kernel_host_and_path_reach_kernel_client, and describe the exchanged-token rule in CONNECTION_PARAMETERS.md. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
|
Both findings from the latest review are addressed in 973b4e2.
The session tests (200), kernel client tests (282), mypy on Automated reply from the driver feature factory (Isaac). Not reviewed by a human. |
There was a problem hiding this comment.
🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).
✅ No issues identified.
🔀 1 finding considered and dismissed by the judge
- 🔵 Low
src/databricks/sql/session.py:259—_check_oauth_originreuses_split_host, which raises when_portis used with a malformed host. So OAuth plus a valid_connection_urifails with "malform
Judge's evidence:f"Invalid server_hostname {_redact(host)!r}: malformed IPv6 literal or port"
Summary
On the kernel backend (
use_kernel=True),_portand_connection_uriare now honored the way Thrift honors them. Before this PR the kernel path ignored both. Thrift is unchanged.Behavior
_portis added to a bareserver_hostname.server_hostnamewins. If_portis set to a different non-443 value, the driver logs a warning._portmeans the default._portraisesValueError; Thrift fails on it too._connection_uri(http(s)://host[:port]/path?query) sets the host, port and path, and wins over_port. A URI-only connection works._connection_uriwhose origin differs fromserver_hostnameraises before any request. OAuth here includes anaccess_tokenthat is a JWT from another issuer, or any token sent withidentity_federation_client_id. PAT is unaffected.urllib.parse, and error messages redact credentials.Testing
test_session.py(200) andtest_kernel_client.py(282) pass, plus the real-kernel loopback tests. mypy and black are clean._port, and URI-only all pass.Note
This overlaps databricks-sql-python#915, another engineer's PR that changes the same function. Only one of the two should merge.
Related PRs: databricks/databricks-sql-nodejs#636, databricks/databricks-sql-go#581, https://github.com/databricks/databricks-driver-test/pull/2188
This pull request and its description were written by Isaac.