Skip to content

sqlmesh init -t dlt / dlt_refresh does not recognise dlt ducklake destination #5914

Open

Description

@michaldrobena

Bug report

When using dlt with a ducklake destination, SQLMesh fails to initialise/refresh from the dlt pipeline metadata.

I see:

Error: Unknown connection type 'ducklake'


Environment

  • SQLMesh: 0.236.0
  • dlt: 1.26.0
  • DuckDB: (installed as dependency of the above)
  • Python: 3.13.x
  • OS: Debian 13 (trixie), x86_64

Minimal reproducible example

1) Create clean environment

mkdir sqlmesh-ducklake-mre && cd sqlmesh-ducklake-mre
python -m venv .venv
source .venv/bin/activate
pip install "sqlmesh==0.236.0" "dlt[duckdb,ducklake]==1.26.0"

2) Create simple dlt pipeline with DuckLake destination

Create a run_pipeline.py file and paste in the following:

import dlt

pipeline = dlt.pipeline(
    pipeline_name="mre_ducklake",
    destination=dlt.destinations.ducklake,
    dataset_name="mre",
)

pipeline.run([{"id": 1, "name": "a"}], table_name="items")
print("done")

Run the pipeline

python run_pipeline.py

3) Run SQLMesh dlt integration command

See the docs here

Running this command:

# sqlmesh init -t <template> --pipeline-name <pipeline-name> --dlt-path <pipelines-directory> dialect
sqlmesh init -t dlt --pipeline-name mre_ducklake --dlt-path ~/.local/share/dlt/pipelines duckdb

Generates this error:

Error: Unknown connection type 'ducklake'.

Notes

This appears to be destination-type mapping logic in the dlt integration path, where dlt.destinations.ducklake is present in dlt pipeline state but not recognised by SQLMesh. I do not experience this issue where dlt.destinations.ducklake is present the dlt pipeline.

Activity

  1. kingjaiteh commented on Sep 7, 2026

    @kingjaiteh
    Contributor

    I Reproduced with dlt 1.30.0 and sqlmesh main: sqlmesh init -t dlt passes the dlt destination name straight through as the SQLMesh connection type, and ducklake is not one.

    I have a fix that generates a duckdb gateway with the lake attached as a ducklake catalog (path, data_path and metadata_schema mirror dlt's own ATTACH for duckdb, sqlite, postgres/mysql and MotherDuck catalogs). With it, sqlmesh plan against the generated project reads the lake. Opening a PR now, will adjust if you'd rather map it differently plz lmk

  2. AnnasMazhar commented on Oct 4, 2026

    @AnnasMazhar

    I reproduced this on main (849d6ed). A `ducklake` destination falls through the generic credentials path and produces an invalid gateway config, which then surfaces from `connection.py` as `ConfigError: Unknown connection type 'ducklake'.` — the same message raised for a genuinely unknown type, so the real diagnostic is lost.

    DuckLake is a DuckDB gateway with the lake attached as a catalog, so that block needs to be built explicitly. While fixing it I also found that the config scalars (catalog database, storage URL, name, metadata schema) were interpolated into the YAML unquoted, so a path containing a leading quote or a newline produced invalid YAML and the generated project would not load.

    I have a fix with tests ready and will open a PR shortly. One scope note for that PR: unsupported catalog drivers now raise a clear `ClickException`, and I have documented the remaining generic case (any other unrecognised `db_type` still raises the generic `ConfigError` at that boundary) rather than widening this change — happy to widen it here if you would rather I did.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions