Skip to content

VDE_MODE only derived from project config, not from state #5887

Open

Description

@davem-bis

Steps to Reproduce

  1. The prod environment is initially planned with vde_mode = All
  2. vde_mode is set to dev_only
  3. A plan of a dev environment will assume that prod is set as physical models, and perform table copy operations as if this is the case, despite prod still being in vde_mode = All.

For Databricks, this can cause errors as SQLMesh attempts to clone the prod environment assuming it is a table, and fails with [DELTA_CLONE_UNSUPPORTED_SOURCE] as the prod model is still a view at this point.

Location in Code

The VDE mode is set in the self.config object of the loader, which is set as self.context.configs[self.config_path] in sqlmesh/core/loader.py::Loader::__init__.

Proposed Solution

When in a plan or run, VDE mode should be derived from both the config, and the current state of the prod environment in the target gateway. Potentially a warning should be raised when they do not agree.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions