Skip to content

SQLMesh Projects

Quick Summary

SQLMesh projects in dagctl are containerized SQLMesh deployments that sync with your Git repository and manage environment variables and credentials. State management is handled through either a managed Postgres instance or your own connection string.

Setting Recommended Why
Branch main Standard default branch
SQLMesh Environment prod For production deployments
SQLMesh Version Latest (0.227.1) Latest features & bug fixes
Managed State DB Enabled No external database setup needed
SSH Authentication For private repos Set up during project creation

Key Takeaways

  • ✅ Use managed state database unless you need external state management - simplifies setup
  • ✅ Use the latest SQLMesh version for new projects - get the newest features and fixes
  • ✅ Environment variables can be shared across projects or scoped to individual projects

Creating Projects

Create projects through the web UI. You'll need:

  1. Git repository URL (HTTPS or SSH)
  2. Branch name (usually main)
  3. Environment variables (warehouse credentials)

You will be prompted to create an SSH key to connect to your private Git repositories during project creation.

Project Lifecycle

  1. Create - Define repository and settings
  2. Deploy - Deploy to dagctl (creates all required infrastructure)
  3. Sync - Continuously sync with Git repository
  4. Monitor - View deployment status and logs
  5. Undeploy - Remove from execution environment

State Database

SQLMesh requires a state database to track model versions, environments, and interval completion. dagctl provides two options.

dagctl provisions a Postgres instance for your project automatically. No external setup required. The database is scoped to your organization and isolated per project.

Enable managed state during project creation by selecting Managed State DB. dagctl handles provisioning, credentials injection, and connection string configuration.

Self-Managed State Connection

If you need to use your own database (for data residency or existing infrastructure reasons), provide a connection string as an environment variable during project creation.

# Example: connect to your own Postgres instance
SQLMESH__STATE_CONNECTION__TYPE=postgres
SQLMESH__STATE_CONNECTION__HOST=your-db-host.example.com
SQLMESH__STATE_CONNECTION__DATABASE=sqlmesh_state
SQLMESH__STATE_CONNECTION__USER=sqlmesh
SQLMESH__STATE_CONNECTION__PASSWORD=your-password

Warning

Self-managed state databases must be accessible from the dagctl execution cluster. Ensure your database allows inbound connections from the cluster's egress IP range. Contact support for the current IP range for your region.

Local Development

The dagctl CLI connects your local environment to the project's state database. This lets you run sqlmesh plan and sqlmesh run locally against the same state used in production.

# Connect your local environment to a dagctl project's state
dagctl use-project <project-name>

After running use-project, your local sqlmesh commands use the project's configured state connection. This is required to create plans that dagctl can execute - both your local environment and the execution cluster must point at the same state database.

SQLMesh Version Configuration

dagctl supports pinning your project to a specific SQLMesh version. This controls which version of the sqlmesh library runs inside the execution pod.

Set the version during project creation or update it in Project Settings > SQLMesh Version.

Version strategy When to use
Latest New projects, non-critical workloads
Pinned (e.g., 0.220.0) Production projects requiring stability

Warning

Upgrading SQLMesh versions on an existing project can require a state migration. Test version upgrades in a dev environment first. Downgrading is not supported once a migration has run.

The version setting maps directly to the Docker image tag used for your execution pods. If a version is unavailable, project deployment will fail with an image pull error.

Next Steps