Skip to main content
toolsSource-backed
dbt Core logo

dbt Core

Apache-2.0 dbt engine for transforming warehouse data with SQL models, Jinja, YAML configs, tests, documentation, lineage, metadata, and build artifacts.

by dbt Labs · submitted by oktofeesh1·added 2026-06-04·
HarnessCLI
Review first review before installing

Open the source and read safety notes before installing.

Citation facts

Source-backed facts for citing this resource, derived directly from the registry — also available as plain text for AI assistants.

Source URLs
https://docs.getdbt.com/docs/introduction, https://github.com/dbt-labs/dbt-core, https://www.getdbt.com/
Brand
dbt Core
Brand domain
getdbt.com
Brand asset source
brandfetch
Safety notes
dbt runs transformation SQL against a data platform and can create, replace, or mutate warehouse objects, so development and production targets should be separated and permissioned carefully., The current `dbt-labs/dbt-core` README warns that `main` hosts dbt Core v2.0 alpha, that behavior, APIs, and on-disk formats may change, and that dbt Core v1 development has moved to `1.latest`., Version, adapter, package, and artifact compatibility should be pinned and tested before upgrading shared projects or production jobs., Model tests, contracts, lineage, and documentation improve confidence, but they do not replace data review, access controls, warehouse governance, freshness checks, or incident response., Threads, full refreshes, incremental logic, and CI jobs can consume warehouse budget or lock shared resources; teams should set concurrency, timeout, and rollback expectations before broad automation., Profile files and environment variables can contain sensitive warehouse credentials, so `profiles.yml` should stay out of git, logs, generated docs, screenshots, and shared support artifacts.
Privacy notes
dbt workflows can process SQL models, Jinja macros, YAML configs, sources, tests, seeds, snapshots, metrics, exposures, connection profiles, warehouse relation names, logs, and generated artifacts., Command output and `logs/dbt.log` can include invocation arguments, runtime context, thread names, node metadata, warehouse relation identifiers, errors, and other debugging details., dbt artifacts are written to the project's `target/` directory by default and may include manifests, run results, catalogs, source freshness output, semantic manifests, invocation IDs, adapter types, project metadata, and selected environment metadata., The artifacts docs say environment variables prefixed with `DBT_ENV_CUSTOM_ENV_` can be included in artifact metadata, so teams should avoid placing secrets in those variables., The usage-stats docs say dbt telemetry is enabled by default and does not track credentials, raw model contents, or model names; dbt Core users can opt out by setting `send_anonymous_usage_stats` to false or `DO_NOT_TRACK=1`.
Author
dbt Labs
Submitted by
oktofeesh1
Claim status
unclaimed
Last verified
2026-06-04

Decision playbook

Review trust signals before you adopt

Signals are present but mixed. Use the checklist below to confirm the source and operational safety for your environment.

Compare context
Selected

0

Current score

78

Baseline

Delta

No baseline selected

No major trust-signal divergence detected in the current selection.

Source and provenance checks

Complete

Confirm ownership and provenance before trusting install instructions.

  • Source link availableRequired

    Open the canonical repository and verify ownership.

    Done
  • Source provenance statusRequired

    Marked as source-backed.

    Done
  • Metadata reviewed

    Registry metadata indicates a reviewed listing.

    Done

Safety and privacy checks

Complete

Validate risk disclosures before installation or API wiring.

  • Safety notes presentRequired

    Review the listed safety guidance before running commands.

    Done
  • Privacy notes presentRequired

    Review data handling notes before connecting accounts or secrets.

    Done
  • Trust level risk gateRequired

    Trust level does not block evaluation.

    Done

Package and install checks

Needs review

Check package metadata and artifact integrity signals.

  • Install payload available

    Install or copy payload is available for review.

    Done
  • Package verification flag

    No package verification flag provided.

    Pending
  • Checksum metadata

    No checksum provided for downloaded artifact.

    Pending

Compare-driven decision checks

Needs review

Use compare context to validate trade-offs before adoption.

  • Compare tray has multiple entries

    Add at least one more entry to compare trust differences.

    Pending
  • Baseline comparison available

    No baseline peer selected yet.

    Pending
  • Diverging trust signals identified

    No major trust-signal divergence found.

    Pending

Setup at a glance

Copy & paste

Copy-ready — paste the snippet to get started.

Adoption plan

Balanced adoption plan

Current risk score 16/100. Use staged verification before broader rollout.

Risk 16

Pre-adoption checks

Validate source and review signals before any execution.

  • Confirm source provenanceRequired

    Source URL/provenance metadata is present.

    Done
  • Confirm metadata review state

    Listing has review metadata.

    Done
  • Verify install payload

    Install/config payload exists and can be inspected.

    Done

Security checks

Confirm safety, privacy, and package integrity signals.

  • Review safety notesRequired

    Safety notes are present.

    Done
  • Review privacy notesRequired

    Privacy notes are present.

    Done
  • Verify package integrity metadata

    No package verification/checksum metadata.

    Pending

Rollout

Adopt in controlled steps based on the selected plan.

  • Run in isolated sandbox firstRequired

    Use a constrained sandbox and observe behavior across multiple tasks.

    Pending
  • Roll out graduallyRequired

    Roll out to a small cohort before wider usage.

    Pending
  • Set monitoring and fallback

    Define rollback path and monitor errors after adoption.

    Pending

Evidence readiness

Evidence readiness matrix · balanced

Required evidence gates are covered (5/6 signals complete).

Risk 15

Source provenance

Present

Source repository/provenance is listed.

Required in this preset

Metadata review

Present

Review metadata is present.

Required in this preset

Safety notes

Present

Safety notes are present.

Required in this preset

Privacy notes

Present

Privacy notes are present.

Optional in this preset

Package integrity

Missing

Package integrity metadata is missing.

Optional in this preset

Install payload

Present

Install payload is available.

Required in this preset

Required evidence gates are covered for this preset.

Decision timeline

Decision timeline · balanced

5/6 steps complete with no blocking gaps for this preset.

Risk 14

triage

Confirm source provenanceRequired

Source/provenance metadata is available.

Done

triage

Check metadata review statusRequired

Review metadata is available.

Done

verify

Review safety notesRequired

Safety notes are available.

Done

verify

Review privacy notes

Privacy notes are available.

Done

verify

Validate package integrity metadata

Package integrity metadata is missing.

Pending

rollout

Verify install payload and commandsRequired

Install payload is available.

Done

No required blockers for this timeline preset.

Prerequisite readiness

Prerequisite readiness

5 prerequisites to line up before setup. Have accounts and credentials ready first. Includes a review or approval gate.

0/5 ready
Account & credentials1Configuration1Permissions & scopes1Review & approval1General1

Safety & privacy surface

Safety & privacy surface

6 safety and 5 privacy notes across 6 risk areas. Review closely: credentials & tokens, permissions & scopes.

6 areas
  • SafetyPermissions & scopesdbt runs transformation SQL against a data platform and can create, replace, or mutate warehouse objects, so development and production targets should be separated and permissioned carefully.
  • SafetyLocal filesThe current `dbt-labs/dbt-core` README warns that `main` hosts dbt Core v2.0 alpha, that behavior, APIs, and on-disk formats may change, and that dbt Core v1 development has moved to `1.latest`.
  • SafetyGeneralVersion, adapter, package, and artifact compatibility should be pinned and tested before upgrading shared projects or production jobs.
  • SafetyPermissions & scopesModel tests, contracts, lineage, and documentation improve confidence, but they do not replace data review, access controls, warehouse governance, freshness checks, or incident response.
  • SafetyData retentionThreads, full refreshes, incremental logic, and CI jobs can consume warehouse budget or lock shared resources; teams should set concurrency, timeout, and rollback expectations before broad automation.
  • SafetyCredentials & tokensProfile files and environment variables can contain sensitive warehouse credentials, so `profiles.yml` should stay out of git, logs, generated docs, screenshots, and shared support artifacts.
  • PrivacyLocal filesdbt workflows can process SQL models, Jinja macros, YAML configs, sources, tests, seeds, snapshots, metrics, exposures, connection profiles, warehouse relation names, logs, and generated artifacts.
  • PrivacyExecution & processesCommand output and `logs/dbt.log` can include invocation arguments, runtime context, thread names, node metadata, warehouse relation identifiers, errors, and other debugging details.
  • PrivacyLocal filesdbt artifacts are written to the project's `target/` directory by default and may include manifests, run results, catalogs, source freshness output, semantic manifests, invocation IDs, adapter types, project metadata, and selected environment metadata.
  • PrivacyCredentials & tokensThe artifacts docs say environment variables prefixed with `DBT_ENV_CUSTOM_ENV_` can be included in artifact metadata, so teams should avoid placing secrets in those variables.
  • PrivacyCredentials & tokensThe usage-stats docs say dbt telemetry is enabled by default and does not track credentials, raw model contents, or model names; dbt Core users can opt out by setting `send_anonymous_usage_stats` to false or `DO_NOT_TRACK=1`.

Disclosure: editorial

Safety notes

  • dbt runs transformation SQL against a data platform and can create, replace, or mutate warehouse objects, so development and production targets should be separated and permissioned carefully.
  • The current `dbt-labs/dbt-core` README warns that `main` hosts dbt Core v2.0 alpha, that behavior, APIs, and on-disk formats may change, and that dbt Core v1 development has moved to `1.latest`.
  • Version, adapter, package, and artifact compatibility should be pinned and tested before upgrading shared projects or production jobs.
  • Model tests, contracts, lineage, and documentation improve confidence, but they do not replace data review, access controls, warehouse governance, freshness checks, or incident response.
  • Threads, full refreshes, incremental logic, and CI jobs can consume warehouse budget or lock shared resources; teams should set concurrency, timeout, and rollback expectations before broad automation.
  • Profile files and environment variables can contain sensitive warehouse credentials, so `profiles.yml` should stay out of git, logs, generated docs, screenshots, and shared support artifacts.

Privacy notes

  • dbt workflows can process SQL models, Jinja macros, YAML configs, sources, tests, seeds, snapshots, metrics, exposures, connection profiles, warehouse relation names, logs, and generated artifacts.
  • Command output and `logs/dbt.log` can include invocation arguments, runtime context, thread names, node metadata, warehouse relation identifiers, errors, and other debugging details.
  • dbt artifacts are written to the project's `target/` directory by default and may include manifests, run results, catalogs, source freshness output, semantic manifests, invocation IDs, adapter types, project metadata, and selected environment metadata.
  • The artifacts docs say environment variables prefixed with `DBT_ENV_CUSTOM_ENV_` can be included in artifact metadata, so teams should avoid placing secrets in those variables.
  • The usage-stats docs say dbt telemetry is enabled by default and does not track credentials, raw model contents, or model names; dbt Core users can opt out by setting `send_anonymous_usage_stats` to false or `DO_NOT_TRACK=1`.

Prerequisites

  • Choice of dbt Core version and engine path, including dbt Core v1 on the `1.latest` branch or dbt Core v2 alpha on `main` as the Rust-based Fusion foundation.
  • Supported adapter or driver for the selected data platform, warehouse credentials, target schemas, profiles configuration, and environment-specific dev, staging, and production targets.
  • dbt project structure for SQL models, Jinja, YAML configs, sources, seeds, snapshots, tests, documentation, exposures, metrics, macros, packages, and model contracts.
  • Warehouse permission model for creating, replacing, and reading relations, plus cost controls for threads, incremental models, full refreshes, CI builds, and scheduled jobs.
  • Governance plan for generated artifacts, logs, usage stats, environment variables, manifests, run results, catalog output, source freshness, docs hosting, and project metadata.

Schema details

Install type
copy
Troubleshooting
No
Source repository stats
Scope
Source repo
Tool listing metadata
Pricing
open-source
Disclosure
editorial
Application category
DeveloperApplication
Operating system
macOS, Windows, Linux
Full copyable content
## Editorial notes

dbt Core is useful when Claude-adjacent teams need repeatable analytics engineering workflows around warehouse transformations, model dependencies, tests, documentation, metadata artifacts, source freshness, and CI-reviewed SQL changes. It gives data teams a code-first way to build trusted data products from SQL select statements while keeping transformations versioned, testable, reviewable, and documented.

This is distinct from Apache Airflow and Dagster. Airflow schedules and monitors workflow DAGs. Dagster orchestrates assets and operational metadata. dbt Core is the data transformation engine and project framework that compiles, tests, documents, and runs SQL-based transformation graphs in a data platform. It is also distinct from the hosted dbt platform and from optional AI or Fusion workflows, which may have separate product terms.

## Source notes

- The official repository says dbt enables data analysts and engineers to transform data using the same practices that software engineers use to build applications.
- The official docs say dbt transforms raw warehouse data into trusted data products by letting users write SQL select statements while dbt creates modular, maintainable data models.
- The docs say dbt projects create structured context such as lineage, tests, contracts, metrics, and governance.
- The docs describe the dbt framework as a language and engine, with SQL select statements, Jinja templating, YAML configs, tests, and metadata.
- The docs describe dbt Core v1 as the open-source Python-based engine and dbt Core v2 as the open-source foundation for the Fusion engine that is currently in alpha.
- The current repository README warns that dbt Core v1 development has moved to the `1.latest` branch and that `main` hosts dbt Core v2.0 alpha.
- The README says dbt Core v2.0 is Apache-2.0 licensed, built for performance at scale, produces Parquet artifacts, and is distributed as a self-contained binary.
- The `1.latest` README describes dbt models as SQL select statements that build on one another, with relationship management, visualization, and testing.
- The installation docs describe dbt Core as the open-source engine for running dbt locally and distinguish dbt Core v1 from dbt Core v2 alpha.
- The profiles docs say command-line dbt uses `profiles.yml` for data-platform connection details, target definitions, execution parameters, and credential separation.
- The artifact docs describe manifests, run results, catalogs, source freshness, semantic manifests, target-directory output, invocation metadata, adapter type, and environment metadata.
- The events and logs docs describe command-line output and `logs/dbt.log` debug logs with runtime context.
- The usage-stats docs describe default telemetry, stated exclusions for credentials, raw model contents, and model names, and dbt Core opt-out paths.
- The repository is `dbt-labs/dbt-core`, is Apache-2.0 licensed, and is active.

## Duplicate check

Checked current `content/tools/`, `content/mcp/`, agents, hooks, rules, skills, commands, guides, collections, open pull requests, live issue state, and repository-wide content for `dbt Core`, `dbt`, `dbt-labs/dbt-core`, `github.com/dbt-labs/dbt-core`, `docs.getdbt.com`, `analytics engineering`, and `SQL models`. No dedicated dbt Core tools entry, source URL duplicate, target file, issue duplicate, or open duplicate PR was found.

## Disclosure

Editorial listing. No paid placement or affiliate link is used. dbt Core is Apache-2.0 open-source software; dbt platform, dbt Fusion engine, dbt Wizard, dbt Copilot, data platforms, adapters, packages, cloud warehouses, CI systems, and downstream analytics tools may have separate licenses, billing, terms, privacy obligations, and access controls.

About this resource

Editorial notes

dbt Core is useful when Claude-adjacent teams need repeatable analytics engineering workflows around warehouse transformations, model dependencies, tests, documentation, metadata artifacts, source freshness, and CI-reviewed SQL changes. It gives data teams a code-first way to build trusted data products from SQL select statements while keeping transformations versioned, testable, reviewable, and documented.

This is distinct from Apache Airflow and Dagster. Airflow schedules and monitors workflow DAGs. Dagster orchestrates assets and operational metadata. dbt Core is the data transformation engine and project framework that compiles, tests, documents, and runs SQL-based transformation graphs in a data platform. It is also distinct from the hosted dbt platform and from optional AI or Fusion workflows, which may have separate product terms.

Source notes

  • The official repository says dbt enables data analysts and engineers to transform data using the same practices that software engineers use to build applications.
  • The official docs say dbt transforms raw warehouse data into trusted data products by letting users write SQL select statements while dbt creates modular, maintainable data models.
  • The docs say dbt projects create structured context such as lineage, tests, contracts, metrics, and governance.
  • The docs describe the dbt framework as a language and engine, with SQL select statements, Jinja templating, YAML configs, tests, and metadata.
  • The docs describe dbt Core v1 as the open-source Python-based engine and dbt Core v2 as the open-source foundation for the Fusion engine that is currently in alpha.
  • The current repository README warns that dbt Core v1 development has moved to the 1.latest branch and that main hosts dbt Core v2.0 alpha.
  • The README says dbt Core v2.0 is Apache-2.0 licensed, built for performance at scale, produces Parquet artifacts, and is distributed as a self-contained binary.
  • The 1.latest README describes dbt models as SQL select statements that build on one another, with relationship management, visualization, and testing.
  • The installation docs describe dbt Core as the open-source engine for running dbt locally and distinguish dbt Core v1 from dbt Core v2 alpha.
  • The profiles docs say command-line dbt uses profiles.yml for data-platform connection details, target definitions, execution parameters, and credential separation.
  • The artifact docs describe manifests, run results, catalogs, source freshness, semantic manifests, target-directory output, invocation metadata, adapter type, and environment metadata.
  • The events and logs docs describe command-line output and logs/dbt.log debug logs with runtime context.
  • The usage-stats docs describe default telemetry, stated exclusions for credentials, raw model contents, and model names, and dbt Core opt-out paths.
  • The repository is dbt-labs/dbt-core, is Apache-2.0 licensed, and is active.

Duplicate check

Checked current content/tools/, content/mcp/, agents, hooks, rules, skills, commands, guides, collections, open pull requests, live issue state, and repository-wide content for dbt Core, dbt, dbt-labs/dbt-core, github.com/dbt-labs/dbt-core, docs.getdbt.com, analytics engineering, and SQL models. No dedicated dbt Core tools entry, source URL duplicate, target file, issue duplicate, or open duplicate PR was found.

Disclosure

Editorial listing. No paid placement or affiliate link is used. dbt Core is Apache-2.0 open-source software; dbt platform, dbt Fusion engine, dbt Wizard, dbt Copilot, data platforms, adapters, packages, cloud warehouses, CI systems, and downstream analytics tools may have separate licenses, billing, terms, privacy obligations, and access controls.

Source citations

Add this badge to your README

Show that dbt Core is listed on HeyClaude. Paste this Markdown into your README — it renders the badge and links back to this page.

Listed on HeyClaude
[![Listed on HeyClaude](https://heyclau.de/badge/tools/dbt-core.svg)](https://heyclau.de/entry/tools/dbt-core)

How it compares

dbt Core side by side with 3 alternatives on trust, install, platform support, and disclosed safety notes — all from reviewed registry metadata.

Field

Apache-2.0 dbt engine for transforming warehouse data with SQL models, Jinja, YAML configs, tests, documentation, lineage, metadata, and build artifacts.

Open dossier

MIT-licensed embedded analytical SQL database for local OLAP workloads, data files, notebooks, Python and R clients, extensions, and single-file analytics workflows.

Open dossier

Apache-2.0 GX Core Python library for data quality Expectations, validation definitions, checkpoints, Data Docs, metadata stores, and pipeline quality checks.

Open dossier

Apache-2.0 reactive Python notebook stored as pure Python for reproducible experiments, SQL-backed data workflows, script execution, app deployment, and AI-assisted editing.

Open dossier
Next steps
Trust
Review statusReviewedMaintainer reviewedReviewedMaintainer reviewedReviewedMaintainer reviewedReviewedMaintainer reviewed
Package trustPackage not verifiedPackage not verifiedPackage not verifiedPackage not verified
Source provenanceSource-backedSource-backedSource-backedSource-backed
Submitteroktofeesh1oktofeesh1oktofeesh1oktofeesh1
Install riskReview firstReview firstReview firstReview first
Notes Safety ✓ Privacy ✓ Safety ✓ Privacy ✓ Safety ✓ Privacy ✓ Safety ✓ Privacy ✓
Branddbt Core logodbt CoreDuckDB logoDuckDBGreat Expectations logoGreat ExpectationsMarimo logoMarimo
Categorytoolstoolstoolstools
SourceSource-backedSource-backedSource-backedSource-backed
Authordbt LabsDuckDB FoundationGreat ExpectationsMarimo Team
Added2026-06-042026-06-042026-06-042026-06-04
Platforms
Harness
Source repo
Safety notesdbt runs transformation SQL against a data platform and can create, replace, or mutate warehouse objects, so development and production targets should be separated and permissioned carefully. The current `dbt-labs/dbt-core` README warns that `main` hosts dbt Core v2.0 alpha, that behavior, APIs, and on-disk formats may change, and that dbt Core v1 development has moved to `1.latest`. Version, adapter, package, and artifact compatibility should be pinned and tested before upgrading shared projects or production jobs. Model tests, contracts, lineage, and documentation improve confidence, but they do not replace data review, access controls, warehouse governance, freshness checks, or incident response. Threads, full refreshes, incremental logic, and CI jobs can consume warehouse budget or lock shared resources; teams should set concurrency, timeout, and rollback expectations before broad automation. Profile files and environment variables can contain sensitive warehouse credentials, so `profiles.yml` should stay out of git, logs, generated docs, screenshots, and shared support artifacts.DuckDB SQL should be treated like executable code because queries can read and write files, access network resources through extensions, load extensions, consume system resources, and mutate attached databases. Applications that accept user-controlled SQL, file paths, table names, filter expressions, or data-source settings need sandboxing and allowlists rather than passing those values directly into DuckDB operations. Extensions run with the same privileges as the DuckDB process, and community extensions should only be installed from trusted sources after reviewing their maintenance and distribution path. Statements such as `ATTACH`, `COPY`, `EXPORT DATABASE`, `CREATE SECRET`, `INSERT`, `UPDATE`, and `DELETE` can change local files, databases, or connected services when permissions allow it. Analytical queries can use substantial CPU, memory, temporary disk, and object-store bandwidth, so shared automations should configure memory, thread, timeout, temp-directory, and retry expectations. Persistent database files and write-ahead logs need backups, file permissions, and recovery procedures before DuckDB is used for durable or production-adjacent analytical state.GX Core validations can query databases, scan files, evaluate DataFrames, and compute metrics over real datasets, so production runs should use scoped credentials, tested queries, and bounded resources. Checkpoints can trigger Actions such as updating Data Docs, sending notifications, or running custom logic based on Validation Results; notification endpoints and custom Actions should be reviewed before automation. Data Docs generate static human-readable documentation from Expectations, Validation Results, and metadata, so hosted sites and generated folders need access controls before they include sensitive details. Result formats and unexpected-row retrieval can expose row-level failures or sample values; teams should tune result verbosity before publishing results to logs, tickets, chat, or docs sites. Custom Expectations, custom Actions, SQL-based custom Expectations, and orchestration integrations run team-provided code or queries and should be treated as trusted project code. GX Core compatibility depends on Python, data source, integration, and optional dependency support, so upgrades should be tested against the compatibility reference and existing validation suites.Marimo notebooks execute Python and SQL, can write files, query databases, call APIs, access object storage, install packages, and start web servers, so notebooks should be treated as trusted project code. Reactive execution automatically tracks variable dependencies and can run downstream cells after upstream changes; expensive, destructive, or side-effectful cells need lazy runtime, disabled cells, startup autorun, and manual-run policies. The docs note that Marimo tracks variable definitions and references statically, not arbitrary mutations across cells, so mutable shared state should be designed carefully to avoid misleading results. App mode uses `marimo run` to serve notebooks as web apps with code hidden by default, but public deployments still need authentication, authorization, rate limiting, reverse proxy policy, and traceback disclosure review. Disabling token protection, passing access tokens in URLs, or exposing edit servers can give unauthorized users access to notebook execution and should be avoided outside controlled environments. SQL cells can interpolate Python values, query local files and remote databases, and use engines or extensions such as DuckDB, so SQL strings, credentials, object paths, and output destinations should be reviewed before automation. Built-in AI and copilot features may inspect notebook code, prompts, tool context, and referenced variable values; provider selection, API keys, local model behavior, and cost controls should be configured deliberately. Package-management features can serialize requirements and auto-install dependencies into notebook-specific environments, so teams should pin, review, and scan packages before sharing or deploying notebooks.
Privacy notesdbt workflows can process SQL models, Jinja macros, YAML configs, sources, tests, seeds, snapshots, metrics, exposures, connection profiles, warehouse relation names, logs, and generated artifacts. Command output and `logs/dbt.log` can include invocation arguments, runtime context, thread names, node metadata, warehouse relation identifiers, errors, and other debugging details. dbt artifacts are written to the project's `target/` directory by default and may include manifests, run results, catalogs, source freshness output, semantic manifests, invocation IDs, adapter types, project metadata, and selected environment metadata. The artifacts docs say environment variables prefixed with `DBT_ENV_CUSTOM_ENV_` can be included in artifact metadata, so teams should avoid placing secrets in those variables. The usage-stats docs say dbt telemetry is enabled by default and does not track credentials, raw model contents, or model names; dbt Core users can opt out by setting `send_anonymous_usage_stats` to false or `DO_NOT_TRACK=1`.DuckDB workflows can process local files, database files, notebooks, query text, table names, column names, object-store paths, data-frame contents, connection strings, secrets, extensions, and generated result sets. The files-created docs describe global files such as `~/.duckdb_history`, extension directories, and stored persistent secrets, so users should avoid typing credentials or sensitive data into ad hoc SQL history. Persistent secrets are stored under DuckDB's configured secret directory, and `duckdb_secrets()` redacts sensitive fields by default; enabling unredacted secret output is unsafe with untrusted SQL. On-disk databases can create database files, write-ahead logs, and temporary directories next to the database file or working directory, depending on connection mode and configuration. HTTP, S3, and other external-data workflows can expose object-store identifiers, paths, credentials, request metadata, and result data to the connected service and any configured logs or monitoring.GX Core workflows can process source data, schemas, table names, file paths, SQL queries, Batch metadata, Expectation Suites, Validation Results, Checkpoints, Actions, Data Docs, and generated stores. The credentials docs say tokens and connection strings should be stored securely outside version control, using environment variables, uncommitted config files, or supported secrets managers. File Data Context Stores can persist Expectation Suites, Validation Definitions, Checkpoints, Validation Results, and Suite Parameters in project folders or configured backends. Data Docs are static web pages generated from Expectations, Validation Results, and metadata; publishing them can disclose validation outcomes, column names, dataset structure, and failing examples. GX Core tracks analytics events by default, including feature usage, operating system, and Python version, and the docs describe disabling collection with `GX_ANALYTICS_ENABLED` or `analytics_enabled`.Marimo workflows can process notebook source code, cell outputs, variable values, DataFrames, SQL queries, schemas, database rows, object-store paths, generated apps, CLI arguments, logs, and exported artifacts. User configuration can store runtime, server, completion, and AI-provider settings, while app configuration can live inside notebook files; secrets should stay in environment variables or secret stores rather than committed notebooks. AI-assisted editing can send prompts, notebook context, code, schemas, and referenced in-memory values to configured hosted providers, or to local model services when those are selected. Database and remote-storage workflows can expose connection strings, credentials, table names, bucket names, object keys, query text, sample rows, and download paths to notebooks, logs, cloud services, and deployed apps. Token login, query-parameter access tokens, Basic auth headers, reverse-proxy headers, and server logs should be handled as sensitive operational data.
Prerequisites
  • Choice of dbt Core version and engine path, including dbt Core v1 on the `1.latest` branch or dbt Core v2 alpha on `main` as the Rust-based Fusion foundation.
  • Supported adapter or driver for the selected data platform, warehouse credentials, target schemas, profiles configuration, and environment-specific dev, staging, and production targets.
  • dbt project structure for SQL models, Jinja, YAML configs, sources, seeds, snapshots, tests, documentation, exposures, metrics, macros, packages, and model contracts.
  • Warehouse permission model for creating, replacing, and reading relations, plus cost controls for threads, incremental models, full refreshes, CI builds, and scheduled jobs.
  • DuckDB distribution and client choice for the workflow, such as the CLI, Python, R, Java, Node.js, C or C++ APIs, Rust, ODBC, JDBC, or WebAssembly.
  • Data access plan for local DuckDB files, in-memory databases, CSV, Parquet, JSON, Arrow, pandas, R data frames, lakehouse formats, HTTP sources, S3-compatible storage, and mounted working directories.
  • Version, extension, and file-format compatibility policy for shared notebooks, CI jobs, production scripts, persisted database files, and generated analytical artifacts.
  • Resource controls for memory, threads, temporary directories, maximum temporary directory size, checkpointing, write-ahead logs, and long-running analytical queries.
  • Supported Python environment for GX Core, currently Python 3.10 through 3.13, with deployment expectations that do not assume official Windows support.
  • Data Context choice, project layout, version control policy, and environment-specific configuration for development, CI, staging, and production validation workflows.
  • Data Source and Data Asset plan for SQL databases, filesystem data, pandas DataFrames, Spark DataFrames, supported cloud storage, Batch Definitions, and runtime parameters.
  • Expectation Suites, Validation Definitions, Checkpoints, Actions, Data Docs, Stores, result formats, and alerting rules designed around the data quality questions the team actually needs answered.
  • Python environment and package-management plan for the selected notebook, app, script, SQL, visualization, and optional AI features.
  • Notebook execution model for reactive dependency graphs, deterministic cell ordering, lazy or stale runtime behavior, disabled cells, startup autorun, and side-effectful cells.
  • Data access plan for local files, DataFrames, SQL cells, databases, warehouses, cloud object storage, remote filesystems, environment variables, and credentials.
  • Deployment and access-control plan for edit servers, read-only apps, token or password protection, reverse proxies, ASGI middleware, public sharing, rate limits, and error reporting.
Install
Config
Citations
ClaimUnclaimedUnclaimedUnclaimedUnclaimed
Open 4 picks in the interactive comparison tool

Signals

Loading live community signals…

More like this, weekly

A short, calm digest of reviewed Claude resources. Unsubscribe any time.