dbt¶
The dbt adapter needs no extra and does not import dbt. It reads the
target/manifest.json that dbt compile, dbt run and dbt build write,
and runs all SQL through a DuckDB or Postgres adapter connected to the
database dbt built into.
For a dbt project, point the adapter at the manifest that dbt run (or
dbt compile) writes, and at the warehouse dbt built into:
import duckdb
from dataveil.adapters.dbt import DbtAdapter
from dataveil.adapters.duckdb import DuckDBAdapter
from dataveil.core.profile import profile_table
adapter = DbtAdapter("target/manifest.json", DuckDBAdapter(duckdb.connect("shop.duckdb")))
adapter.list_tables() # ['main_analytics.customers', ...]
profile_table(adapter, "main_analytics.customers")
Models are named schema.alias, the relation dbt builds. Ephemeral models
are skipped, because they have no relation. Every classifier works, because
the warehouse adapter registers the checksum functions.
Cleansing is not durable yet
An operation changes the relation dbt built, outside dbt. The next
dbt run rebuilds a table model and undoes it. Views cannot be
cleansed at all. Durable cleansing, written into the model, is planned
for 0.5.