dbt Core: dbt 0.21.0

## dbt Core 0.21.0 - Louis Kahn (October 4, 2021) - 💬 [Discourse: v0.21.0](https://discourse.getdbt.com/t/3077) - 📖 [New & changed documentation](https://docs.getdbt.com/docs/guides/migration-guide/upgrading-to-0-21-0) ### Breaking changes - Rename source freshness command, add full node selection, and align selection syntax with other tasks ([#2987](https://github.com/dbt-labs/dbt/issues/2987), [#3554](https://github.com/dbt-labs/dbt/pull/3554)) - `dbt source snapshot-freshness` -> `dbt source freshness` (backwards compatible) - `dbt source freshness --select source_name` --> `dbt source freshness --select source:source_name` (_not_ backwards compatible) - **dbt-snowflake:** Disable most transactional logic. See note below for details. ### Features - Add `dbt build` command to run models, tests, seeds, and snapshots in DAG order. ([#2743](https://github.com/dbt-labs/dbt/issues/2743), [#3490](https://github.com/dbt-labs/dbt/issues/3490), [#3608](https://github.com/dbt-labs/dbt/issues/3608)) - Skip downstream nodes during the `build` task when a test fails. ([#3597](https://github.com/dbt-labs/dbt/issues/3597), [#3792](https://github.com/dbt-labs/dbt/pull/3792)) - Update cli and rpc flags for the `build` task to align with other commands (`--resource-type`, `--store-failures`) ([#3596](https://github.com/dbt-labs/dbt/issues/3596), [#3884](https://github.com/dbt-labs/dbt/pull/3884)) - Introduce `on_schema_change` config to detect and handle schema changes on incremental models ([#1132](https://github.com/dbt-labs/dbt/issues/1132), [#3387](https://github.com/dbt-labs/dbt/issues/3387)) - Capture changes to macros in `state:modified`. Introduce new `state:` sub-selectors: `modified.body`, `modified.configs`, `modified.persisted_descriptions`, `modified.relation`, `modified.macros` ([#2704](https://github.com/dbt-labs/dbt/issues/2704), [#3278](https://github.com/dbt-labs/dbt/issues/3278), [#3559](https://github.com/dbt-labs/dbt/issues/3559)) - Enable setting configs in schema files for models, seeds, snapshots, analyses, tests ([#2401](https://github.com/dbt-labs/dbt/issues/2401), [#3616](https://github.com/dbt-labs/dbt/pull/3616)) - Make `--models` and `--select` synonyms, except for `ls` (to preserve existing behavior) ([#3210](https://github.com/dbt-labs/dbt/pull/3210), [#3791](https://github.com/dbt-labs/dbt/pull/3791)) - Customize `ls` task JSON output by adding new flag `--output-keys` ([#3778](https://github.com/dbt-labs/dbt/issues/3778), [#3395](https://github.com/dbt-labs/dbt/issues/3395)) - Added default field in the `selectors.yml` to allow user to define default selector ([#3448](https://github.com/dbt-labs/dbt/issues/3448), [#3875](https://github.com/dbt-labs/dbt/issues/3875), [#3892](https://github.com/dbt-labs/dbt/issues/3892)) - Log tests that are not indirectly selected. Add `--greedy` flag to `test`, `list`, `build` and `greedy` property in yaml selectors ([#3723](https://github.com/dbt-labs/dbt/pull/3723), [#3833](https://github.com/dbt-labs/dbt/pull/3833)) - Scrub environment secrets prefixed with `DBT_ENV_SECRET_` from logs and console output ([#3440](https://github.com/dbt-labs/dbt/pull/3440), [#3617](https://github.com/dbt-labs/dbt/pull/3617)) #### Artifacts - Added timing and thread information to `sources.json` artifact ([#3804](https://github.com/dbt-labs/dbt/issues/3804), [#3894](https://github.com/dbt-labs/dbt/pull/3894)) - Bump artifact schema versions for 0.21.0 ([#3945](https://github.com/dbt-labs/dbt/pull/3945)) ### Fixes - Fix type coercion issues when fetching query result sets ([#2984](https://github.com/dbt-labs/dbt/issues/2984), [#3499](https://github.com/dbt-labs/dbt/pull/3499)) - Handle whitespace after a plus sign on the project config ([#3526](https://github.com/dbt-labs/dbt/pull/3526)) - Fix table and view materialization issue when switching from one to the other ([#2161](https://github.com/dbt-labs/dbt/issues/2161)), [#3547](https://github.com/dbt-labs/dbt/pull/3547)) - Avoid caching schemas for tests when `store_failures` is not enabled ([#3715](https://github.com/dbt-labs/dbt/issues/3715), [#3716](https://github.com/dbt-labs/dbt/pull/3716)) - `dbt debug` shows a summary of whether all checks passed or not ([#3831](https://github.com/dbt-labs/dbt/issues/3831), [#3832](https://github.com/dbt-labs/dbt/issues/3831)) - Fix bug with initializing a dataclass that inherits from `typing.Protocol`, specifically for `dbt.config.profile.Profile` ([#3843](https://github.com/dbt-labs/dbt/issues/3843), [#3855](https://github.com/dbt-labs/dbt/pull/3855)) - Introduce a macro, `get_where_subquery`, for tests that use `where` config. Alias filtering subquery as `dbt_subquery` instead of resource identifier ([#3857](https://github.com/dbt-labs/dbt/issues/3857), [#3859](https://github.com/dbt-labs/dbt/issues/3859)) - Use `group by column_name` in `accepted_values` test for compatibility with most database engines ([#3905](https://github.com/dbt-labs/dbt/issues/3905), [#3906](https://github.com/dbt-labs/dbt/pull/3906)) - Avoid infinite recursion in `state:modified.macros` check ([#3904](https://github.com/dbt-labs/dbt/issues/3904), [#3957](https://github.com/dbt-labs/dbt/pull/3957)) - Cast log messages to strings before scrubbing of prefixed env vars ([#3971](https://github.com/dbt-labs/dbt/issues/3971), [#3972](https://github.com/dbt-labs/dbt/pull/3972)) ### Under the hood - Improve default view and table materialization performance by checking relational cache before attempting to drop temp relations ([#3112](https://github.com/dbt-labs/dbt/issues/3112), [#3468](https://github.com/dbt-labs/dbt/pull/3468)) - Move the example project used by `dbt init` into `dbt` repository, to avoid cloning an external repo ([#3005](https://github.com/dbt-labs/dbt/pull/3005), [#3474](https://github.com/dbt-labs/dbt/pull/3474), [#3536](https://github.com/dbt-labs/dbt/pull/3536)) - Better interaction between `dbt init` and adapters. Avoid raising errors while initializing a project ([#2814](https://github.com/dbt-labs/dbt/pull/2814), [#3483](https://github.com/dbt-labs/dbt/pull/3483)) - Update `create_adapter_plugins` script to include latest accessories, and stay up to date with latest dbt-core version ([#3002](https://github.com/dbt-labs/dbt/issues/3002), [#3509](https://github.com/dbt-labs/dbt/pull/3509)) - Allow the default seed macro's SQL parameter, `%s`, to be replaced by dispatching a new macro, `get_binding_char()`. This enables adapters with parameter marker characters such as `?` to not have to override `basic_load_csv_rows`. ([#3622](https://github.com/dbt-labs/dbt/issues/3622), [#3623](https://github.com/dbt-labs/dbt/pull/3623)) - Add `adapter_unique_id` to invocation context in anonymous usage tracking, to better understand dbt adoption ([#3713](https://github.com/dbt-labs/dbt/issues/3713), [#3796](https://github.com/dbt-labs/dbt/issues/3796)) - Specify `macro_namespace = 'dbt'` for all dispatched macros in the global project, making it possible to dispatch to macro implementations defined in packages. Dispatch `generate_schema_name` and `generate_alias_name` ([#3456](https://github.com/dbt-labs/dbt/issues/3456), [#3851](https://github.com/dbt-labs/dbt/issues/3851)) #### Packages - Better dbt hub registry packages version logging that prompts the user for upgrades to relevant packages ([#3560](https://github.com/dbt-labs/dbt/issues/3560), [#3763](https://github.com/dbt-labs/dbt/issues/3763), [#3759](https://github.com/dbt-labs/dbt/pull/3759)) - Alert users on package rename ([hub.getdbt.com#180](https://github.com/dbt-labs/hub.getdbt.com/issues/810), [#3825](https://github.com/dbt-labs/dbt/pull/3825)) - Retry transient GitHub failures during download ([#3546](https://github.com/dbt-labs/dbt/pull/3546), [#3729](https://github.com/dbt-labs/dbt/pull/3729)) #### Performance - Experimental parser now detects macro overrides of ref, source, and config builtins. ([#3581](https://github.com/dbt-labs/dbt/issues/3866), [#3582](https://github.com/dbt-labs/dbt/pull/3877)) - Add performance regression testing [#3602](https://github.com/dbt-labs/dbt/pull/3602) - Don't reload and validate schema files if they haven't changed ([#3563](https://github.com/dbt-labs/dbt/issues/3563), [#3888](https://github.com/dbt-labs/dbt/issues/3888)) #### Dependencies - Require `werkzeug>=1` ([#3590](https://github.com/dbt-labs/dbt/pull/3590)) ### Plugins #### dbt-postgres - Add optional `sslcert`, `sslkey`, and `sslrootcert` profile arguments to the Postgres connector. ([#3472](https://github.com/dbt-labs/dbt/pull/3472), [#3473](https://github.com/dbt-labs/dbt/pull/3473)) - Add `connect_timeout` profile configuration for Postgres and child adapters. ([#3581](https://github.com/dbt-labs/dbt/issues/3581), [#3582](https://github.com/dbt-labs/dbt/pull/3582)) #### dbt-redshift - Introduce `ra3` profile configuration on Redshift to support cross-db sources ([#3236](https://github.com/dbt-labs/dbt/issues/3236), [#3408](https://github.com/dbt-labs/dbt/pull/3408)) #### dbt-snowflake - **Breaking:** Turn off transactions and turn on `autocommit` by default. Within dbt materializations, wrap DML statements in explicit `begin` and `commit`. Note that it is not recommended to run DML statements outside of dbt materialization logic. If you do this, despite our recommendation, you will need to wrap those statements in explicit `begin` and `commit`. Note also that this may affect user-space code that depends on transactions, such as pre-hooks and post-hooks that specify `transaction: true` or `transaction: false`. We recommend removing those references to transactions. ([#2748](https://github.com/dbt-labs/dbt/issues/2748), [#3480](https://github.com/dbt-labs/dbt/pull/3480), [#3510](https://github.com/dbt-labs/dbt/pull/3510)) - Fix batching for large seeds on Snowflake ([#3941](https://github.com/dbt-labs/dbt/issues/3941), [#3942](https://github.com/dbt-labs/dbt/pull/3942)) #### dbt-bigquery - Enhance BigQuery copy materialization ([#3570](https://github.com/dbt-labs/dbt/issues/3570), [#3606](https://github.com/dbt-labs/dbt/pull/3606)): - to simplify config (default usage of `copy_materialization='table'` if is is not found in global or local config) - to let copy several source tables into single target table at a time. ([Google doc reference](https://cloud.google.com/bigquery/docs/managing-tables#copying_multiple_source_tables)) - Add support for `execution_project` on BigQuery through profile configuration ([#3707](https://github.com/dbt-labs/dbt/issues/3707), [#3708](https://github.com/dbt-labs/dbt/issues/3708)) - Support BigQuery-specific aliases `target_dataset` and `target_project` in snapshot configs ([#3694](https://github.com/dbt-labs/dbt/issues/3694), [#3834](https://github.com/dbt-labs/dbt/pull/3834)) - Separated table vs view configuration for BigQuery since some configuration is not possible to set for tables vs views. ([#3682](https://github.com/dbt-labs/dbt/issues/3682), [#3691](https://github.com/dbt-labs/dbt/issues/3682)) #### dbt-rpc - Fix for RPC requests that raise a RecursionError when serializing Undefined values as JSON ([#3464](https://github.com/dbt-labs/dbt/issues/3464), [#3687](https://github.com/dbt-labs/dbt/pull/3687)) - Add `build` RPC method, and a subset of flags for `build` task ([#3595](https://github.com/dbt-labs/dbt/issues/3595), [#3674](https://github.com/dbt-labs/dbt/pull/3674)) ### Contributors Thank you all for your contributions to dbt! 🎉 - [@kostek-pl](https://github.com/kostek-pl) ([#3236](https://github.com/dbt-labs/dbt/pull/3408)) - [@matt-winkler](https://github.com/matt-winkler) ([#3387](https://github.com/dbt-labs/dbt/pull/3387)) - [@tconbeer](https://github.com/tconbeer) [#3468](https://github.com/dbt-labs/dbt/pull/3468)) - [@JLDLaughlin](https://github.com/JLDLaughlin) ([#3473](https://github.com/dbt-labs/dbt/pull/3473)) - [@jmriego](https://github.com/jmriego) ([#3526](https://github.com/dbt-labs/dbt/pull/3526)) - [@danielefrigo](https://github.com/danielefrigo) ([#3547](https://github.com/dbt-labs/dbt/pull/3547)) - [@xemuliam](https://github.com/xemuliam) ([#3606](https://github.com/dbt-labs/dbt/pull/3606)) - [@sungchun12](https://github.com/sungchun12) ([#3759](https://github.com/dbt-labs/dbt/pull/3759)) - [@dbrtly](https://github.com/dbrtly) ([#3834](https://github.com/dbt-labs/dbt/pull/3834)) - [@swanderz](https://github.com/swanderz) [#3623](https://github.com/dbt-labs/dbt/pull/3623) - [@JasonGluck](https://github.com/JasonGluck) ([#3582](https://github.com/dbt-labs/dbt/pull/3582)) - [@joellabes](https://github.com/joellabes) ([#3669](https://github.com/dbt-labs/dbt/pull/3669), [#3833](https://github.com/dbt-labs/dbt/pull/3833)) - [@juma-adoreme](https://github.com/juma-adoreme) ([#3838](https://github.com/dbt-labs/dbt/pull/3838)) - [@annafil](https://github.com/annafil) ([#3825](https://github.com/dbt-labs/dbt/pull/3825)) - [@AndreasTA-AW](https://github.com/AndreasTA-AW) ([#3691](https://github.com/dbt-labs/dbt/pull/3691)) - [@Kayrnt](https://github.com/Kayrnt) ([3707](https://github.com/dbt-labs/dbt/pull/3707)) - [@TeddyCr](https://github.com/TeddyCr) ([#3448](https://github.com/dbt-labs/dbt/pull/3865)) - [@sdebruyn](https://github.com/sdebruyn) ([#3906](https://github.com/dbt-labs/dbt/pull/3906)) ----- ## Louis Kahn (1901–1974) > Thanks to @grantwinship for writing this biography of Louis Kahn—a famous Philadelphian, and the namesake of this dbt Core release. The architecture of Louis Kahn inimitably and improbably unites the forces of mass and light. His often hefty concrete structures embrace weighty, intentional interior spaces, huge shapes and curves, and angled baths of light to create a take on mid-century brutalism that is suffused with a utopian optimism and playfulness. Born in the Russian Empire, in what is now Estonia, Itze-Leib Schmuilowsky emigrated to Philadelphia in 1906 at age four. He became a US citizen in 1914, and changed his name, along with his family, to Louis Kahn in 1915. A talented artist from a young age, he repeatedly won Philadelphia's annual award for best watercolor by a high school student, but turned down a full scholarship to the Pennsylvania Academy of the Fine Arts to pursue architecture at the University of Pennsylvania. There he studied a restrained variant of the Beaux-Arts school with Paul Philippe Cret, who influenced the young Kahn greatly in his dedication to grand form and proportion, if not the style's signature ornamentation. Kahn would, after achieving international renown, return to Philadelphia to teach at the University of Pennsylvania from 1957 until his death, becoming the Paul Philippe Cret Professor of Architecture. First though, he cut his teeth taking public housing projects in Philadelphia, many of which remain unbuilt, but are revered by students of architecture. It was a trip abroad through the Mediterranean in 1950, seeing the ruins and monuments of Egypt and Greece again as an established architect, that drove him to push his personal style further. He returned and built an extension to Yale University's Art Gallery that immediately established him as unique voice in American architecture. He would over the next decade and a half design many other works of genius, chief among them the Salk Institute in La Jolla, California and what many consider to be his masterwork, the Jatiya Sangsad Bhaban government building that serves as the capitol of Bangladesh. Both buildings' balance of massive concrete geometries with the light, space, and environment of their locations create a breathtaking aesthetic that is simultaneously ancient and futuristic. Experiencing either of them, the immense impact that Kahn has had on modern architecture is immediately apparent.