Snowplow Normalize v0.4.0

Summary

This version adds new features including Apache Spark with Iceberg support, flexible timestamp partitioning, and compatibility with the new BigQuery loader naming conventions.

Features

• Added support for Apache Spark with Iceberg
• Introduced configurable partition timestamp column name for derived tables
• Added compatibility with new BigQuery loader naming conventions

Upgrading

To upgrade simply bump the snowplow-normalize version in your packages.yml file.

If you want to start using the new partition timestamp feature, you can configure the column name in your project configuration:

vars:
  snowplow__partition_tstamp: your_custom_column_name

You have to be sure that the field you add is included in the table, and you do a full refresh.

Snowplow Normalize v0.3.5

Summary

This release adds support for schema grants

Features

• Add support for schema grants

Under the hood

• Enforce full refresh flag to refresh manifest tables

Upgrading

To upgrade simply bump the snowplow-normalize version in your packages.yml file. Note the minimum version of snowplow-utils required is now 0.16.2

Snowplow Normalize v0.3.4

Summary

This version bumps the package dependency to add support for the latest snowplow utils package. Please note that from this version onwards this package is under the SPAL license.

Under the hood

• Bump support for latest utils
• Migrate to SPAL license

Upgrading

To upgrade simply bump the snowplow-normalize version in your packages.yml file.

Snowplow Normalize v0.3.3

Summary

• Include the new base macro functionality from utils in the package
• Allow users to specify the timestamp used to process events (from the default of collector_tstamp)

Under the hood

• Simplify the model architecture

Upgrading

Bump the snowplow-normalize version in your packages.yml file.

Snowplow Normalize v0.3.2

Summary

Bumps the max supported snowplow-utils version to allow usage with our other packages.

Upgrading

Bump the snowplow-normalize version in your packages.yml file.

Snowplow Normalize v0.3.1

Summary

This version bumps the requirement of the jsonschema package to validate schemas with the MultipleOf property.

Fixes

• Bump jsonschema minimum version (Close #33)

Upgrading

To upgrade the package, bump the version number in the packages.yml file in your project.

Snowplow Normalize v0.3.0

Summary

This version migrates our models away from the snowplow_incremental_materialization and instead move to using the built-in incremental with an optimization applied on top.

🚨 Breaking Changes 🚨

Changes to materialization

To take advantage of the optimization we apply to the incremental materialization, users will need to add the following to their dbt_project.yml :

# dbt_project.yml
...
dispatch:
  - macro_namespace: dbt
    search_order: ['snowplow_utils', 'dbt']

For custom models please refer to the snowplow utils migration guide and the latest docs on creating custom incremental models.

Features

• Migrate from get_cluster_by and get_partition_by to get_value_by_target_type
• Migrate all models to use new materialization

Docs

• Update readme

Upgrading

Bump the snowplow-normalize version in your packages.yml file, and ensuring you have followed the above steps. You can read more in our upgrade guide

Snowplow Normalize v0.2.3

Summary

This release allows users to disable the days late data filter to enable normalizing of events that don't populate the dvce_sent_tstamp field.

Features

• Allow disabling of days late filter by setting snowplow__days_late_allowed to -1 (#28)

Upgrading

To upgrade the package, bump the version number in the packages.yml file in your project.

Snowplow Normalize v0.2.2

Summary

This release fixes an issue with column aliasing in BigQuery when the schema order differs from the table in the warehouse. It also adds the ability to alias your user_id column and add flat atomic.events columns into your users table.

Features

• Fix column alias ordering issue in BigQuery
• Add ability to alias user_id column
• Add ability to add flat columns to the events table

Under the hood

• Alter github pages publishing action

Upgrading

To upgrade the package, bump the version number in the packages.yml file in your project. To use the new features please see our docs for the new values to add to your config file.

Snowplow Normalize v0.2.1

Summary

This release fixes the expected path for a private registry to require it to end with /api in line with other resolvers. It also upgrades the schema requirement for the resolver to 1.0.3 so you can use newer resolver files without issues.

Features

• Fix private registry uri requirement
• Bump iglu resolver schema version

Under the hood

• Fix github pages publishing action

Upgrading

To upgrade the package, bump the version number in the packages.yml file in your project.