Transform's Fast Cache is a release candidate feature and not generally available. Please reach out to firstname.lastname@example.org to learn more.
What is the Fast Cache in Transform
The Transform Fast Cache Feature builds on our materialization capabilities. The fast cache stores materializations in a DB-based cache (e.g., MySQL) for faster access. This is in contrast to the normal materializations, which by default store data in your data warehouse, and can have latency of many seconds. This feature is most useful for organizations that need low latency access to metrics. This is in contrast to the normal materializations, which by default store data in your data warehouse and can have a higher latency even for basic select-queries.
The Fast Cache Feature can be thought of as an additional capability on top of the main Materialization Feature.
Fast Cache Components
- Configuration & Definition:The Fast Cache definition occurs in the
materializations objectas a destination.
- Calling a Fast Cache: When the Fast Cache is specified as a destination for a materialization, it will be generated after the table is built using the
mql materializeendpoint of the CLI and Python Interface.
- Writing the Fast Cache table: The Materialization endpoint will write the table to the Fast Cache.
- The metrics you define in the Fast Cache configuration must already be defined in Transform.
- There must be a primary time dimension that is consistently named across your data sources. This time dimension will be used to set the date parameters of the Fast Cache.
- You must have provisioned a database for Transform to use as the Fast Cache. Please refer to our setup documents for more information.
Defining a Fast Cache Destination
Here is an example of a Fast Cache Configuration that reflects data from a booking service company.
name: user_bookings_summary # name your materialization. this will be the table name written in your data warehouse.
description: # add an optional description.
- location: fast_cache
metrics: # list all the metrics you want to materialize in your materialization. all metrics must be defined in your metrics.yaml.
dimensions: # list all corresponding dimensions you want to include for these metrics. these must all be defined in your data sources.
Same as Materializations!
Backfills should be done manually by running
materialize with the backfill dates. If you don't provide a date range, it will materialize the data for all time. This may be expensive and so the CLI will prompt you to confirm.