Transform Product Components
Transform consists of three key components: (1) MetricFlow, (2) the Transform App, and (3) the APIs. The most important is MetricFlow, which is a metrics layer where an organization will define the key metrics that will be viewed, explored, and queried in the Transform app and in downstream BI tools.
The objective of MetricFlow is to define and manage the logic used to build a company's metrics. MetricFlow is an opinionated set of abstractions, created in YAML, that enable the data consumer to retrieve metric datasets efficiently from a data warehouse. There are a few key principles:
- Flexible, but complete - Define logic in abstractions that are flexible enough to construct any metric on any data model
- Don't Repeat Yourself (DRY) - Whenever possible enable the definition of metrics
- Simple with progressive complexity - Rely on known concepts and structures in data modeling to ensure that MetricFlow is approachable. Create escape hatches that enable folks to pursue more advanced and unsupported features.
- Performant and Efficient- Enable performance optimizations of centralized data engineering while enabling distributed definition and ownership of logic.
From a technical perspective, there are two critical objects to the metric definition in MetricFlow: the Data Source and Metric. Understanding these will allow you to configure metrics and dimensions in a way that will minimize redundancy while maximizing the potential of MetricFlow on top of your data.
Metrics are built on top of Data Sources in most cases and are defined as a function of Measures. Data Sources together produce a data model that can then be compiled and consumed through Transform's various APIs which then constructs SQL on top of the various data sources to resolve metrics at various granularities.
The Transform App is the UI by which Transform surfaces the metrics defined in MetricFlow to an organization.
A user might consume metrics by first visiting a metric page in Transform. They can use the metric page as a jumping off point for visualizing and doing further analysis by using the Query feature which allows users to explore existing queries or create their own queries from scratch. They can also build content like Boards to both monitor and share the pieces of data they care about.
Technical and non-technical users should access the Transform App in order to glean insights around metric governance, metric definitions (in code and human-readable formats), and for the ability to do exploratory analysis on how a metric behaves. Users can also discuss metric changes by asking questions of metric owners and can provide context around a metric's behavior by adding annotations directly to the metric.
MetricFlow exposes a GraphQL API with a number of clients built on top of that including:
- Command Line Interface (CLI) to pull data locally and improve the dev workflow
- SQL over JDBC to integrate with BI Tools and other SQL Clients
- Python Library to pull metrics into Jupyter or other analytical interfaces
- GraphQL interface which underlies all of these and is also exposed for the end-user to build their own interfaces