Skip to main content

Deploying MetricFlow or Transform

Transform is a metrics store that allows organizations to execute on their vision for data democratization in real, practical terms. Organizations can define a source of truth for key KPIs, which can be leveraged in a number of different ways across the data stack and in downstream applications.

This source of truth is defined by leveraging MetricFlow, Transform's Open Source underlying metric layer, which allows you to define metrics in code as YAML files.

Please use the following to install MetricFlow and Transform.

MetricFlow

Config setup

Once you have successfully installed MetricFlow with pip install metricflow, run mf setup to get started.

mf setup will prompt a dialogue and seed a template configuration file to {HOME}/.metricflow/config.yml or override the directory with the ENV $MF_CONFIG_DIR.

Data warehouse setup

After running mf setup a configuration file will be created with the appropriate inputs based on the data warehouse you selected.

Snowflake

dwh_dialect: snowflake  # Dialect (one of BigQuery, Snowflake, Redshift)
dwh_host: '' # Snowflake account name
dwh_user: '' # Username for the data warehouse
dwh_password: '' # Password associated with the provided user
dwh_database: ''
dwh_warehouse: '' # Provide the warehouse to use
dwh_schema: ''

Redshift

dwh_dialect: redshift  # Dialect (one of BigQuery, Snowflake, Redshift)
dwh_host: '' # Host name
dwh_port: ''
dwh_user: '' # Username for the data warehouse
dwh_password: '' # Password associated with the provided user
dwh_database: ''
dwh_schema: ''

BigQuery

dwh_dialect: bigquery  # Dialect (one of BigQuery, Snowflake, Redshift)
dwh_password: '' # Provide the path to the BigQuery credential file
dwh_schema: ''

For more details, check here: SQLAlchemy Database Urls.

To ensure that your data warehouse is set up properly with MetricFlow run mf health-checks which will perform all necessary access checks against your data warehouse.

Model setup

After setting up the data warehouse connection, run mf tutorial which will walk through how to define and create an example model as well as using some of the basic commands provided by MetricFlow.

With your models configurations defined, go to the configuration file {HOME}/.metricflow/config.yml and input this field:

model_path: '' # Replace with the path to the directory containing your defined model configurations

After that is done, run mf validate-configs to ensure that the model pass all validation checks.

Now you can test some basic commands! MetricFlow Tutorial

Transform

Creating your Organization

To create your transform organization please reach out to hello@transformdata.io to receive a link to the signup form. Once you've set up your organization, you'll be prompted to complete installation within the Transform app at https://app.transformdata.io/install.

Connecting your Data Warehouse

To use Transform you will have to connect your data warehouse to Transform's MQL server. We currently support Snowflake, Redshift and BigQuery.

To learn more about the requirements of connection for each data warehouse, visit the Data Warehouse page.

You will be able to test your connection in the app when you set up. The user that Transform uses to connect should have read, write, delete and update permissions on the schema you specify.

Setting Up Your GitHub Repository

This is where you will configure and validate you data source and config files. You can download the Transform template repo here. You will need to set up GitHub secrets in order to validate your config files. There is a step-by-step guide on setting up your repo in the installation guide.

After this is complete you are ready to start using Transform! We recommend installing the Transform CLI for a better experience developing with Transform.

Installing the Transform CLI

To install the CLI run

pip install transform

Python 3.7+ is required to run transform. You can learn more about our Transform MQL client library here

Then to connect you your MQL server run

mql setup

Requirements to access the Transform app

These are the requirements to use the Transform app:

Memory2 GB minimum, 4 GB recommended
Screen resolution1280x1024 or larger
Application window size1024x680 or larger
Internet connectionRequired
Supported Data Warehouse (Snowflake, Redshift, BigQuery)Required
Git repo - GitHub recommendRequired

Supported Browsers: Chrome Safari Firefox Edge We do not yet support mobile browsers, but plan to in the future. If you are interested, please let us know at support@transformdata.io

Windows requirementsMac requirementsLinux requirements
Operating SystemWindows 8 or latermacOS Sierra 10.12 or later64-bit, Ubuntu 14.04+, Debian 8+, openSUSE 13.3+, or Fedora Linux 24+
ProcessorIntel Pentium 4 or laterIntel or M seriesIntel Pentium 4 or later