Dimensions let you drill down into experiment results. For example, if you define a
browser dimension, you can
see how Safari users behaved vs Chrome.
Be careful, the more dimensions and metrics you look at for an experiment, the more likely you are to see false positives - something that looks significant when it really isn't. For example, if you break out results by country, it's pretty likely that at least one of the 100+ countries in your dataset will be signficantly different just by random chance.
It's best to treat dimensions as an exploratory tool and not something to directly draw conclusions from.
The two best use cases are identfying bugs (the
browser example) and getting ideas for dedicated follow-up experiments.
There are two types of dimensions: User Dimensions and Experiment Dimensions. Currently, these are only supported for SQL data sources.
These are attributes of your users that are relatively stable over time. For example,
A user dimension is defined by a simple SQL query that returns two columns:
value. For example:
SELECT user_id, plan_type as value FROM subscriptions
It's best to keep the number of unique values for a dimension small if possible to avoid the False Positive issues discussed above. For example, instead of a
dimension with 200 possible values, maybe you could make the values
Other. Depending on your SQL engine, you
can use something like a
CASE WHEN statement:
SELECT user_id, ( CASE WHEN country = 'us' THEN 'US' WHEN country = 'uk' THEN 'UK' ELSE 'Other' END ) as value FROM users
These are attributes that are specific to the point-in-time that a user was put into an experiment. For example,
Instead of a standalone SQL query, experiment dimensions are simply extra columns you return from the Experiment query defined for your data source.
As an example, if you set the following as your experiment query:
SELECT user_id, anonymous_id, received_at as timestamp, experiment_id, variation_id, browser FROM experiment_viewed
The first 5 columns are standard, but
browser is a custom one that can be used as an Experiment Dimension.