Lift and confidence - A4T FAQ
This topic contains answers to questions that are frequently asked about lift and confidence when using Analytics as the reporting source for Target (A4T).
Can I perform offline calculations for A4T?
You can perform offline calculations for A4T, but it requires a step with data exports in Analytics. For more information, see "Performing Offline Calculations for Analytics for Target (A4T)" in Confidence Level and Confidence Interval .
How is lift calculated?
Lift is the percent difference between your control page results and a successful test variant.
How is confidence calculated?
The confidence level is the probability that the measured conversion rate differs from the champion page conversion rate for reasons other than chance alone.
Why can't I see lift and confidence on calculated metrics?
Calculated metrics are not currently supported in lift and confidence functions. This is due to the fact that Analytics calculates metrics at an aggregate-level, rather than at a visitor-level. Confidence, in particular, is a visitor-level calculation.
Non-calculated (standard) events are supported in lift and confidence. They become the numerator in the lift function; the numerator cannot be a calculation itself. The denominator is the normalizing metrics (impressions, visits, or visitors). Some examples of standard events include orders, revenue, activity conversions, custom events 1-1000, etc. This means that common optimization metrics, such as conversation rate (Orders/Visitor) and RPV (Revenue/Visitor) are supported in lift and confidence.
Examples of unsupported metrics or use cases include:
- Average Order Value (Revenue/Order, per Visitor). AOV is not supported because the numerator is a calculated metric. Instead, the recommendation is to consider the two influencing metrics of AOV - Revenue Per Visitors and Conversion Rate.
- Calculated metrics that are the sum of standard events. For example, you might track ten different lead forms into ten separate events, and then add them together to get total lead submissions. A recommended method to track these events is to implement a single lead submission event in Analytics and then use an eVar to collect the type of lead form. Using this method requires fewer variables and ensures that you can use the single lead submission metric in lift and confidence functions.
How does A4T handle confidence calculations?
A4T uses non-binary metric calculations with the sum of square data. The variance is calculated using the sum of squares data. Extreme orders are not taken into account. Additionally, the confidence calculation does not apply a Bonferroni correction for multiple offers.
Do lift and confidence work in Ad Hoc and Report Builder? If it's not native, can I do it in there myself?
Lift and confidence do not work in Ad Hoc or Report Builder, and cannot be calculated yourself for continuous variables. It is possible to calculate it manually for binary metrics.