How to do a Comparison Rate Test

This article outlines key differences in attribution reporting, the steps to conduct a valid comparison rate test, and how to analyze the results effectively.

Introduction to the Comparison Rate Test

When assessing the effectiveness of your marketing campaigns, comparing conversion data from platforms like Wicked Reports and Meta Ads Manager can provide valuable insights. However, it's crucial to acknowledge that discrepancies in reported numbers are not only common but often expected. By understanding the factors that contribute to these differences, you can interpret the data more accurately and make informed decisions that enhance your marketing strategy. This guide outlines key differences in attribution reporting, the steps to conduct a valid comparison, and how to analyze the results effectively.

Key Differences in Attribution Reporting

While it’s healthy to compare Wicked Reports to other attribution software, it’s important to understand that no two attribution platforms will ever report the exact same numbers. Here's why:

  • Multi-Channel vs. Single-Channel Attribution: Wicked Reports tracks conversions across all your marketing channels, while platforms like Facebook only focus on their own. Wicked ensures that credit is assigned to the most impactful touchpoint in your entire funnel, rather than isolating conversions to just one platform. This gives you a more comprehensive view of your true marketing performance, avoiding the tunnel vision of single-channel reporting.
  • Short-Term vs. Long-Term Attribution Windows: Ad platforms like Facebook typically use a short attribution window, often looking forward 7 days after a click (or sometimes just a view), which limits their understanding of long-term customer behavior. Wicked Reports, however, enables you to use a much longer time window(90 days+) from the point of conversion. This allows Wicked to give credit to marketing efforts that influenced a customer over a longer period, capturing a fuller picture of your sales cycle and marketing impact.

For a deeper dive, learn more about why your Wicked conversions might not match other platforms.

Steps for a Valid Comparison

If you're comparing results between Wicked Reports and another tracking software, then you should compare accurately through these steps:

  1. Open the FunnelVision Report
    Ensure you’re using the Full Impact Attribution setting.

  2. Match the Date Ranges
    Both Wicked and the comparison platform should cover the exact same time period.

  3. Set Facebook View-Through Confidence (if applicable)
    If you’re comparing Facebook results, set the view-through confidence to 100%.

  4. Calculate the Comparison Rate

    1. Use this formula to compare the revenues between platforms:
      Wicked Revenue ÷ Platform Revenue

    2. Use this formula to compare the leads between platforms:
      Wicked Leads ÷ Platform Leads 

Analyzing The Results

In most cases, Wicked will track more revenue than the software you are comparing it against. That is especially true for businesses with longer sales cycles or repeat purchases because our attribution models can track and attribute sales over a longer period of time. That being said, the expectation is that Wicked will have, at least, an 80%+ comparison rate in sales and lead attribution. 

  • Why Wicked Might Have a Negative 20%- Variance (Passed Test)
    When Wicked reports slightly lower numbers, it’s often because platforms like Meta Ads Manager or email software are single-channel and, as a result, tend to over-report in their favor. This is an expected behavior and should not be a cause for concern.
  • Why Wicked Might Have a Negative 20%+  Variance (Failed Test)
    It is rare for Wicked to track less than 80% of what your comparison software reports. However, if you do notice a rate below 80%, this is an indicator that something may be off in your tracking setup. As a result, it's worth running the Tracking Validation Routine to investigate and determine if there are any critical tracking issues negatively impacting your attribution.

Closing Summary
In summary, while it’s common for Wicked to track higher revenue due to its comprehensive attribution capabilities, maintaining an expected comparison rate of 80% or more is essential for accuracy. If you encounter a variance below this threshold, it’s a signal to review your tracking setup. By conducting the Tracking Validation Routine, you can identify and address any issues that may be affecting your results, ensuring that you have a reliable tracking setup and quality data.