A/B Testing
Evaluating Entry Points
A major challenge in product integration is determining the most effective placement for a partner company's product within our platform. The goal is to maximize user engagement and conversion rates by strategically positioning the entry points
My Role
Having previous experience in designing native iOS App I knew that I had to led all facets of design including Information Architrcure, User task flow, Interaction, visual, product & prototyping.
Another task was to conduct User research using methods such as interviews, focus groups and participatory design in order to address both user behaviour and attitudes. p
Tools Used:
Figma, HotJar, Google Analytics
Platforms:
A/B Testing
Problem Statement
A major challenge in product integration is determining the most effective placement for a partner company's product within our platform. The goal is to maximize user engagement and conversion rates by strategically positioning the entry points. This experiment evaluates two entry points for the partner product:
Sidebar Placement – A persistent link to the partner product in the left-hand navigation panel.
Snippet at the Bottom of the Page – A brief preview with key information and a call-to-action placed at the bottom of relevant pages.
The Goal
The primary objective of this A/B test is to identify the most effective entry point for integrating the partner product by measuring:
Click-Through Rate (CTR): The percentage of users who click on the entry point.
Conversion Rate (CVR): The percentage of users who complete a desired action (e.g., sign up, purchase, etc.) after clicking.
Time Spent on Partner’s Platform: The average duration users stay engaged after transitioning.
Test Setup
For the test we created 2 scenarios the “A” version (the original one and control scenario) and the “B” version (the new version). We decided to redesing the home page and the prototype of some elements (for instance the radio buttons and check boxes).
Variant A (Sidebar Placement): The entry point appears as a persistent sidebar widget.
Variant B (Bottom Snippet Placement): The entry point is displayed as a contextual snippet at the bottom of the page.
For Test Methodology we used Bayesian Approach instead of Frequentist
We utilized a Bayesian approach to continuously update our probability estimates based on observed data, allowing for real-time decision-making.
The test was conducted for 2 weeks and there were 2000 users tested for each variant
Results & Insight
01 Key Findings
| Metric | Sidebar Placement (A) | Bottom Snippet (B) |
|---|---|---|
| Click-Through Rate (CTR) | 3.2% | 5.8% |
| Conversion Rate (CVR) | 1.1% | 2.3% |
| Avg. Time on Partner Site | 1m 45s | 2m 30s |
Test Analysis
Probability of Bottom Snippet outperforming Sidebar: 94%
Expected Uplift in Conversions: +109% (compared to Sidebar)
Conclusion & Recommendations
Bottom Snippet (B) is the clear winner, driving higher engagement and conversion rates.
Implement Bottom Snippet placement site-wide.
Conduct a follow-up test with different snippet designs to optimize further.
Explore personalizing the snippet based on user behavior.
