Retail chain and e‑commerce · CRM
Disproved the client's earlier estimate: +77% turned out to be +5%
- initial vs. correct estimate — overstated ~16x
- +77% → +4.8%
- uplift in purchase conversion across the whole test group
- +4.8%
- uplift in conversion among customers who received the message
- +7.2%
- uplift in reactivation in the high‑basket segment inactive for 6–12 months
- +0.32 pp
Challenge
The client needed to know how much CRM communications (SMS, push, email) contribute on top of purchases customers would have made anyway
What we did
- Designed a global holdout group — customers who received no communications for a month
- Checked group balance on historical data before the test started
- Found a selection effect: when uploading to the messaging service, 52% of customers dropped out of the test group versus 18% of the control group, and those who dropped out were less active
- Recalculated the effect on the original random split and separately ran a segment‑level SMS A/B test
Outcome
The company now has a working estimate of CRM incremental revenue and a scaling criterion: the effect must cover discounts and channel costs. Instead of messaging everyone, the team now knows exactly whom to contact
Important! The effect is positive but borderline statistically significant: the revenue confidence interval includes zero. We flagged this ourselves and scheduled a repeat test with a properly filtered control group
Facing a similar challenge? See the offer: Honest CRM Evaluation: Campaign Incrementality → Holding company: retail, wholesale, manufacturing, marketplaces
Data marts reconciled with accounting to within 0.2% — no more manual checks
- variance between data marts and accounting on revenue and cost of goods
- 0.2%
- faster customer report refresh
- ×3
- customer histories from two retail chains merged
- 2 → 1
Challenge
Report figures didn't match the accounting system, marketplace, wholesale and CRM data lived in different places, and weekly commercial reporting was compiled by hand
What we did
- Integrated marketplace data, wholesale data from the ERP, the order management system and the CRM platform into the warehouse
- Built data marts for revenue and cost of goods, year‑over‑year sales, digital goods and pre‑orders
- Launched a group‑wide glossary and agreed on disputed definitions (e.g., “new release”)
- Built an accounting data quality dashboard and started a data lineage map
Outcome
Revenue and cost of goods in the data marts can be trusted without manual reconciliation; weekly commercial reporting is built on the same data, and the question “where does this number come from?” has a documented answer
Facing a similar challenge? See the offer: Unified Reporting Reconciled with Accounting → Online game publisher · performance marketing
Found “orphaned” traffic and restored paid search's true contribution
- more new users attributed to the channel
- +5.4%
- more returning users attributed to the channel
- +8.1%
- loss for other ad sources
- 0%
- faster attribution recalculation
- ×16
Challenge
Acquisition reports undervalued paid search, and recalculating attribution for returning users took half a day, slowing down budget decisions
What we did
- Audited link tagging and found visits with an untagged parameter that were not credited to any source
- Fixed the tagging and recalculated a year of historical data
- Rewrote the attribution logic for returning users
- Configured the web analytics tag on the sign‑in domain so sessions no longer break, and set up offline conversion uploads to ad platforms
Outcome
The channel had been performing better than the reports showed all along. Budget decisions now rely on accurate numbers, and attribution is recalculated in a short window instead of half a day
Facing a similar challenge? See the offer: Attribution and Marketing Data Audit → Online game publisher · MMORPG
Showed that social mechanics are a lever for the game economy
- higher average spend among guild members
- ×12.5
- higher payer share (17% vs. 3%)
- ×5.7
- more purchases per player
- ×10.8
Challenge
The team needed to understand whether to invest in mechanics that bring players together, and where players drop off before joining the core audience
What we did
- Built a social activity dashboard across several titles — over a million rows of data
- Compared players in and outside guilds by spend, purchases and payer share
- Ran a cohort analysis of churn before players reach the core audience
- Built a report on player gear and progression and automated data collection
Outcome
Social mechanics came to be seen as a lever for the game's economy rather than social features for their own sake. The cohort analysis identified a group of players with accelerated churn at a specific lifecycle stage — a clear target for product changes
Important! This is an observed correlation, not a proven cause: already engaged players may simply be more likely to join guilds. To test causality, we proposed designing an experiment
Facing a similar challenge? See the offer: Online Game Economy and Community Analytics →