Marketing science at Carmoola: a deeper dive into customer journeys

The effectiveness of marketing - the impact per pound spent - is a vital data point that we track, but at times it can be a bit like attempting to solve a complex puzzle with missing pieces. 

Like many other businesses, we face a lot of challenges. We have a long time-to-conversion with multiple customer touchpoints. We’re an app-based business affected by increasing privacy restrictions on Apple devices, especially since iOS 14.5. We also have an online web journey impacted by cookie consent - which customers then continue in-app, introducing another potential point of data loss when customers make the transition. It’s safe to say that when I first joined the business I really couldn’t tell you which of our channels was driving performance!

Despite these hurdles, our approach to understanding customer journeys has allowed us to navigate these complexities, delivering actionable insights and tangible results.

Beyond the last touch

There was once a pizza place that hired three different promoters to hand out flyers with coupons on. Each flyer was a different colour so the place knew which one had driven the customer there.

At the end of the day, one of the three promoters was a clear winner, with way more flyers returned from customers than the other two.

When they asked him what he did, he said that he simply stood outside the store and handed them to people who were about to walk in (shoutout to Gil David for this great parable!)
In a similar way, one of our highest performing campaigns on some early dashboards appeared to be the automated email a customer received when they saved their dream car - a fairly late stage in the journey after they had passed their credit check, downloaded the app, and already started exploring its features. Should this be the touchpoint that we give full credit to for driving the loan? Can we hit our growth goals by sending many more of these emails? Of course not.

In another way, strategies like retargeting campaigns on Meta and brand-bidding on Google Search or Apple Search Ads may appear to deliver impressive looking results, but without a proper incrementality analysis may not be providing much tangible business value other than making some of my graphs look more impressive in our weekly trading meeting! 
Understanding “how did they enter the retargeting funnel,”, “why did they Google our brand” or even “what was the channel that drove the click just before the customer completed our webform” is crucial. Expanding attribution still to include even earlier touchpoints can provide a more comprehensive view of what truly drives customer engagement and will continue to lead to insights that will help us grow.

Massive thank you to our Engineering and Data teams, especially Sviatoslav Malinovskyi in Lviv, for their tireless work and innovation to build the technical capability and data products that underpin these models for us.

Valuable lessons from imperfect data

It’s important to remember that even when data isn’t perfect, it can still offer valuable insights. When testing a new strategy, in an ideal world I would A/B test both the new and old approach against the quantity and quality of loans generated. But this could take months, reduce our testing cadence, and have a serious financial cost and opportunity cost.

To highlight this, early on we identified a keyword theme on Google Search that was indirectly related to our product, but had low competition and seemed to drive a very low cost per application. So naturally, we scaled spend. At one point, this strategy accounted for 25% of the channel’s budget!  

Back then we still had patchy data linking these applications through to loans, and there weren’t enough loans to have any sort of statistical confidence in its performance.

Fortunately, we’ve learned that identifying leading indicators in the customer journey that correlate strongly with loans can allow us to make faster but still data-driven decisions. 

Analysis revealed that this keyword theme converted poorly from top-of-funnel engagement to loan, and had a predicted customer acquisition cost (CAC) three times higher than other strategies within the same channel. Armed with this insight, we slashed the budget for this strategy. 

The result? The channel’s overall performance improved significantly, demonstrating the power of even partial data in driving smarter decisions.

The power of simply asking

Sometimes, the simplest solutions can yield the most valuable insights. Last year we added a “How Did You Hear About Us” (HDYHAU) question to our forms. While this doesn’t resolve all attribution challenges, it’s proven to be a very useful piece of the puzzle.

To test its accuracy, we included a response option for a channel we don’t actively use, and only 0.1% of respondents selected it. This, along with insights we can glean from other tools, confirmed that customers’ answers are largely reliable and consistent with what we see elsewhere. Moreover, when we compare web touchpoints with HDYHAU responses, they align strongly, providing a layer of validation for the data we’re gathering.

The “Other” field in our HDYHAU question has proven particularly insightful. It revealed some unexpected sources of traffic—for example, dealerships recommending us to their customers, a channel we hadn’t focused much on as part of our marketing strategy. This discovery has opened up exciting new opportunities for collaboration and growth. We’ve since launched a pilot scheme to work more closely with select dealerships, a program that we plan to expand over time.

Using HDYHAU has also helped to highlight channels such as YouTube that were underappreciated by our analysis based purely on clicks. This is opening up a new avenue of testing for us to see how effective video can be in driving high-value behaviours from our target audience in 2025.

The never-ending road to improvement

Attribution is not a problem you solve overnight. It’s an iterative process, much like the old adage: “Art is never finished, only abandoned.” Over time, through continuous experimentation and refinement, and barrelloads of elbow grease from Sviatoslav, we’ve seen steady progress in uncovering how different channels and strategies contribute to our overall success. This journey involves juggling numerous moving parts, embracing imperfection, and staying committed to gradual improvement.

Attribution will never be a one-size-fits-all solution. For us, it’s about embracing the journey: iterating, testing, learning, and refining. Each step forward helps us better understand our customers, optimise our strategies, and allocate resources more effectively. It’s a process that requires patience, creativity, and a commitment to continuous improvement. But as we’ve seen, the rewards make the effort well worth it.

This philosophy is at the core of how we approach marketing at Carmoola. By embracing imperfection and uncovering insights through experimentation, we continue to innovate and deliver superior service, fairer rates, and a better customer experience—all while keeping our customer acquisition costs and operational expenses low. Maintaining this balance is critical to our mission and ensures our success in a competitive landscape.