Cutting Through the AI Hype

I started my career as a statistician in financial services - I know, I’ve lost some of you already! The role appealed to me, straight out of university, as a way to apply some of that theory I’d spent the last three years struggling with to real world problems. The team was sitting in a corner of the top floor of the office and we were pretty much left to our own devices, happy with our confidence intervals and sample size calculators. 

But about 15 years ago, everything changed…

Data in the spotlight 🔦

The combination of an explosion in the amount of data being produced through the digitisation of our world, and the dramatic reduction in data storage costs, blew apart the pre-existing constraints around what data could do.

From being in the shadows, those with data and algorithmic skills (the newly labelled ‘data scientists’) were now hot property and in demand!

Now, I love data and I love being a data scientist, so it's absolutely in my best interest to keep on doing this amazing profession.

This means successfully walking the ‘hype tightrope’ - being open to the opportunities new advances bring, whilst not over-promising what is (currently) possible. This tightrope has become even more precarious in the last few years given the huge leaps forward in AI capabilities. 

As such I wanted to lay out my top three tips to help you navigate through the hype:

1. Focus on the goal: What are you trying to achieve with AI? What are the specific business challenges you are facing? Once you know the answers to these questions, you can start to identify the data you need and the techniques that will be most effective. An analogy I often use is that a great data scientist has a bookcase of approaches available to them and they know, for a given problem, exactly which book to reach for. Essentially, the right tool for the right job - don’t use AI for AI’s sake!
   
   
2. Always be curious: Key to evolving into new technologies is knowing they exist in the first place. This means cultivating a strong network (importantly, across a range of different industries), setting aside time for prototyping, and generally being outward facing. It’s likely the next best idea for your business will come from inspiration found elsewhere. I feel like one of my key responsibilities at Carmoola is to bring in the newest and best ideas from elsewhere and find applications that solve some of our next challenges.
   
   
3. Impacts in market: I position myself as a pragmatic data scientist and my view is that AI applications need to get out there and make a difference, not just sit around being elegant. Chasing the perfect solution can hold things back and stop real progress. It's all about finding that sweet spot between innovation and actually getting stuff done. The market is the ultimate testing ground, giving instant feedback, user input, and chances to fine-tune things. You should still hold the highest bar for perfect deployment but bear in mind a solution not yet deployed generates no in-market value.

In a world of shiny trinkets, maintain your focus 🪩

As a data scientist with 25 years of experience, I've seen my share of data and AI hype.

Here's my advice: focus on your goal and don't use AI just because it's trendy.

Be curious and open to new ideas from all sorts of places.

Remember that real-world impact and market feedback are more valuable than a perfect solution. Find a balance between innovation and practicality – that's the key to success.

Also, if we’re not already, let’s connect on LinkedIn - it’s never too soon to grow that network!