The Depressed Data Science Unicorn
What AI Product Managers should know about Data Scientist's Legacy
#beyondAI
Not long ago, and even today, companies held—and often still hold—enormous expectations for Data Scientists. The belief was that once a data scientist joined the team, they would possess the golden key to operational excellence.
These expectations were, indeed, unrealistic.
While companies aggressively sought out data scientists, they also lured them with high salaries, convincing many early data scientists to join under these lofty promises.
However, it didn't take long for these data scientists to realize that companies were not merely asking for data science tasks like modeling and gathering insights. These tasks, while essential, were not enough to drive process efficiency or effectiveness.
What companies truly wanted were end-to-end AI solutions.
Many data scientists tried to cover all these tasks on their own, and unsurprisingly, too many failed. The staggering number of failed AI projects can, in part, be attributed to the unrealistic expectations companies placed on these professionals, expecting them to be data science unicorns.
A few years ago, I wrote an article on this very topic. As I reviewed it recently, I felt it was crucial to share it again, especially with the new generation of AI Product Managers who are now entering companies and working alongside those earlier-generation data scientists.
Understanding this precious generation is key for AI Product Managers. The challenges they have faced—how they navigated through years of unrealistic expectations until companies finally realized that data science unicorns only exist in fairy tales—are often underestimated. We can become more empathetic AI Product Managers once we fully grasp the legacy of these Data Scientists.
With this empathy, we will better understand why some data scientists take on roles akin to AI Product Managers, why they sometimes mistrust the work of others, and why they hold such high expectations.
Empathy is sometimes called a secret weapon. For AI Product Managers, it’s not just a secret weapon—it’s your daily business. Understanding the people around you and the roles they’ve played, both today and in the past reveals a lot about how best to approach them—and how best not to.
This reflection on the past serves as a hidden guide for how AI Product Managers can better understand and collaborate with Data Scientists going forward.
Happy reading 🛋️
The Data Scientist’s Legacy
Let’s assume we’ve found one—the one and only treasured unicorn in the data science universe. The one with the perfect combination of hard and soft skills needed to finally convert data into cash. The Special One who could do it all on their own: starting from the ideation of a data science use case with the right business impact, to establishing a robust data pipeline that produces quality-assured data features, to providing the optimal model that solves the problem, and even ensuring performance monitoring for sustainability.
Leaving out the in-between tasks like stakeholder management, presenting results, designing roadmaps, and keeping it even simpler, let’s also assume that in this unicorn world, a fantastic ready-to-use data science tech stack is available.
Given these perceptions and assumptions, what do we expect from this rare, beautiful creature?
Do we expect this lonely unicorn to be competitive in a harsh business reality where the time-to-market for an AI product also defines the success of data science use cases?
If so, poor little thing! But let’s play this one out:
This unicorn develops an idea for an AI product and pitches it to decision-makers. The idea gets approval, and everyone is excited about the promised optimization of a business-critical marketing process, nurtured by the model insights the unicorn has conceptualized. The only issue that doesn’t sit well with us as stakeholders is the expected time-to-market the unicorn has provided. So, what could happen next are three possible scenarios:
In the first scenario, we could say: “The idea is extraordinary, let’s go for it and take the risk of a long time-to-market”—which, in the worst case (but not unlikely), could lead the competition to catch up by having the same idea but implementing it in a shorter time. The hoped-for advantage would be gone.
Unhappy unicorn, unhappy stakeholders.
Second, we could say we’ll go for the idea but decide on a quick-and-dirty approach, deprioritizing essential quality-assurance initiatives. But this creates the so-called hidden technical debt, which will slow us down in the long run, and the hoped-for advantage will be gone.
And again, unhappy unicorn, unhappy us.
Third, we decide everything suggested should be implemented by the unicorn, but in a shorter time frame. Translated into work-life balance, this means a massive workload under time pressure, leading to a burned-out unicorn.
Have you ever seen a burned-out unicorn?
They refuse food, lose the joy of life, and eventually become spirit-broken ordinary horses. Over time, the unicorn will leave us, willingly joining the bull covered in fire to reunite with other unicorns in the sea. And we will never see The Last Unicorn again.
In these three scenarios, which are not exhaustive, the unicorn has no other choice but to become depressed.
But maybe there’s one scenario left that might work for both us and the unicorn. If we have a unique AI product idea where the odds are not against us—where our competitors won’t quickly develop an equivalent idea—and if our concept is so unique that even a long time-to-market won’t harm its success, then maybe we’ll have a chance at a romantic ending. And even then, this will probably only suffice to produce a prototype. As soon as it comes to operationalizing and scaling everything up, we’ll need a functioning data team again to accelerate and keep our competition at a distance.
In sum: A rare data science unicorn can only survive—but not thrive—in a rare business environment. For a while, at least.
Well, now I’m depressed too.
A herd of unicorns is called a blessing.
So why look for a data science unicorn at all if we will eventually need a fully specialized team anyway? Indeed, there seems to be no realistic scenario where a unicorn as a solo player can find happiness working with us.
I do not doubt the existence of unicorns—I’ve seen some during my career, but they all ended up in one of the above fates. I believe unicorns are not rare because there are only a few out there. They are rare because they are hiding. They are aware of what could happen to them if they show up. We should take that fear away and open up new visions and perspectives where a unicorn can thrive.
The only reason I can imagine looking for a unicorn is not to expect them to perform as a unicorn. I want a unicorn because I know it can cause miracles when it comes to leading an AI product team. They can oversee the entire AI product development process, understand what should be tackled first, and wait for the next development iteration. Instead of letting them implement every aspect of the AI solutions themselves in an unreasonable time, we should encourage them to enable AI product team members to become the best versions of themselves and deliver appropriately. A unicorn can organize data engineers, data architects, data scientists, and all other data professionals in that team, allowing them to become specialized unicorns. With this in mind, we could build a herd of unicorns.
People often say a herd of unicorns is called a blessing; similarly, a well-established team of specialists is a blessing too, regardless of the existence of a beloved unicorn. When you find a data science unicorn, take the bull by the horns and help them understand their influential role as a team member. Support them in becoming a team leader.
Sounds like a plan! But what if the unicorn just wants to be a data scientist?
Well, yes. That might happen, and actually, it’s very likely since that unicorn applied to be a data scientist, right?
In this case, we should forget everything written above, stop looking for unicorns, and instead hire a product manager who is comfortable creating AI Products and leading a team of data and AI specialists. To be honest, this is likely the best solution for everyone involved. A product manager is specifically meant to organize, coordinate, challenge, and encourage a great team of specialists to focus on the higher goals: solving problems and creating business opportunities. And, of course, the AIPM takes responsibility for the in-between tasks mentioned earlier.
Wait! What? — Yes.
I love unicorns, but we don’t need them because we cannot offer the fairy tale environment in which a unicorn can thrive. So let’s leave them free. They will show up when they are ready to take on new heights and dimensions.
And after all, a well-coordinated team of specialists is a blessing too, right?
So why even bother?
JBK 🕊
P.S. If you’ve found my posts valuable, consider supporting my work. While I’m not accepting payments right now, you can help by sharing, liking, and commenting here or on my LinkedIn posts. This helps me reach more people on this journey, and your feedback is invaluable for improving the content. Thank you for being part of this community ❤️.