#138 - Why most AI projects fail before they even begin

Joubin Rahimi

Brilliant that you're back for a new episode of insights! My name's Joubin Rahimi and today I have Matthias Bauer with me. Hi, Matthias.

Matthias Bauer

Hi, hello there, Joubin.

Joubin Rahimi

Matthias is one of our AI specialists and I'd say, also usually on stage, and given your stature and expertise, a figurehead on this topic too. Would you like to say a couple of sentences about yourself and the topic of AI?

Matthias Bauer

Yes, great, gladly. So first about me, born in '85, I got into working life sometime around 2007, and since then I've actually done nothing other than everything to do with data and AI. Back then we still called it data mining, process mining, data science, but ultimately the methods have simply evolved over time. Otherwise: married, two children, and an enthusiastic scout.

Joubin Rahimi

Well, that's nice. Matze is one of the few who brings personal moments in here too. I'll skip the scouts detour, but maybe we'll come back to it in conversation. You're with an awful lot of our clients. Matze is a Fellow. Fellows are the ones who act as C-level trusted advisors on certain topics. What are the topics discussed at management level that you come across?

Matthias Bauer

Yes. I try to too … It's like scouting. It's said, yes, indeed. So finding your path also comes, to some extent, from finding your way through life. You can obviously transfer that to a company too. That's actually quite a good anecdote. What I actually encounter is … We're talking about insights here. That means full, clear vision. And what I very often encounter is actually a very, very high expectation of artificial intelligence, because people pick up bits and pieces here and there. Recently I was asked: "How far are we now from programmable matter?" So I said: "Teleportation probably comes before that." And you always have to be a bit careful there, because ultimately we've just been through a real peak of hype. It started in October, November 2022 with ChatGPT and similar products, such as from OpenAI. And here and there people also got the impression that this was really the absolute silver bullet, or the panacea for all problems, somehow.

Joubin Rahimi

Yes, you put it in and then it's great.

Matthias Bauer

That reminded me a bit of the years 2012, 2013. Back then IBM was going around winning with Watson. And in particular the manufacturer sales reps went from door to door selling it as a magic black box. Tip all your data in and all your problems get solved. It's not that simple, still isn't today either, even though of course we now have completely new possibilities, but I do notice high expectations. But on the other hand I also notice that many have by now realised, well, it's definitely a chance for change, for optimisation within their own business processes, but also a transformation of their own business model or the opening up of completely new business models or products.

Joubin Rahimi

When a C-level approaches you, the first thing you notice is: expectations are too high, along the lines of: we'll do a bit of this Open AI thing and then it's great.

Matthias Bauer

I always just say: well, if it were that easy, I wouldn't be here now.

Joubin Rahimi

And others would have done it already and you'd have far more problems. Probably like that too, right? Exactly. What are the typical pitfalls companies fall into when they take these steps, on this topic?

Matthias Bauer

So ultimately, I always say: firstly, ChatGPT as, I'd say, an AI application class in itself, is only one of very, very many different methods that artificial intelligence actually offers. Things like classic machine learning, or perhaps some regression calculations for example for a cash flow forecast or something like that, are still just as relevant as before. Or also making image information usable, computer vision. Those are all different areas, sub-fields of AI. And what they all have in common, of course, is that they need a solid data foundation. You hear that everywhere, of course, but what I notice is, well, along the lines of, this is a new method, so surely we must have overcome the old problems now. No, they're still there. I don't know if we can maybe show it later. There's a funny image, a Dilbert-style image, isn't there? Yes, something like that. On the left-hand side there are the data pots, and then listed alongside are the various methods, so data science, some kind of machine learning, AI, Gen AI, agentic AI. And on the right-hand side you then have data crap-heaps in unicorn shapes, or in multiple unicorn shapes. And that actually captures it really well as an illustration. Whatever method you use, you need to somehow have your data under control. Not just for AI, ultimately also for the organisation itself, if you intend to make more data-driven decisions or run the business that way, or even enable automation at all.

Joubin Rahimi

Now you say you need to have data under control. That means the data isn't good. If a C-level person hears that, they'll probably nod along. But what does it mean? When exactly is data not good? Because this person is probably being told: the data's great, their staff say so, but actually it isn't great.

Matthias Bauer

In 9 out of 10 cases, everyone always says their data is great, but actually, when we do projects, we're dealing with some business case, some use case. Let me bring in a small example here, a little anecdote. Gladly. This was with a manufacturer of plastic fibres, very, very thin plastic fibres. This is essentially the intermediate product from which artificial membranes are later woven, which are used in dialysis systems. Okay. They had the problem: 40 production lines, sometimes running three shifts, and it kept happening that at some point within a week or so, this annoying thread would snap. Then the paste runs into the machine somehow, you have to shut it down, clean it, recalibrate it, blah blah blah, a lot of money is lost. Then we were called in: yes, we've now spent three years trying to figure out with our engineers what's causing this. We don't know, can something be done with data? Also the usual, everything's great, our data is excellent. We have an MDE, a BDE system, we have an MES, an ERP, and all the data is somehow brought together there, so it shouldn't be a problem. And then I say: yes, okay, the case is interesting, you can indeed run the numbers against it. But before I say we'll do something, I'd like to have a look at the data first. And that's what we call data understanding in the process. And what came out was that the data was only recorded as a ten-second average, essentially. And I said, if I want to find out what's actually, which cause-and-effect relationships are actually immediately relevant at the moment the thread snaps, these ten-second averages are worthless. Absolutely worthless. And then it's like: yes, but maybe you could do something approximately, and then we're more the ones who say: well, we're a service provider. We'll sell you the project, but we'd advise against it.

Joubin Rahimi

Doesn't make sense.

Matthias Bauer

Simply makes no sense. Better to invest first in somewhat more optimised data management and enable yourselves to gather the necessary resolution of data. That's perhaps a rather graphic example now, but I really see this very, very often. You simply have to have this under control.

Joubin Rahimi

Great example. I think it makes sense to dip a bit deeper into that, doesn't it? Exactly.

Matthias Bauer

So by the way, this is a method now. I mentioned earlier I've been in working life since around 2007 or so, and this comes from the 90s. Stands for CRISP-DM, Cross-Industry Standard Process for Data Mining, or by now further developed into CRISP-ML for machine learning and so on. Yes, but it's a logical cycle. You engage with the business problem and really try to name the business problem properly. That's often a process issue, ultimately. Do we need to optimise a process here, or automate a sub-process to optimise costs, etc. Then you first ask very precisely: How can this be delimited and what actually needs to be achieved to really generate business value, added value? And then you look at the data and if everything fits, then you say: OK, then we build models or use Gen AI or whatever.

Joubin Rahimi

And then we actually come to the second step. Then we really build things. How well do these work within the company? Are we doing well there? Or can we still optimise it? That's a wonder point, isn't it?

Matthias Bauer

This is a magic point, because that's … how shall I put it? I come from a world that has always been very concerned with automating processes, optimising business processes, and also building new business models. And there you often have situations where you're actually coming into some existing setup. There's already something there. That means you have to somehow … That means a lot of custom, custom solutions. And when I now look at what standard tools are available on the market today and then I get feedback: oh, it's given the wrong result again. I did actually link a OneDrive folder or whatever to the AI, but somehow it still can't find it. Or recently I had a question where I knew exactly that there's an e-mail sitting in my inbox and the tool of choice just doesn't surface it and that shouldn't happen at all. So this kind of thing comes up constantly, and I have to say, that can be done better. It really can be done better even today. However, I do have to say there's a difference between a very strong domain orientation or business process or business unit orientation versus vendors and products that try to cover as broad a base as possible. And of course, if you look at the pipeline over the coming years, even at the big vendors, for standard products too, it's foreseeable that this will get better, but at the moment I have the feeling with many that they're finding: we bought a whole stable of licences. It somehow doesn't work the way we thought. We're scaling that back down again, and so on. Now they're coming back to us and saying: didn't you explain a year ago that this can be done better too? And indeed it is.

Joubin Rahimi

Funny thing. Before ChatGPT, I think it was even before Covid, we had a project where, based on product data, we calculated the core product data of new products and then also calculated substitute products, as well as recommended products and complementary products. Brilliant. The project was never put into use. Everyone doubted it would work. It was a research project, it worked, and then we were told: we can't even get it into sales. I thought: incredible, they're just burning half a million like that.

Matthias Bauer

Now they know it works. But they don't do it. If you want to hear a similar story, I'm happy to share another one – a service provider for billing private medical services. We somehow got in there, had a long sales cycle. They said: yes, we want to try this out now. And then we said: fine, let's try it with one partial step. I believe it was about the automated extraction of data from accident reports from attending physicians. Anyway, they gave us the corresponding data. We built a small POC. Hugely successful. Recognition rate, I believe, around 97, 98%.

Joubin Rahimi

If everything's over 80, that's already good.

Matthias Bauer

But with that we were actually better than the ENI, the natural intelligence, which had a higher error rate in the process.

Joubin Rahimi

That didn't work out.

Matthias Bauer

Exactly, exactly. And we presented all of that, it was a big conference room and it's great, but we have to tell you, we're not doing the project.

Joubin Rahimi

Make it worse, make it worse, I can do that.

Matthias Bauer

Why not? Yes, someone worked out that over 50% of medical reports for occupational accidents still come in on paper. And they calculated that, in terms of process costs, it's more expensive for someone to remove the paper clip or take out that staple and put it in the scanner than for someone to type it in manually.

Joubin Rahimi

But that's really an objection-handling and a pretext.

Matthias Bauer

And I think: hang on a moment. That was the same customer who, somehow three, four months earlier, had said: we need to change something, because we're finding fewer and fewer people who want a job like this. And if we don't change anything and bring in more automation, we won't exist in ten years. That's actually completely strange.

Joubin Rahimi

We don't want to dwell on the customer now. No, no, no. But these are quite natural challenges we have right now. I think that's also born out of fear.

Matthias Bauer

Could well be.

Joubin Rahimi

That you say: okay, I can't change sales, don't want to change it right now. That's why we always ask: what happens if it's successful? How is it implemented? How do you do that? How do you do that as a team?

Matthias Bauer

Yes, indeed, we place more and more emphasis on saying it's not just the piece of technology, but ultimately also a change in mindset, a change in the organisation. In some cases, processes will also change, in some cases job roles will change. That's just how it is. You have to state that clearly. Today all of that is called change management, everything that revolves around it, and I think that's a very, very key factor. Especially this fear. I've noticed lately, I don't know why, but in the last few months I've had a lot to do with works councils and data protection officers, because there are also the wildest ideas out there. So the data protection officer comes along and says, they just hear AI, red flag straight away. Where you think: actually, the requirement for data protection is the same whether you have software with AI in it or software without AI. Exactly the same. No different. But somehow there's still this... Something resonates there. And of course with works councils, staff councils and so on, it immediately becomes about the situation of employees, which I fully understand, but ultimately it's also an opportunity.

Joubin Rahimi

Total.

Matthias Bauer

So it's also an opportunity to say you're ultimately transforming the organisation with the help of technology, in order to hold your own in the market too. It's not just a question of employee satisfaction, it's also somehow about taking the company into this era, in order to ultimately hold its own in the market, over decades too, and in competition.

Joubin Rahimi

Yes, mad idea. That way the company stays around longer. Everyone benefits.

Matthias Bauer

Yes, exactly. And I think it's important that this mental groundwork in particular is addressed at C-level, rather than just being dictated from above. You have to bring people along to some extent.

Joubin Rahimi

And I think that's a really, really great point. Change is important, processes are important, bringing people along is totally important. It's not about the tech. So it's got great potential. Yes, exactly. And if you want to unleash your potential, i.e. discover your potential, write to us. Matze, his team, the Fellows, have seen a lot and can help you in these situations, or also tell you: don't do that, do something else instead. Exactly. Matze, thanks for the insights, always great to have. And how do scouts do it when it starts? Is there a sign for that? Not really, okay. No? “Always ready” is of course more the farewell greeting, or you also say “good trail” there. Good trail, we say. Good trail. Exactly. Great that you watched and listened. Thanks, Matze.

Matthias Bauer

Yes, gladly.