insights! episode #63: data-driven publishing: how "Kölner Stadtanzeiger Medien" achieved an 80% CTR boost with AI

Joubin Rahimi: 

Fantastic to have you back for a new episode of insights. Today with a special guest, namely Robert Zilz from Kölner Stadtanzeiger Medien. And yes, the KSTA is a client of ours when it comes to analytics, but that's not what today is about. We might touch on the topic briefly. But Robert is Head of Data, meaning analysis, data science, AI topics, lots and lots of exciting topics beyond that too, where you're hugely innovative. So I'm really pleased that we can welcome you here today. Hello Robert.  

Robert Zilz: 

Joubin, thank you very much for the invitation. 

Joubin Rahimi: 

I'm pleased too. Very much so. You're the paper read by our developers and analysts, because they're all Köln fans, football Köln fans that is, and you're held in very high regard there. But regardless of that, with Kölner Stadtanzeiger and Express you have two media outlets here in the market that we know. But the industry is changing enormously. I'd like to talk with you a bit about how you're responding to all this. 

Robert Zilz: 

Yes, very, very, very gladly. 

Joubin Rahimi: 

So first of all, how did the Google update hit? 

Robert Zilz: 

The last update, the Helpful Content Update, has indeed, I'd say, had quite an effect on a great many publishers. Simply a massive drop in SEO traffic. We're seeing this in our own figures accordingly too, of course, and have to respond to it. Meaning Google and Search as a reliable source. Honestly, it never really was, to be fair. But even regarding it as a reasonably reliable source is definitely no longer the case today. And when we look at things like the SGE developments, so also the topic of AI in the search space there, these are transformative, disruptive measures that of course also affect publishing and of course affect us too, and of course our strategy for how we handle that and how we regard Google and Google traffic as a traffic source going forward. 

Joubin Rahimi: 

And you speak straight from our hearts with what you want to say, because we say we can't entrust ourselves to hyperscalers of the Google, Facebook and Metas of the world. They have their own agenda in that context and their own plan. And you, I'd say, you're making the Netflix move. So from a business you've been running for a very long time, well name the figure, but it's really unbelievable. You could all take a leaf out of that book, being on market that long. 

Joubin Rahimi: 

Yes, I mean, as a publisher we have a tradition going back many hundreds of years behind us. It's really something, and of course we're keen for that to continue for a very, very long time yet. And honestly, those four hundred years, they're only achievable, and accordingly we've only been in the market that long, because – and this is a story we like telling both internally and externally – because we've always transformed ourselves. So we've embraced change accordingly at every stage of our history, and accordingly we now naturally have another change ahead of us again. I think we all have this ahead of us, especially in the field of artificial intelligence, which is now having an increasingly disruptive effect. Exactly. And that's simply our personal change right now, that we have to respond to.  

Joubin Rahimi: 

And there I think you can really look at your industry, or what you do, from the outside first. Because there are many industries that are still lucky. They don't have to change so much currently, but it will certainly come with AI. And you're leading the way. And it's not just products you're changing and processes, but you also said there's the main topic.  

Robert Zilz: 

Yes, the main topic, people and culture, meaning bringing the people in the company along. I'll quote our CEO, who, when it comes to AI, says it's only 20 percent tech and 80 percent people. And honestly, that's exactly how I see it too. And we have to go along that path and view it as a central factor in our change management.  

Joubin Rahimi: 

And what about you? You've positioned yourselves a bit differently, but you've also developed new products and been recognised for it too. I'd love to hear about that, it's not really a secret insight, but we should talk about it more anyway. Tell us a bit about your product and what it does and what the added value is. 

Robert Zilz: 

Yes, generally perhaps. We were already at it long before the ChatGPT topic came up last November, which really caught public attention accordingly. But before that, and we're talking two, three years before that, we were already experimenting with machine learning. So really, behind the fancy buzzword AI, it's the machine learning topic. What's behind it, and we experimented a lot with collaborative filtering. Back then still on one of our football verticals, Glattbach Live, a monothematic product. 

Joubin Rahimi: 

Why not Cologne Live? 

Robert Zilz: 

We wanted to try it first with the Glattbach readers. Maybe the next vertical is a good idea, Joubin. Perhaps we take that as an idea. But Glattbach Live is actually also with us from Express back then, I don't know if you remember, as a heavy user, it was one of the sections alongside 1. FC Köln of course, which was consumed very intensively. And that was really the pilot for us. And there we also played around with an article recommendation, wrote it ourselves and also applied discipline, which you might also know from ChatGPT, when you rate ChatGPT's results with thumbs up, thumbs down, we then also applied reinforcement learning accordingly. At that time, using the latest technologies, we were able to gather our experience and then actually placed a recommender on the homepage with the Kölner Stadtanzeiger last November, which simply achieved fantastic results and for which we have now also been awarded a prize. 

Joubin Rahimi: 

What does fantastic results mean? 

Robert Zilz: 

Plus 80 per cent CTR, so the click-through rate rose by 80 per cent. Of course, always challenged against manual curation by an editor or the editorial team. And we also had downstream quality KPIs, such as reading behaviour on the recommended article, which rose by 13 per cent. So really a very, very cool product. 

Joubin Rahimi: 

What does that do, you said, people is also with the managing directors, people is the most important thing. What does that do to people when they see they're losing this game in the KPI competition.  

Robert Zilz: 

This is of course a very, very important topic, and honestly a topic that hasn't just accompanied us since the AI buzzword came along, but also with the topic of data analytics before that. Because you naturally create transparency with it — in the best case you achieve that with it — and can act in a very data-driven way. And data-driven action doesn't have to overrule everything. There's still instinct too, and the journalist who has a very particular feel for certain topics. But there are a lot of insights to be had. And of course there's always a certain defensive stance and a certain resistance you can sense, where you naturally have to look at how you bring people along to work in a more data-driven way. Not even just from the angle of encountering certain fears and rejection, but also access to data, for example. When we look at data, it's usually via some kind of interface like screens and so on. And my team then set itself the task of, for example via a voice assistant — so via speech as an interface — approaching things differently and making data-driven work much more accessible for all stakeholders in the company. And that's naturally something we're working on very intensively, my team is working on very intensively too, to open that up accordingly and make it available to everyone, and to establish understanding of and the topic of data-driven working.  

Joubin Rahimi: 

And I have... I have three points to tie in there directly. I don't know if I'll manage all of them one after another. But the first is, if you have the tool that already plays out the first page, let's say, mega personalised, you've got one-to-one personalisation. Naturally you can't tackle that with people alone, that topic. So maybe you don't even need to see it as a competition, but rather how can you combine the two and how do you lead it? But how do you handle measurement? Because if every single person then has their own website, that's a real challenge. What actually works? 

Robert Zilz: 

Yes, it's definitely a really big challenge. And we made the whole thing measurable in the classic way, via an A/B test. You can also set the whole thing up as multivariate, but we really did say, we have the control group. That's the group that's then steered and populated editorially accordingly, which then also serves as a fallback if need be, should something be off with the recommendation, etc. And then we introduced group B on our side and rolled it out to a certain percentage, and then we really measured every signal, every interaction with the recommendation, as soon as the recommendation enters the viewport, as soon as there's interaction with the recommendation. And that way we could track really, really, really well which type of recommendation, we've also got various models running in the background, performs how well, lands how well. And that's how we were able to make that CTA increase visible accordingly. 

Joubin Rahimi: 

And is the whole thing in real time? 

Robert Zilz: 

The whole thing is genuinely real-time. It also has to respond very quickly. So we’ve set ourselves the requirement that the recommendation’s response time is definitely under 300 milliseconds, simply for reasons of user experience. And on the other hand, in the case of the Kölner Stadtanzeiger recommendation, the entire model is now completely retrained every 15 minutes with all the data we have available. That’s a lot. The training takes a few minutes, and then the recommendation is available again and can be served again. 

Joubin Rahimi: 

So really exciting. You say that so easily in subordinate clauses. For everyone else who's also generating some revenue in e-commerce. Robert is already leading the way when it comes to microtargeting or one-to-one targeting and analysis behaviour, that it simply presents entirely new challenges, but also the potential within it. Absolutely. The second question I ask myself: who is your real competition? I only have limited time, I have one screen. And is it, when I read Express or Kölner Stadtanzeiger, it's not the Hamburger Abendblatt or the Berliner Zeitung, but TikTok, Instagram?  

Robert Zilz: 

I'd think the same way. I'd really move on a meta level. And what's the competition? Honestly, we're operating in an environment where we have a certain... a certain slice of the cake in media spending per day with each individual user of ours as well. And we naturally try to shape that as much as possible. That means, honestly, the competition is just as much Netflix, if we're on the couch in the evening, it's TikTok and co. So in that sense that's our competition too. And we naturally have to look at how we counter that with our content products, and of course bring in and convey the asset of journalism. And that's actually the competitive situation we have to face in this digital world - and I'm only talking about the digital world from our perspective.  

Joubin Rahimi: 

It was interesting on the meta level. Short anecdote from Netflix, I read it somewhere in a study once, they also said, who is your biggest competitor? Sleep. That's why episodes play, one after another. Although, they're not running adverts at the moment, but that could still change. But okay, that's your competition. Then for me the next question is, is it different in the way content gets delivered. Is all of it still written by human hand? Question mark? 

Robert Zilz: 

Also an interesting question, Joubin. Actually quite new now. I won't name the publication. We were also criticised, I'd almost say, by another publisher for our AI journalist. We call the AI 'Indernacht' for short. And she actually has her own profile on Express, with a picture generated with Midjourney, of course.  

Joubin Rahimi: 

When you commit, commit fully. 

Robert Zilz: 

Exactly, once it's carried through. And this colleague actually writes, in our case, 4% of the articles at the moment, and of course it's not fully automated. That has to be said. I think that's also a bit of a pipe dream that's currently being built up a lot in the field of generative AI. We have guidelines, we have editorial policies, so that a human always still looks over it. But Klara is the trigger for many articles, or also the inspiration, to find a certain trigger, to possibly rephrase texts as well. And that's actually growing more and more, more and more with us. And we're also noticing that more and more traffic is being generated accordingly via Klara too. And of course, and this is the important part, it's not meant to replace the journalist, but rather to free up resources for the journalist, who then isn't tied up with tasks that aren't actually part of their core profile anyway, like page management etc., and can put their resources fully into the topic of story, storytelling, journalism accordingly.  

Joubin Rahimi: 

Okay, obvious in a way. But you still have the challenge, TikTok has millions of users creating content differently. Now you have a great tool that helps you. Is it just for you, or do you say you'll market this tool for other publishers or others outside the industry too? 

Robert Zilz: 

Interesting question, Joubin. And at this point I don't want to give too much away yet. But this much can be said, we are indeed thinking about offering that to other publishers accordingly too. Far beyond the Klara topic, far beyond just the AI topic. We're really talking there about frameworks that we've introduced completely from scratch, with a single page application in mind and also the CMS behind it, which for us is also then, for example, augmented via further, in this case again AI applications like an NLP service. That's already in our minds, that one could offer that to others too. And that's amazing of course, because it shows you're right in the middle of transformation with this. The business model suddenly becomes something completely different.  

Joubin Rahimi: 

If you did that, you'd suddenly become a SaaS provider and among other things a SaaS provider.  

Robert Zilz:  

And honestly, that isn't so far removed from our business modalities overall. If you look at the group as a whole, perhaps. We're continuing to diversify, and this could be a further expression of that.  

Joubin Rahimi: 

And with that I'd actually like to leave it, because I'd say that's already so huge. I don't think anything after that can be quite as exciting. So you're topping this now.  

Robert Zilz: 

That's actually the core of it. 

Joubin Rahimi: 

So thank you Robert for being here. I'm looking forward to comments, or rather we're looking forward to comments. And if you have questions, that's really the format: write to us and we'll gladly put them to Robert, or Robert will answer them directly if they're posted below and want to open a discussion. 

Robert Zilz: 

Very happy to. 

Joubin Rahimi: 

Thank you for the time and for all the information. 

Robert Zilz: 

That was great. Thank you. 

Joubin Rahimi: 

Thanks for that.