insights! #119: 99.99% availability – how to achieve high availability in the public cloud

Warm welcome to high availability in the public cloud. Before we start with the topic, just a few brief words about me, but not too much self-promotion. I'm Marc Achsnich, Head of Cloud at synaigy, part of the strategy consultancy of TIMETOACT GROUP. We've been around for 14 years. We operate with a focus on e-commerce and cloud and try to build holistic solutions there, simply to enable a shared vision. synaigy is part of a group of companies. That means we're a total of 1400 people, 500 might take another month or so. And ultimately we're represented at 22 locations in the DACH region and count among the leading IT service providers. But what's today about? Namely, high availability in general. This has become more and more important, more and more of a bigger factor, in recent times. And high availability itself simply describes, first and foremost, the possibility that individual sub-components of an application can fail without it having a real impact on your application itself. The question is: who needs this fun? And there are generally two topics. One part is that you simply have a business where the infrastructure matters at all times. That means, no matter at what time some selection happens, it has a direct impact on your infrastructure. 

In a figurative sense, this means, is this an area where we're talking about e-commerce, for example, which simply has to sell 24/7, and there every minute counts, it's simply revenue being lost. The second area, and this is one that's becoming increasingly exciting now, is simply where someone dictates to you that it has to be that way. There are legal requirements, there are contractual requirements, and we're very often then in the area of banking, insurance, or generally critical applications. Within the public cloud, there's a relatively rewarding way to achieve this, and that's Kubernetes. I don't know your technical background, so here are a few very basic facts about it. There are three features that are very, very good and efficient simply for high availability. The first part is horizontal scaling. In a figurative sense, this means that it's possible for a sub-component of your application to exist not just once, but two, three, four, five times. And if two components of that then fail, others still exist, so that your application can continue to provide its responses in that area. In addition to that, there's load balancing. This means that these five applications are simply served with requests in equal measure. 

As a consequence, that also means every single component has less load and a calmer life. Ultimately, Kubernetes can carry out self-healing to some extent. In the end, only as good as you teach it to be. That means you can write a script in the end, you can define what's needed for this sub-component to be in a healthy state or count as defective. And in case Kubernetes detects that a component is simply becoming unstable, it will independently create a new sub-component and ensure that it can start up without load in that sense, carry out cache warming, and only then, once it determines that this component is reachable, swap out the defective one, thereby guiding a light self-healing process. Within OVH that's relatively easy to do. There's the OVH Manager, where you can click this together within a few minutes, and what's particularly nice about it is that on this basis the expertise you need for it isn't especially deep, and it's available across several regions. In addition to the standard plans, there'll soon be – I don't know if it's already been announced – in any case there'll soon be a premium plan that simply promotes even more resilience. 

This means it's simply possible that by switching to the premium plan, availability goes from 99.5 to 99.8, or whatever range that may be. I'm not familiar with that yet myself, but through this, simply by switching the component, there are then, for example, some redundancies created in the Control Plane, so that operation simply becomes more stable. Kubernetes can be configured very flexibly. You can select it in the area of high availability. This means it decides itself when... Now it mustn't catch fire and you're not allowed out anymore. In any case, in the area of flexible security, it's the case that you can simply define: do I always want, whenever a security patch comes out, to ensure it's applied ASAP? Do I want to define my own maintenance windows? In this area there's the possibility to configure things flexibly. The cluster can be set up in a private network, or it can be completely public. This means, depending on which setting you choose, a gateway is then automatically created, a public internet interface is created as well. And I believe almost all OVH instances have access. This means it's possible to select GPU instances, should you want an AI-heavy infrastructure. 

You can set up test environments where you simply choose cheap D or C instances, so small CPU-heavy ones. And there's a large bouquet to choose from there, and it's scalable up to 100 nodes. And in the area of high availability I find it particularly charming that there's a feature called anti-node affinity. Here you can, more or less by ticking a box, define that the nodes get distributed across multiple OVH physical servers. That means in the event of a failure on OVH's side, it ultimately isn't your problem, because you've distributed your infrastructure across multiple physical OVH servers. And it's not the case that one server fails, but you've put all your eggs on it and then the scaling doesn't work anymore either. And the last nice feature is that everything you do via the UI you can also simply do in Terraform scripts. That means fault tolerance can drop, reproducibility can rise, and what you do at one location you can then simply do again elsewhere via script. And I've been talking too long here already, because in that time the cluster has already been initialised. On the right-hand side I tried to keep to it, but locking the door threw me off, I think. 

Yes, well, S3 is also available quite normally at OVH. Actually, I have to say, we've never really had a real storage leak over the years. We've been there for four or five years, and storage has never been a problem for us. Generally speaking, I think you can say that you shouldn't rely on local VM storage. That's just a free gimmick you place somewhere — instead you should rely on block storage. And there we've really never had any problems, and we mainly make sure that we regularly back up and secure the storage itself. One option that would exist is that, right at the end, in the spoiler, there is eventually a 3AZ, where the storage is then simply distributed across three as well. That would be a case where you can simply create more resilience. Or there are other options, but then you get into an area where you have to leave the managed OVH environment, i.e. leave the managed OVH Kubernetes. There's also a kind of enterprise storage from OVH. That would be NetApp, for example, or DHAnas. Through that area you then get a genuinely resilient area as well, but bottom line — we ultimately never had, I actually don't even know the availability zone of the logs storage, but we've never had any leak there. 

Yes, we now have a high-availability system with all the storage types, I think, so how does synchronisation between the availability zones work? Is that essentially real-time or? 

It depends. So the 3AZ, that lives in its own magic. That means you don't have to worry about all that fuss. If you want to distribute high availability across multiple sites, it really depends heavily on latency. That means, depending on which data centres you choose, it's entirely possible to say you go for complete active-active synchronisation and then build, for example, your own MySQL cluster, which then exists, say, as a Percona or Galera cluster, so you simply set up two physical nodes. But you need the right locations for that. So what you can achieve is roughly up to four milliseconds or so. Above that you should rather think in the direction of an active-passive scenario and say there's simply your physical primary node, which replicates with a second server, and this ensures that the primary site keeps providing the responses the whole time, and only when everything's on fire do I have the state from two milliseconds ago and can then switch over to that. 

You said earlier managed Kubernetes is available in several regions. Can you also run a Kubernetes cluster across multiple regions, spanning two regions? So touching a bit on banking and payment, where geo-redundancy is discussed, and these days it needs to be a few hundred kilometres — is that already possible too? 

It depends. Maybe the question answers itself across three slides. Maybe. Exactly, let me carry on. So generally, very briefly, the basis of Kubernetes is: there's a... So everything at OVH starts with the vRack. That's one of those USPs of it. That means you simply have a completely virtual network which you can use across all the universes of OVH. That means you can take services from the public cloud, services from the private cloud, services from the bare-metal world, and they're all in the same virtual network. That's why, for example, it's entirely possible to say I'll combine my Kubernetes with HANAS or something, because in the end they can all play together in the same world. Built on top of that would be the Kubernetes layer, which then has a control plane and a data plane, in which you can configure your nodes. The control plane is something you simply don't have access to. That's within OVH's domain. They ensure it's available, and you only have access to the node pools you configure. Which likewise makes stretching across clusters more difficult. But more on that later. You can also replicate these clusters multiple times within one project. 

That means, in this case, you could assume, for example, that there's simply a production Kubernetes, a test Kubernetes, an acceptance Kubernetes and so on, all with different node pools, so that you have the right sizing, so that you don't have to throw out too many costs. And above that there are the OVH load balancers, which then simply handle access from outside into the Kubernetes field and control the scaling, or the load distribution, across the individual compute instances. Via DNS, the world is then ultimately made reachable from outside via the domain. The question is, and we'd already spoiled this a bit, with that you achieve 99.5%. That might be fine in many cases, but sometimes it simply isn't enough, going back to the first points in time we mentioned. Although it should also be said, I think we've had managed Kubernetes in the 99.5 variant in use for a good five years now, and not just once, but probably ten times or so. And over the years we've had one outage, but that wasn't down to Kubernetes. Something in the network control was completely broken, but otherwise it just keeps running along quietly. So I do have quite a bit of trust in it. We then also came up with a few alternatives to it. 

That means there are ways to go from 99.5 to 99.8, 99, you'd have to calculate or configure that yourself, and there are three options. One is Multi-Region Setup. That's what you just mentioned. That means you have a control plane that's simply available in multiple regions, and underneath that node pools that are then either in one AZ, in a second AZ, and on this basis you can design your infrastructure so that it doesn't just concentrate on one location, but is distributed completely evenly, or however you configure it, residing within this overarching Kubernetes world. This also exists at OVH. The second variant is Active-Active clusters. That means you simply choose two locations, deploy your application there, and put a load balancer in front of both worlds and let them be distributed evenly. That means the request comes in and goes either to Strasbourg or to Limburg, for example, and depending on which currently has less load, gets the request. In this case, you simply have to pay close attention to not having a synchronisation problem, because ultimately both locations can write simultaneously. 

Depending on how you've designed your application, it's sometimes easier, sometimes harder, especially when changes need to be persisted within Kubernetes, or when a Redis instance is supposed to sit there that suddenly isn't just a cache but also a persistence cache, because number ranges get generated in it, and then suddenly the connected ERP says: I already know this number, but the other data centre just created it. Those are things you simply need to think about. That means it's simply advisable to pull such things out of the co-tenancy, for example putting them into a Galera cluster, putting the persistence in there, which are then simply fast enough to keep their data in sync within two, three, four milliseconds. Beyond that it really gets difficult. The third use case is an active-passive cluster. That means you try to make use of the advantages of two cluster sites, but you actually concentrate on one location for as long as it simply exists, and only in that case do you switch to the second. It has the advantage in that sense that you can simply save a lot of latency, because you're simply always, for example, here in Nuremberg, but in the worst case you're back up and running within a few milliseconds, because the second site then scales up quickly enough and can become operational again. 

Yes? 

One follow-up question: are you with Active Passive? That would essentially be with a complete hot-standby buy? 

That's a hot-stand buy. Exactly. Meaning, it's shown graphically here. Now it's on the other side. 

Do we get good response times that we provide? Does it go via DNS, where it then automatically detects the error? 

We'll look at that in a moment. I briefly worried I'd be through this slot within 15 minutes, but not any more. Right, so with this active-passive part it's simply important to know which locations you have and how they can be combined. At synaigy we started a long, long time ago thinking about how this could be made simple. Which locations can you combine? And then we had the idea: let's just try it out. And then we started measuring every location against every other location, and that produced this beautiful matrix that you can't really read. But you can see, towards the middle they're fast, towards the outside they're slow. And on that basis you can consider: where should my primary location be, and what's then a good secondary location? Because I thought you wouldn't be able to read it, I picked out a few. And ultimately the simplest decision — if you're in Frankfurt, you should go to Strasbourg, because there you can typically be at four milliseconds or below. If your main business is, say, more in Paris, or in France, then there's either the 3AZ in Paris, or you'd typically go with Gravelines or Roubaix, because then you're talking about two milliseconds, and that's already very fast by the end, if you consider that within a single data centre you're in the region of one millisecond or so, or below. 

And for our colleagues from the UK, London would also harmonise very well with Grave Line. This table here isn't a secret to me either. So if you want it, you can get your phone out and take a photo of it, or ask me afterwards, then you can have it too. I can see already, I'll wait. Yes. 

Yes, otherwise you can also come here if you want to photograph it from there. Thank you very much. You'll land on the landing page, can enter some data there, or the correct ones, in any case right after that the PDF has exactly this image in high resolution. Right, in any case the question is, if you want that now, what does it look like overall? 

This is now the representation of the cluster I just showed. 

This would now need to be very easily adjusted. That is, we currently have vRack, managed Kubernetes, a load balancer and a DNS. And in this case, I would put another load balancer in front of the load balancer. The reason being that the public load balancer is simply a component that only wants to take care of its own Kubernetes cluster. And in the variant I want to sketch out, we have one in front of it that, in this scenario, would for example sit in Gravelines, because a very high availability is guaranteed there too, and it's then simply independent. And that one gets addressed — we see this now — it then simply has access to the two load balancers and checks, on that basis, whether to go to the left one or the right one. Right, that means, here for example the Kubernetes will break, the load balancer no longer gets a request from its Frankfurt location and would then, on that basis... Wait a moment. Does this work? It works. Will then, on that basis... I've turned into a graphics artist. Right. Will then swing over to the other location on that basis. So much for that. If you now want to know how this can really work. 

I've provided this here, I think, and because I didn't know if I'd have internet, I recorded it. On the left-hand side you can see … 

The right side is really good, here on the left side you see the classic OVH look, the classic OVH manager. 

On the right-hand side is Lens. Relatively little actually happens straight away there. So I'll start the video and try to show asynchronous alongside it now. 

So you can see here, we have a, as I said everything starts with the private network. We've created a network located in Germany and France, in DE Nuremberg, SBG Strasbourg, and then built two Kubernetes load balancers, which are then each available in the individual countries and share this common network. Both have their own load balancer, where you need to set up listeners and pools. In the listener, let's say, there'll later be a small web application. That means I need port 80 and, in the pool section, define a member area, to which the entire node pool from the one cluster is then added. 

From this point, that one cluster could already run, but only at 99.5. 

This here is the load balancer, which sits in the hierarchy on the bare-metal side. There's one, well there are different ones, the load balancers, they're all called load balancer here. And it's important that you essentially also have several … So this is now the IP of the public load balancer at the individual site and there you can enter a configuration so that there's a backup and a primary location, and the clever bit here is – I had to speak really fast there –, that you just saw there's a category called "First. 

That means it doesn't consider whether to go left or right, but always pushes its traffic to the main location until that no longer exists. This simply checks every 30 seconds: is my location still reachable or not? 

We'd see that here now. That's the location. As you can nicely see, a beautiful Hello World page. And what's happening on the right-hand side now is that I'm removing the north, and on that basis simply symbolising that the cluster isn't there. 

We can talk about storage again too, promised. Right, but later. 

Look at the note again. 

Right, so. 

Now let's get to storage. Bridged that nicely anyway, I drained the path. That means the traffic isn't there and it now takes a full 30 seconds for it to detect that its location isn't there, and from now on it's straight into SGB, or I've already spoken far too long, it tries to swing back again. 

I'm thinking yes, definitely about 30 seconds is roughly how long he does that. 

That's quite long. 

That's a long 30 seconds. There is actually also the option of not doing that with the … 

Now that's actually right, in "SGB" and now I'd make it available again. I think you can see it. Now you really can't see anything at all. But there's the button to uncord it now. 

Right, so now traffic gets routed back to its primary location. In this case the smallest setting on this IP load balancer is 30 seconds. There are also scenarios where you simply don't use this. For that, though, you'd need a DNS, sitting for example at Cloudflare, where you can simply set up two A records, to carry out a kind of DNS failover there. But because I wanted to script everything via Terraform, I focused in this example simply on the load balancer, via the load balancer. 

And exactly, if you want that, it's all also provided as Terraform with a short guide, and what I've just described you deploy at the push of a button, so to speak, so you can also take a closer look at it yourselves afterwards. 

Exactly. And the game starts again, right there, if you want that too. Yeah? Was that a multi-reason multi-cluster setup? That wasn't a multi-reason multi-cluster setup. That was a failover setup. Meaning you have two, and that's the drawback of this variant. With classic Kubernetes, you can't really distribute across multiple locations, because each location has its own control plane. For a multi-region cluster you'd need a spread-out control plane that can decide where it wants to distribute its load. There's another variant for that, but it's not available in Germany. So is that then a single cluster multi-AZ? This here would be a classic failover cluster, like an active-passive setup. That's a three-way cluster, that's what I meant. What you meant. One cluster, several. Exactly, meaning you just have one cluster, and I'm not sure I can show it because the projector's black. But here you can see it. Right, so this here, I don't think I can quite... I quite like black. That's the background. 

Let me just check quickly whether I here … 

Normally you can always switch on a nice white background too. So, here, there. Right, let's see how far we get here, but I don't know if I've still got it up. Unfortunately I can't show it. I'd have to check that in a moment. Otherwise I could perhaps just show it here afterwards. So there's also the option of displaying a 3AZ. Exactly, there you simply have the option of delivering a control plane that's distributed across several data centres, and on that basis a proper multi-region setup is then possible. Can that only be supported? No. Exactly, also? Soon in Milan too. 

Balt in Milan, okay. 

Exactly. And because that's the case, we thought about how we could ultimately get the 99.5 better. At that time there wasn't yet an announcement of a premium plan. I might have saved myself all these thoughts, but so be it. Anyway, what's additionally available are these two variants, and that means on left side the premium plan. That means, in that sense, everything stated here is currently actually what applies to the 3AZ. 

And I haven't seen a proper guide or description for the 1AZ yet. 

That definitely means, so if you This is then analogous to an offer to what exists on Azure, on AWS and co. There are also these premium plans there, where redundancy layers are simply built in, where you talk about a highly available control plane, so that it's simply not there just once, but that OVH ensures redundancies are created there too, on this basis then simply moving up from the 99.5 somewhere. I thought, hoping for 99.8, because then it would be exactly what you can get with Azure and AWS, but I don't know yet. Maybe you know. Do you know? Okay. Exactly, and the second setup is the 3-AZ, and that's more in the direction of the crème de la crème of Kubernetes, because up there it really goes, you have one control plane. You can use the cluster as if it were simply one location. The latencies are so low that you can essentially distribute your application completely, so that some are in one AZ, some in the second, and if one were to fail, it can automatically distribute this cluster's load to other node pools. 

How that would roughly look, you can see here: At the moment there's only EU West, Paris A, B and C, where the control plane is ultimately spread across. That means, is that exactly your scenario in the end, and you can manage that flexibly there. I thought at first, five more minutes, I can manage that too. Right, I thought at first it would be some kind of active-active cluster, that you simply mirror everything you do on one side, because that's how the database works in the world, but actually you really can design it freely. You can say, I have an application, I'll call it A, I'll have that distributed to every single location and thereby ensure high availability. You can equally say, I just want one pod that should only be in location B, or I want to concentrate entirely on multiple nodes in location A. That's all configurable via instructions. And for that I haven't written any instructions, because OVH has already done that. 

And that's here on the right-hand side, or on the link you can clearly see down there. 

So search for OVH 3Z, that's somehow the first link for it. And the great thing about it is, there are completely ready-made Terraform scripts for how you can then also do a deployment there, horizontally or vertically. And yes, so that definitely makes getting started a lot easier. The last question, which I think was already half answered, was: why do I go through all this hassle when there's a 3-DC setup? Because this 3-DC setup is simply a very good service. You get to 99.99% with it. That's a genuinely multi-region service that can really be competitive too. For me it would be Paris and Italy. I'd be glad if there were one in Germany at some point. We all still need to talk to Falk about that again. Right, and because it simply doesn't exist, we thought about it: what could a world look like without a 3-DC setup that's better than what you have? And that's a compromise approach that's just as flexible, so you can say: I'll take Frankfurt and Strasbourg, I'll take UK and Gravelines, or, or, or. And by choosing active-passive you're also a bit freer from latency. 

Four milliseconds is brilliant. For active-active, in that sense somewhat borderline, but still great. But when it comes to active-passive failover, you basically focus on Frankfurt, then build a database layer, for example, more via MySQL replicas, and then if Frankfurt ever burns down And if it does, it only burns in Strasbourg. Otherwise someone always says it. Right, in any case you then have enough time in that scenario. The replication then works without issue and then you can switch over, and 30 seconds isn't the end of the world in that sense. And if you need more, you take the 3RZ. So much for me. That's my LinkedIn. Otherwise no further names need to be entered there. I look forward to the conversation. I don't know if we still have more time for questions. I'd ask once more, basically, whether there are still questions.