← Back to the blog
Data security

The day everyone’s data went up in flames, and how Lifen saved the day for France’s largest healthcare provider network

Solid recovery protocols, automated coding and stellar customer service rose an entire data infrastructure back from its ashes in under 48 h


Data loss is probably the last thing one worries about when a fire breaks out. At the cry of ‘FIRE’, you probably picture thick smoke, smoldering flames, crackling wood and bending steel. Dozens, maybe hundreds of men in heavy duty reflective gear, shiny red trucks and hoses being thrown around. Dismal infrastructure loss. And then sometimes, catastrophic human loss too.

But in the wee hours of the morning of March 10th 2021, the cry spelled ‘FHIR’, a lesser-known type of inferno. The people standing in front of the five-storey, 500 square-meter unit watching the flames rise tens of meters high, were not particularly worried about infrastructure or human loss. For there were no people, nor material goods of value inside the SBG1, 2, 3 and 4 units of the datacenter owned by French company OVHcloud. And yet, the apprehension was palpable - whatever burned inside those walls was infinitely more valuable than the sizzling circuits and shattering screens. Inside lay the very foundation of a big chunk of the internet.

For dust data is, and to dust it shall return

The OVHcloud datacenter, in the outskirts of Strasbourg, could look unassuming, but little did the general public know that it played the role of an umbilical cord to thousands of websites and web services across Europe.

Amidst them was some of Ramsay Santé’s most critical medical data… and the data management system that goes with it. Ramsay Santé, Europe’s largest private healthcare enterprise, weighs no less than 350 establishments across five countries and 8,600 practitioners who deliver care to a whopping 7 million patients every year. The operating system behind Ramsay Santé’s medical correspondence and health patient record integration capacity is Lifen, a solution that uses cutting-edge Artificial Intelligence to optimize healthcare workflows, deliver optimal patient record completion and save staff precious time by eliminating the bulk of all repetitive administrative tasks. And the portion of Ramsay Santé’s data that Lifen’s managed just so happened to be stored in the OVHcloud datacenter, as requested by the former.

In the midst of a global pandemic, and just days after French President Emmanuel Macron declared the beginning of a third nation-wide lockdown, it was inconceivable for Ramsay Santé staff to go back to manual typing, record filing, and faxing for an extended period of time. The resources needed to revert back into an archaic workflow were not even available anymore. And, most importantly, with the Lifen OS offline, there was no guarantee that Electronic Health Records would be filed properly, resulting in suboptimal patient care. Critical clinical data would take longer to make its way from the outside into Ramsay Santé’s practitioner dashboards, hampering diagnosis and decision-making in situations where clinicians simply couldn’t afford to wait.

A lesser-known, albeit significant consequence of losing all data flow assistance would have been financial. In France, Lifen acts as more than a technological partner - grants are allocated to establishments that reach a certain level of digitisation for medical communication and data structuration. This was the case of Ramsay Santé, and of so many other healthcare networks that see in their synergies with the Paris-born health tech company more than just an open door to novel tech. On top of the significant loss of government aid, add the financial burden of growing backlogs, increasing mail budget, contract staff, and a generalized patient disengagement due to suboptimal care. The healthcare enterprise is, after all, also an enterprise.

Learn more about Lifen

Get in touch with us

And from the ashes it shall rise

Three hundred miles west, as the sun rose above a drowsy Paris, phones began chirping with the day’s first (of many) pings. National media was all over the story, but it was unclear if the average web surfer fully grasped its implications.

In the Lifen offices, engineering staff - who had been up since sunrise as they learned what happened - watched as VPNs went dark, host indicators were lost, and memory alerts started flashing right and left. Indeed, one of OVHcloud’s main units had been destroyed, and another had been irreversibly damaged, effectively wiping out most of the Ramsay Santé data managed by Lifen.

While a datacenter fire is the stuff nightmares are made of, it is part of an extremely remote array of tragic possibilities few can predict. Lifen’s engineers had, however, crafted a solid recovery plan in accordance with the company’s local and international ISO certification requirements, should a sudden and unfathomable host loss situation ever see the light of day.

In just a few hours, Ramsay Santé received not only the bad news, but also the reassurance that a full recovery plan had been deployed. The latter was reviewed by Ramsay IT decision-makers and signed off. Lifen had the green light to begin rising the phoenix from the ashes-  data had to be recovered, verified, and then hosted on a brand new infrastructure. In its essence, the process is straightforward, but it is not always without its obstacles and pitfalls.

Emergency landing

The first step in any data recovery operation is, logically, to get elbow-deep in the backups. If you’re lucky, and if your customer is serious about their recovery protocols, half of the work is done. Fortunately for Ramsay Santé, Lifen had set up recovery backups in case the former’s data needed to be quickly restored. Thus, Lifen was able to begin recovering the backup right away, with minimal manual coding.

In fact, Industrialized Infrastructure-as-Code, the managing and provisioning of infrastructure through code, instead of through manual processes, was pivotal in helping the team produce an almost exact replica of the original infrastructure, in just under a few hours. The recovered infrastructure was then safely stored on a new host, AWS. This was no arbitrary choice: as part of its disaster recovery protocol, Lifen had designed AWS to be its backup host to the compromised customer environments. Much in the fashion a pilot maps the ground for nearby landing strips, shall he have to emergency land.

Once the applications were back online on the new environment, the backup restored, VPN bridges rebuilt (and oh, had they been burnt!) the whole system had to be checked. This is a step that cannot be skipped: worse than losing all your data is the prospect of only losing a part of it, but not truly knowing which. How would this translate onto a clinical setting? Not knowing if the most recent available X-ray records are up-to-date, or if there have been any recorded decisive developments in your patient’s pneumonia since. Sometimes, a thorough check is a matter of life or death.

Ramsay Santé had its Lifen OS data and full system up and running. The whole process took exactly two working days, with a whooping 95% of data being recovered within that time frame.


Lifen’s ability to recover data - literally from the ashes! - at such a pace, can be explained by the golden triad of disaster recovery: there were solid incident response protocols, whatever tasks could be automated had been programmed to be so, and the team of experienced engineers had benefited from sturdy training on how to respond to such scenarios.

Add to that a genuine commitment to customer happiness, as well as a generalized sense of responsibility towards the bigger picture, a value that Lifen puts at the center of its human resources policy. In French, they call it simply Bienveillance.

And more than just a fancy French word, Bienveillance can also translate into a very tangible contractual commitment, established between Lifen and its customers, to ensure that designated service-related goals are met. This is called a Service Level Agreement, and Lifen wears its 99,9% badge of recorded spotless service with pride. In this context, spotless service means no service disruptions - if the system is down for any longer than 44 minutes, as a provider, Lifen could incur financial penalties such as crediting the customer back.

Lessons learned

Datacenter fires are not an everyday occurrence, but they’re far from being the only threat to your data. In fact, in cyberspace, cyberthreats are garden-variety issues: think ransomware, hosting provider unavailability… or simply good old human error. Lifen’s CTO Dali Kilani offers a few words of advice for healthcare organizations concerned about preserving the integrity of their data under all circumstances:

Double-check your backups

But first, you need to carefully survey the scenarios that can lead to you needing a backup in the first place. Then, it’s time to set specific parameters, such as recovery time objectives, and maximum admissible data loss. Only then can you plan accordingly to set the backups you need. But that’s not the end of the line: remember to monitor backup operations and to regularly validate backup files by restoring them using the predefined recovery procedures.

Triple-check your providers have recovery protocols

Not all providers are alike, and some have more experience than others when it comes to worst-case scenarios. Make sure to ask the question during the auditing process, namely how long they believe it would take to recover and rebuild an infrastructure shall an unexpected event arise.

Automate your code

Solutions such as industrialized Infrastructure-as-Code can save the day in the event that your system goes offline due to a host failure. Having your entire architecture described in industrialized code means that you can easily have it reproduced. This requires minimal human intervention, thus avoiding human error and saving your team precious time.

Service Level Agreements

When choosing a new provider, it is important that you consider enforcing this type of engagement, as well as verifying the historical percentage of customer goals met. Think of it as a guarantee that you will not be dealing with frequent disruptions, and that if things do occasionally go South, you can claim adequate compensation. The interest of agreements of this kind only goes to show how a SaaS partner can make all the difference.

Regular communication

What’s worse than a fire? A fire roaring outside while you stand in a room full of smoke, in the dark. This is exactly what it can feel like on the customer’s end, and believe us, it’s just as nerve-wracking as it sounds. But it doesn’t have to be. If you’re a provider, make sure to practice excellent communication with your customer, and to keep them updated on the recovery status of their data at all times. Be realistic with your estimations, and transparent about the roadblocks. It might be useful to include a designated Recovery Officer in your protocols, and prepare them to act as your main spokesperson and point-of-entry when incidents of this sort happen. If you’re a healthcare establishment, make sure to request accurate ETAs, as well as thorough updates on the integrity of your recovered data.

Remember: lives depend on it.

Lucy Chambel

Lucy is a writer, media specialist and proud polyglot. She is passionate about innovation, linguistics, and art.

Read our posts

Get in touch

Book a chat with our team and discover how Lifen's solutions can be tailored to fit your needs