How holographic technology is helping doctors deliver better care

by Scott Leaman - Mixed Reality Leader, Sopra Steria | minutes read

Long gone are the days when holograms were the stuff of sci-fi movies and video games. Holographic technology is taking the medical world by storm, and by the looks of it, it’s here to stay. So how exactly is this technology helping doctors, and what are the major developments that we expect in the near future?

Back in 2016, Sopra Steria Norway acquired Microsoft’s HoloLens device to test potential applications of holographic technology in healthcare. What started as a test pilot in a few university hospitals quickly grew into the HoloCare consortium, a collection of initiatives across several countries that are developing holographic technology to help doctors deliver better care in fields as wide-ranging as surgery, tumor localization, medical training and remote therapy (or telehealth). 

Understanding “mixed reality”

The first thing to understand about the HoloLens is that the technology underpinning it is different from virtual reality (VR). Whereas VR is about using a device to immerse the user in another digital world, augmented reality (AR) is about interacting with the real world, but with an additional layer of digital objects. 

The HoloLens occupies a third and altogether new realm: that of mixed reality (MR). This means that a person wearing HoloLens can engage and interact with holographic objects in ways that were unthinkable only a few years ago. 

The rise of x-ray vision, and more

In the medicine field, that interactive ability has led to some groundbreaking developments:

  • Hands-free: While operating, surgeons can now keep their hands free as the hologram assists them with specific instructions, thus eliminating the need to ever manually check devices such as tablets or phones.
  • 3D imaging: A team of medical professionals planning for surgery can use the HoloLens to project a 3D image of, say, an infant’s heart. They can observe its real shape and “zoom in” on anatomical relationships. This shared objective view means problems can be spotted sooner and potential solutions can be identified preemptively.
  • Navigation: Before the HoloLens, surgical pathways could only be planned using 2D images that were overlaid on a map of the body. Now, holograms allow surgeons to literally place a map inside the patient’s body, enabling them to better navigate around certain tissue structures and avoid unnecessary damage.
  • X-ray vision: Holograms can also be used to observe the status of organs or bones in real time. Take, for instance, a patient suffering from a bone malformation. Normally a doctor can’t see inside their body—they can only record the patient’s symptoms, meaning a lot of guesswork is still involved. Hologram technology allows you to actually see the bone structure and observe how it’s moving in real time. As one doctor recently said in an interview: “it’s like x-ray vision”.

Shift in communication

Beyond these immediate achievements, holographic technology will also completely disrupt the way medical professionals communicate on the job. Currently, medical science is built entirely around 2D imagery tools. So when students or doctors study the body, they use medical diagrams and communicate what they observe using words. 

But language, as all codified systems, is imperfect. Holographic technology gives medical professionals a complete vocabulary of a disease or malfunction; a way of showing the specific problem without having to explain it with words—and thus of avoiding misunderstandings.

This change in communication will have huge consequences. Holographic tools will enable volumetric communication across different continents and timezones—features otherwise known as 3D Skype calls, or “holoportation”. This will transform the medical industry by eliminating doctors’ and patients’ need to travel.

Obstacles along the way

In many ways, mixed reality (MR) devices like the HoloLens are dependent on machine learning. While the ability of such devices to understand complex contexts will massively evolve in years to come, there is still some way to go.

Currently, devices like the HoloLens understand you’re in a room, but they do not understand the social context (are you by yourself in the room, where in the world in this room located, etc.)

When it comes to developing mixed reality devices, the key is therefore to simultaneously develop computers’ understanding of complex scenarios. And for that, we need to tackle two challenges involving data and design: more data to sync with the world and reconstruct it, and better design to enable users to interact with it in a natural and seamless way.

What the future has in store

Can holographic mixed reality devices conquer wider audiences and extend their influence beyond their specific applications in fields like medicine? As sci-fi author William Gibson once wrote: “The future is already here — it's just not very evenly distributed.” 

While mixed reality devices are already available for purchase on the US market, they remain expensive and relatively niche. That’s because the difficulty in selling a VR headset or an AR device is that it’s not an iteration of an existing need—you’re creating a new and unfamiliar experience, which people need to be coaxed into trying themselves. 

When you’re competing against other devices for entertainment experiences, such as smartphones and game consoles, the entry barrier is therefore quite high.

That might change, however, as perceptions shift. As future generations move beyond the conception of digital as “fake things on a screen,” digital objects will increasingly be treated as yet another material, similar to plastic, wood, and metal. 

That reality is closer than many of us currently think.

More on this topic

Sopra Steria: a key company in innovative ecosystems

"In our open and connected world, no player can claim they have full control of the value chain including innovation. Innovations emerge every day from all digital players, whether they are large companies, start-ups, private and public research laboratories or competitive clusters.” Jean-Bernard Rampini, Executive Innovation & Corporate Venture at Sopra Steria

Supply Chain Management in Aerospace: maximising agility with AI-based risk monitoring

| Benoit Spolidor, Maxime Claisse

One of the main challenges of today’s Aerospace Supply Chain Practitioners is to manage their operations in such a complex and volatile environment. The Supply Chain purpose of fulfilling customer service promise while controlling costs within the overall industrial chain has become harder, in particular because Aerospace manufacturers are facing a lack of visibility in their supply and delivery processes.

How can Artificial Intelligence support the performances of Aerospace Supply Chain?

| Benoit Spolidor, Maxime Claisse

Artificial Intelligence is having a positive impact on almost every industry. It improves decision making processes, creating fast and consistent operations management. In the specific field of Aerospace, our conviction is that to be fully efficient, AI must be developed with dedicated characterics. Sopra Steria invests on these features for sustainable and large scale transformation by AI for Aerospace companies.

Remote experts help technicians on-site

| Torbjørn Meland

New technology helps maintain production and increase productivity at operating facilities by reducing the need to send technical experts between factories. By using HoloLens 2, Microsoft Teams, Intune and Dynamics 365 combined with a design-drive process, you can get a solution that gives on-site technicians support and help from remote experts.

AI lead Software Engineering: Sopra Steria Ecosystem Offerings

| Jérôme Perdriaud, Satish Srivastava

Apart from internally developed IP’s given in the previous edition we also have an ecosystem of mature market leading companies, start-ups as well as labs and universities to build competency in their offerings and use them to help our clients. Following are some of the offerings from the ecosystem.

AI led Software Engineering: Sopra Steria Offerings

| Jérôme Perdriaud, Satish Srivastava

Sopra Steria has been investing in AI led software engineering in order to help our clients not only reduce cost and gain efficiency but also empower their businesses by making the processes more responsive and scalable.

AI led Software Engineering Use cases: Application to Testing, Deployment & Operations

| Jérôme Perdriaud, Satish Srivastava

In the previous edition of the series, we have seen how AI transforms the software engineering lifecycle, specifically Management, Requirements, design and development phases. In this edition we will see how subsequent Testing, Deployment and Operations activities are affected by AI.

AI led Software Engineering Use Cases: Application to Development

| Jérôme Perdriaud, Satish Srivastava

In the previous edition of the series, we have seen how AI transforms the software engineering lifecycle, specifically Management, Requirements gathering, Design phases. In this edition we will see how software development activities are affected by AI.

AI led Software Engineering Use Cases: Application to Requirements & Design

| Jérôme Perdriaud, Satish Srivastava

In the previous edition of the series, we have seen how AI transforms the software engineering lifecycle, specifically Management phases. In this edition we will see how Requirement engineering is affected by AI.

Innovating in Pursuit of Climate Action and Environmental Sustainability

| Avinash Lunj, Siva Niranjan

From reducing carbon footprint to improving energy efficiency, the surge of sustainable business continues to increase in prominence. To attract new business, talent and investment, companies are required to demonstrate, that they are putting their climate change strategies into action.

Digital Innovation Factory: Which technical platform select and how operate it over the time?

| Béatrice Rollet, Simon Herd

As seen previously, digital experience and platform offerings call for a massive amount of software with frequent new services, and regularly updated and deleted new features. Long-established companies adopting an Enterprise Platform model must then own a new Digital Innovation Factory encompassing a Technical Platform.

Digital Innovation Factory: How to reshape your software development activities at the era of cloud-native application?

| Béatrice Rollet, Neil Anderson

60% of backend developers use containers in their work. Relying on cloud-native technologies, defining as modern applications packaged in containers, deployed as micro-services, running on elastic infrastructure, and managed through agile DevSecOps processes fits very well with large enterprise who very often encompass a wide variety of software technologies.

The Enterprise Platform and the CIO at the age of the new normal

| Béatrice Rollet, Marlon Bromfield

Covid-19 pandemic has showed that the most digitalized companies, the digital-first companies, were the un-constable winner of this challenging period. Providing business activities through advanced digital experiences or platform offerings, these companies has kept their customers and partners engaged and happy in this challenging period.

AI led Software Engineering Use Cases: Application to Project Management activities

| Jérôme Perdriaud, Satish Srivastava

Using various AI techniques such as machine learning, deep learning, natural language processing (NLP), information visualization etc it is possible to guide the software engineering professionals with AI enabled decision making and automations. 

AI led Software Engineering

| Jérôme Perdriaud, Satish Srivastava

CIOs are expected to partner business, and at times leads, the delivery of digital transformation. The existing IT landscape of a company needs to be rationalized and modernized to be able to achieve the expected business velocity.

Conversational Assistants: go to scale

| Patrick Meyer

74% of French companies consider chatbots as a lever for digital transformation and more than a third have already deployed one. By 2020, 80% of them could use a chat assistant. A massive deployment that echoes consumer habits: 69% prefer the bot to a human exchange.

How can you use your IT assets to achieve digital transformation?

| Andre Bakland, Simon Herd, Béatrice Rollet

According to Gartner, for every dollar invested in digitalisation in 2020, three dollars will have to be invested in the modernisation of IT assets. Therefore, opting for the right evolution strategy becomes a crucial issue. Read more.

How Data Science can help in a pandemic situation?

| Marlon Cárdenas

With the aim of covering current and future needs of society, Data Science and Artificial Intelligence are seeking to drive the creation of technological solutions that benefit users in their daily lives. Many disciplines are uniting behind this cause, with health sciences to the fore, especially given the current context of the battle against the Covid-19 pandemic.

How holographic technology is helping doctors deliver better care

| Scott Leaman

Long gone are the days when holograms were the stuff of sci-fi movies and video games. Holographic technology is taking the medical world by storm, and by the looks of it, it’s here to stay. So how exactly is this technology helping doctors, and what are the major developments that we expect in the near future?

How will artificial intelligence transform industry?

| Maxime Claisse, Alexis Girin, Benoit Spolidor

Whilst there is no set definition of artificial intelligence as of yet, experts are in agreement that AI can simulate human cognitive capabilities such as perception, reasoning, action, and learning. AI now promises to completely transform the industrial sector – one of its primary applications.

International Paris Air Show: 5 trends to transform aeronautic

| Youssoupha Diop

The 53rd International Paris Air Show 2019 has confirmed the mounting fierce competition in the world of aeronautics. In this context, data, digital tools and artificial intelligence are now understood to be precious bargaining chips to accelerate transformation and turn these challenges into opportunities.

Anticipate cloud migration with FinOps

| Marlène Seif, Béatrice Rollet

Innovative and fast cloud services are crucial to digital transformation initiatives. Whilst there is no textbook model on how to adopt these services, it is nonetheless vital for companies to integrate them as fully optimised services in order to control their ROI.

From product to services: Flying the Aeronautics Industry into the Digital Future

| Philippe Armandon, Gaudérique Garrigue

With increasing travel demand and new competitors entering the market, aircraft manufacturers today are under considerable pressure.

How to control and optimise your cloud costs

| Didier Teixeira, Béatrice Rollet, Frédéric Janicot

Using public cloud services means rethinking your IT financial management. 

ASD S5000F: taking Aircraft MRO to new heights?

| Cyrille Greffe

In the 1990s, the combination of computer-aided design (CAD) and the concept of modular documentation gave rise to the first ASD standards (AeroSpace and Defence Industries Association of Europe).

Application replatforming: the Cloud migration booster

| Benjamin Chossat

Simple set-up, low cost and access to the horizontal elasticity of the Cloud: replatforming is often considered the best solution for porting a business application to the Cloud.

7 key strategies to transform applications with the Cloud

| Benjamin Chossat

How to modernise an application efficiently using the Cloud?

Innovating in pursuit of environmental sustainability

| Siva Niranjan

To attract new business, talent and investment, companies have had to demonstrate their environmental credentials more and more over the past years to wide range of stakeholders including institutional investors, regulators, clients, and employees.

Urban Air Mobility: will the future of mobility be in the air?

| David Elmalem, Sébastien Lautier

While the dream of the flying car has often been reserved for science fiction, a very practical and real future is gradually emerging for urban air mobility.

Guidance is the key for adapting DevOps to big business

| Gauthier Deschamps

DevOps is revolutionising agile transformation for big business. The method was initially focussed on software building but by automating production, it frees up resources so as to better resolve organisational and human malfunctions.

How Blockchain technology can improve Industry 4.0’s cybersecurity

| Alexandre Eich Gozzi

Earlier this year, the world’s largest container shipping company Maersk fell victim to a massive ransomware attack from the infamous NotPetya malware.