Sharing a vision of a virtual future

A Q&A with Holoeyes Inc. & Abdul Latif Jameel Health on their new strategic collaboration on medical VR technology.

In November 2021, Abdul Latif Jameel Health (through Abdul Latif Jameel General Trading Co., Japan) and Holoeyes Inc. announced an exciting new strategic collaboration for the distribution of Holoeyes’ innovative 3D virtual reality (VR) surgery support system and medical education software.

This new collaboration aims to contribute to the enhanced efficiency and safety of surgery, improving standards and the quality of medical education through advanced medical imaging technology.  It will focus on selected markets across the Middle East and Africa, as well as new markets in Japan.  We caught up with Naoji Taniguchi, 谷口 直嗣 Co-Founder and CEO/CTO, Maki Sugimoto, 杉本 真樹 Co-Founder COO and CMO, Holoeyes Inc. and Akram Bouchenaki, CEO Abdul Latif Jameel Health, to discuss the groundbreaking technology and the shared objectives behind the collaboration.

Where did the idea for Holoeyes come from?

NT:  I was working as a freelance software engineer.  I made some applications for robots and video games and did some research and development for computer graphics.  About seven years ago, a friend of mine, an editor from a big publishing company, asked me to make some digital health content.  I was doing some research for the project and I found an article that Maki [Sugimoto] had written about 3D graphics and VR in the medical sector.  He said some kind of VR sensor like Microsoft Kinect or super high-resolution video would revolutionize the medical field.  I was very impressed with the article, so I contacted him.  We met up in Tokyo and discussed his ideas in more depth.   We kept in touch and he taught me how to reconstruct a 3D model from CT scan images.

I made some scale models from the images and thought how great it would be if you could use a VR headset to dive into the skull.  I made a prototype, and it was amazing.  I think I was the first man to dive into a skull!  I showed the application to Maki and he was very interested.  He wanted to use it in the medical field.  He asked me to make an application using data from real patients and he started to use my application in the surgery for urology.  As a result, we won some awards in Japan which boosted our profile.  We realized the technology had massive commercial potential, so we set up Holoeyes Inc. to develop it further.

Can you explain, in simple terms, how the technology works and the key benefits it delivers?

MS: It’s basically like a GPS system for surgery.  The key term is ‘spatial awareness’.  Surgeons need to know as much as possible about an individual patient’s situation to know where to cut, where the surgical margin for this patient is, etc.  Current 2D monitors provide a limited amount of knowledge, but there is no spatial awareness as it is only a 2D image, like a CT or MRI scan.

Our system is completely different.  We take that patient’s real data – information about a particular location inside their body – and we can visualize the data in 3D in the air, like VR or AR.  Being able to ‘see’ inside their body in 3D greatly aids the surgeon’s spatial awareness of a patient’s individual anatomy and disease status.  What’s more, we do it over the web and any surgeon can learn to use the system in under 10 minutes.  So, within 10 minutes, they can turn their patient’s individual data into a VR model using our website. 

In surgery for cancer, as an example, using the Holoeyes technology, a surgeon can feed in the data and use Holoeyes to create a 3D image of the tumor.  They can look at it from behind, from on top, from below and see exactly what they’re dealing with before they make any incisions.  And crucially, they can actually do this during surgery.  Normally it would take 30-60 minutes to create a 3D image, which is too long to be of benefit during surgery.  Our system can reconstruct the 3D models during surgery in under 10 minutes in the VR system, which is fast enough to be used in the operating theatre.

What stage is the technology at?  Is it already being used in the real world or is it still developmental?

NT: Our technology is already being used in, but it is just the beginning.  VR headset technology is still very young and there is a lot more potential to explore.  For example, around voice recognition, so a surgeon can ask for examples of similar procedures.

We ran a dentistry demonstration this morning between Osaka and Tokyo.  An expert dentist in Osaka gave a demonstration to a young dentist in Tokyo via VR using the Holoeyes system and real patient data.  Next week they will do the same for implant surgery.

The big benefit is that it allows the junior dentists to see the actual movements of the others experienced surgical procedure in VR.

Holoeyes can calculate the location and the position of the arms and the devices and the patient data at the same time and share it in the VR space.  Using this ‘Metaverse’, or collective virtual shared space, we can share the movement, voices, and opinions and share the knowledge of how to perform and proceed.  Being able to see the movements in 3D is crucial for that.

What attracted you to the collaboration with Abdul Latif Jameel Health?

NT: The Middle East is a very attractive market and a similar size market to Japan.  It also has a strong strategic location as a mid-point between Asia and Africa or Europe, so it is an important part of our worldwide strategy.

AB: For me there are two key factors behind our collaboration. 

One is the technology and how we see it fitting into the new way medicine is being practiced and taught.  Children today are already used to learning on screens and comfortable with the idea of VR technology.  It is a trend we cannot escape, and it will affect medicine as much, if not more, than any other field. 

Holoeyes is ideally placed to be at the forefront of this technological transformation. 

The second part is how this technology fits with the mission of Abdul Latif Jameel Health.  One of our goals is to bring innovative technologies into parts of the world where they would take longer to get to or would never reach.  This collaboration fits very well with this concept. 

The Holoeyes system also has a smartphone app for viewing VR content in 3D.

This makes the technology even more accessible in less developed markets in Africa, for example.

All that’s needed is a smartphone and 5G.

Another application for Holoeyes’ technology is as a tool to improve and develop the knowledge of healthcare providers around the world, and in our regions in particular, by providing broader access to healthcare training.  This is crucial, because one of the issues facing many of our target markets is a shortage of healthcare providers. 

In countries in Africa, for example, you may have 2.3 healthcare providers per thousand inhabitants, whereas in Europe, Japan or North America, it is 10 or 20 times that number.  Technology can help us bridge that gap by training people remotely using systems like Holoeyes.

All you need is 5G connectivity.  Fortunately, 5G networks are developing rapidly across the globe, even in remote places, and it will become the norm in the next few years.  This is why we are so excited to be working with Holoeyes to expand the potential and the reach of its technology.

In Japan, one of the first sectors you’ve targeted is dental colleges and clinics.  Why did you decide to focus on dentistry?

NT: We started of looking at surgical applications, because Maki is a surgeon.

MS: There is also a point about data.  In Japan, more than 95% of dentists are in private hospitals and individual practices.  That’s one advantage.  The second point is that almost 60% or 70% of dentists in Japan have a CT scan machine in their practice, so they can obtain a CT scan of a patient easily and use the Holoeyes system to turn into a 3D VR model.  

Is Holoeyes developing this technology for more medical fields and applications?

MS: We are focusing on medical fields, but in theory anywhere that uses scanning technology like CT or MRI scans would be suitable for Holoeyes technology, like veterinary practices, for example.

It could also be used as a part of a wellness program, an annual health check or something like that, not only for illnesses and surgery.

How soon do you expect to be taking the Holoeyes technology to new markets in the Middle East and further afield?

AB: Very soon, I hope.  Our goal is by the end of this year, 2021, to complete the initial setups and start pushing out the technology through our networks.  We will start in the GCC and then quickly look for other markets.  We believe there will be a massive appetite for the technology because the benefits are so clear. 

It will also be a good complement for another technology we will be introducing, which is a handheld ultrasound device that connects to your smartphone.  We’ll be introducing it into teaching hospitals and universities first, so young physicians will get used to using ultrasound imaging plus 3D and VR technology to guide their diagnostics and procedures. 

In addition, we see it as an important tool for patient education and information.  Patients are increasingly at the center of the whole healthcare infrastructure, and we believe we can contribute to this important trend, providing new level of information and understanding to patients.

How do you see the relationship between Abdul Latif Jameel Health and Holoeyes developing?

AB: I’m very excited about our relationship and shared opportunities.  We’re building a strong foundation with the great team at Holoeyes and with our team in Japan.  We’ve been partners of Toyota Motors for 65 years, so we hope to be partners with Holoeyes for at least as long!

Do you expect this kind of technology to become the norm in future?

MS: This is the new normal.  The Metaverse.  Mark Zuckerberg, CEO of Facebook, says Facebook is a Metaverse company – everybody can share the same knowledge, technology, everything.  It’s the same in medicine.  Medical technology and data can be shared in the Metaverse in the same way.  Holoeyes is already showing that it can be done.  We are excited to see how it continues to develop in future.