How Augmented Reality (AR) is Revolutionizing Healthcare

Technology can accelerate the transformation of healthcare to improve health, quality of life, safety, and security. Augmented reality (AR) is finding its way into the healthcare ecosystem and there’s greater potential for it to be revolutionary, not just evolutionary.

In this episode of the Global Medical Device Podcast Jon Speer talks to Jennifer Esposito, Vice President and Managing Director of Magic Leap's Health business unit. Listen as Jon and Jennifer talk about the emergence of AR in healthcare, explaining what it is and is not, as well as how it compares and contrasts to virtual reality (VR), machine learning (ML), and artificial intelligence (AI).

LISTEN NOW:

Like this episode? Subscribe today on iTunes or Spotify.

Some highlights of this episode include:

• AR is a relatively new technology that some people don’t quite understand how it is or can be used in healthcare today and in the future. It’s important for people to try Magic Leap’s AR device and platform to expand their thinking of use cases.

• AR is not the same as VR and other technologies. VR is completely immersive and involves wearing a headset. You can’t see the real world. All you’re seeing is digital content. VR has advantages and limitations.

• Magic Leap’s AR headset device allows users to not only see the real world, but digital content can be overlayed on top of the real world while maintaining spatial awareness and sensibility.

• Magic Leap isn’t just about AR, but the convergence of other elements and capabilities, such as AI, 5G, and Internet of Things (IoT). AR is going to be the catalyst that generates additional new sources of data not currently captured.

• AR is not replacing healthcare providers, doctors, nurses, and others. It augments the clinical workforce to present/provide insights that advance and enhance the practice and expertise of medicine when delivering care.

Links:

Jennifer Esposito on LinkedIn

Magic Leap: Augmented reality platform for Enterprise

Medical Device Innovation Consortium (MDIC)

FDA - Medical Devices

Greenlight Guru Academy

The Greenlight Guru True Quality Virtual Summit

MedTech True Quality Stories Podcast

Greenlight Guru YouTube Channel

Greenlight Guru

Memorable quotes from jennifer esposito:

“There are places where the differences between those technologies is actually really important in terms of deciding which one you would use for a certain use case.”

“What we’re doing isn’t really just about AR. It is this convergence of these other capabilities.”

“Augmented reality is also going to be the catalyst to generate additional new sources of data that we don’t currently have.”

Transcription

Announcer: Welcome to the Global Medical Device Podcast, where today's brightest minds in the medical device industry go to get their most useful and actionable insider knowledge direct from some of the world's leading medical device experts in companies.

Jon Speer: On this episode of the Global Medical Device Podcast, I had a chance to speak with Jennifer Esposito. Jennifer leads the Health business unit at Magic Leap, you can learn more about Magic Leap by visiting www.magicleap.com, but basically what Magic Leap has developed is a platform technology in the area of augmented reality or AR. And during my conversation with Jennifer, we talk a little bit about what AR is a little bit about what it's not, how it compares and contrasts to things like VR or virtual reality, AI and things along those lines. But it's really cool. I don't get a chance to talk nerdy about tech all that often. Sometimes it's a lot about regulatory, which is fine too, but definitely a little bit of a change of pace, but I hope you enjoy this conversation on the Global Medical Device Podcast. Hello, and welcome to the Global Medical Device Podcast. This is your host and founder at Greenland Guru, Jon Speer, and joining me today is Jennifer Esposito. And Jennifer is the lead Health... Well, Jennifer, what is it that you do with Magic Leap? I've jotted some notes and I'm like, oh, that doesn't make any sense, but I believe you lead the Health business.

Jennifer Esposito: Yeah, that's right. I'm the Vice President and General Manager of the Health business at Magic Leap.

Jon Speer: All right. Well, that might be a great place to start. Tell folks a little bit about who you are and while you're at it, tell folks a little bit about Magic Leap and then we'll dive into a topic of augmented reality and what all that means.

Jennifer Esposito: Yeah. That's how sounds great. I've been in this intersection between health and technology for my whole career. My graduate training is in actually biostatistics and epidemiology. My first job in the real world was at GE as a Six Sigma Black Belt. I did a lot of work on data analytics and one of the things that I did while I was there was help launch basically a new information system that leveraged industrial IOT information from our medical equipment in the field to help us deliver proactive service maintenance. I ran one of the service maintenance orgs. So we had a lot of fun doing things like that. And then I spent a little bit of time at Intel leading the health and life sciences group there. And that was the place where I also kind of got my first taste of augmented reality and understanding what that technology was, because we were always trying to work on the bleeding edge of things. I did a lot of work on AI, really understanding the use cases for that technology in the health industry and really figuring out the best way to advance it or accelerate it. And I've been here at Magic Leap for almost three years now, and it's been really fun because this is a new technology that it's still a little early for people in terms of understanding what it's all about. So my team and I are spending a lot of time really focusing in on or what are those new use cases that you can deliver with the technology and helping people understand what that means just from a roadmap perspective what's possible today and what's possible sure two years from now, five years from now, whatever it is.

Jon Speer: Yeah. And what I'm about to say next, I mean this as a compliment. Man, you are a nerd aren't you? I'm a nerd too, just a different kind of nerd. I'm a med device quality nerd. But yeah, nerds rule the world.

Jennifer Esposito: We can identify each other, even though...

Jon Speer: Yeah. And I have colleagues all the time and we hire, well, a lot of software developers and other medical device people. And when you talk to somebody that's passionate and cares about what they do, they do get down and nerdy with it. And you know, it's like, wow. I appreciate that this person is really passionate about what they do. And I could tell that from the first moment that I talked with you that wow. Jennifer. She really enjoys... She's very passion about, in this case, augmented reality. Folks, I want to remind you, there's a website to direct you to so you can learn a lot more about Magic Leap and their products and their technologies. It's pretty simple. Magicleap. com. That's where you go. But honestly, I'm still trying to wrap my head around what augmented reality is. Maybe what it is not. It's the same as, or different from things like machine learning and AI. And I hope you can help illuminate that for me and the listeners.

Jennifer Esposito: That's great because it's funny when I talk to people, I sometimes hear people say things like," Oh, AR, VR, XR it's all the same thing." And it's not. There are places where the differences between those technologies is actually really important in terms of deciding which one you would use for a certain use case. So I usually start off by telling people," Okay, VR. VR is completely immersive. You're putting on some sort of headset-"

Jon Speer: And not to cut you off VR, virtual reality.

Jennifer Esposito: Oh yeah, acronyms. Yes. Virtual reality. So when you put on a virtual reality headset, usually it's a headset. It's like some sort of goggles or glasses, whatever people might want to use as a term, it's completely immersive. You can't see the real world. All you're seeing is digital content. So that has limitations. It has advantages and it has limitations. What Magic Leap does is what we call augmented reality. And there's multiple definitions too, even of augmented reality. Some people might think about it in the context of like what you can do with your phone, where you kind of hold your phone in a Pokemon GO kind of way and you can look at something that's more like a sticker on top of the real world. In that case, the digital content is just there. It doesn't really know what the real world is. It's not officially aware of the real world. In our case, you put on our headset, our device, you can see the real world and you can overlay digital content on top of the real world and it can maintain awareness of where you put it. It's spatially aware. It has a sensibility about it. So that presents a lot of different fundamental advantages or use cases, comparatively speaking to the other technologies that I talked about. One of the reasons I think you probably sensed my excitement is because I do believe there are so many use cases for this technology. And I think sometimes when we talk about this in the industry, in the ecosystem, because it is new people don't really fully have the scope or the scale of the potential in their brains yet they may not even be able to necessarily even imagine what the technology can do right now if they haven't ever tried it. So for us, it's very important that we get people to put on our technology, put on our platform and try the use cases so that they can see what it really means and how it can change things. And how much of it in reality is actually really possible today. So that you can expand your thinking of the use cases beyond maybe the obvious or the easy ones. Because people always bring up things like training and education. Those are great use cases. I'm not suggesting that they're not, but there's actually a lot more that can be done even today with the technology that we have.

Jon Speer: Yeah. The training use cases, it's not a leap. It's a small evolution from what is being done today and if I'm hearing you correctly, there's way more potential for augmented reality to be revolutionary, not just evolutionary.

Jennifer Esposito: Yes.

Jon Speer: We just got to think a little bit bigger. To me, I can imagine there's a lot of confusion to your point. In your world there's a very big distinction between VR and AR, but to the layperson, there may not be a distinction. So I think hopefully today listeners will leave with couple salient points that will say, oh, this is that and as this, and they're drastically different, but in the world of med device, one of the interesting things in probably the past, let's go three to five years, there's been a lot more activity on AI, specifically medical devices that incorporate some level of artificial intelligence. This is another one of those confusion points. Usually, not always, but usually when companies identify their tech as having AI, usually it's machine learner. There is a Venn diagram that relates these things. So I think there's more of an appetite for nontraditional tech in med device. So I'm a little interested to hear what you've seen with the Magic Leap technology and how augmented reality is finding a place, or places, in the healthcare ecosystem.

Jennifer Esposito: I think you bring up a really good point, because things like AI have been... Those were the buzzwords. Have been the buzzwords maybe for the last three to five years. I know when I was at Intel three years ago that was a big area of focus. What I think is interesting when you think about that in the context of, for example, augmented reality, there's a very big element of computer vision that's associated with just the underlying technology and the kinds of use cases that you would approach. Actually one of the things that was intriguing to me in the early days when I was coming to Magic Leap was this fact that you have this convergence right now, right between AI and the augmented reality technologies and other things too. Like IOT and certainly things like 5g. So those things coming together now are even more reason to understand that these use cases are more here and now than we might want to think. That's what's I think exciting because what we're doing isn't really just about AR. It is this convergence of these other capabilities. And certainly the other thing that is very interesting is that augmented reality is also going to be the catalyst to generate additional new sources of data that we don't currently have. So if you think about understanding physical spaces in the context of what people are doing in them, whether it's a surgeon or even a patient, in some sort of therapeutic environment, that's really interesting information that we can't really capture today in our known technology elements that I think is going to go open us up also to different use cases, different problems that we can solve with the technology. And that's the other thing that really excites me, because in a lot of ways, and this is true in a lot of the conversations that I've been having, people, once they truly understand the technology it all comes back to the data at the end of the day, because that's where people... Their eyes light up, because that's where the new use cases, the new businesses are going to come from. So that's the other thing not to forget. It's in some ways we kind of tend to fixate very specifically on a technology as opposed to the big picture of all of the things coming together and what's new about what you're able to do.

Jon Speer: Yeah. One of the things I'm hearing from you is without AR, in some respects, we don't know what we don't know, but AR is exposing or uncovering new information or data that we can convert into information and translate into knowledge that we can do something about. One of the things I remember, gosh, I don't remember how many years ago this was, let's call it a decade ago, which is probably about right. I remember the first time I heard about a surgical robot and I'm like," All right. What?" So the first image that popped into my head was there's a robot in the OR and there was a surgeon that was operating it and then is like, no, that surgeon could be completely across the country. I just remember like mind blown at that point in time. That is crazy talk because to even think that that was happening it just seems so science fiction. It seemed like something you'd see on Star Wars or on Star Trek but no, it's reality.

Jennifer Esposito: It's real.

Jon Speer: And then there's some drawbacks with that, because those are complicated pieces of equipment and as you and I talked a couple weeks ago and I did a little bit of digging and research, I'm like, oh wow. It seems like AR and maybe some of the things that Magic Leap was doing, man, that can improve other technologies that are already out there, like surgical robots. Am I thinking about it in the right way?

Jennifer Esposito: No, no. You're absolutely on the right track, because this is actually goes back to what you were saying right before that, which is you have this new data. It gives you new insights. I like to use the word insight because I think it's more actionable. And it's a lot of times in healthcare today we have a bunch of digital data that is just digital and it's really not being-

Jon Speer: It's data.

Jennifer Esposito: ...Taking advantage of. Or it's being used completely for administrative reasons. But the way I think about it is, and it's really exactly what you're saying, this new data coupled with the old and existing data that we have, it becomes a closed loop, because then you're continuously capturing that data and then it can be used by everybody else who continues to use the technology in the future. Whether it's the person running the robot or even the surgeon just operating as a surgeon with regular instrumentation. That data can then be brought back to the clinical workforce through augmented reality, presented to them in a digital way that they can see during whatever it is they're doing. If they're performing a surgery or some other clinical task. So that's the other thing that I think is pretty cool about augmented reality is not just that you're getting new and unique types of data, but then it becomes the mechanism by which you can deliver the insights. And actually it's about augmenting the workforce because a lot of people, once you have these conversations, whether it's robots or AI, people are like," Well, you're going to replace the surgeon?" No, that's not what we're talking about. What we're talking about augmenting the clinical workforce by giving them new insights about what they're doing that is really created by virtue of the fact that we're using this in a normal way of operations. That's why I said closed loop. You constantly updating that information so it benefits the next person who's doing this the surgery, or doing whatever it might be.

Jon Speer: To me, the practice of medicine has always been a little bit fascinating. Just what healthcare providers, doctors, nurses, what those folks are able to do. It's just like, wow. That's amazing. I remember early on in my career as a product development engineer for a medical device company, I had the opportunity to see a lot of surgical procedures and different medical procedures in the OR. Seeing knee replacement, hip replace the open heart procedure. It's just fascinating, but so much of what physicians learn came from tribal knowledge. That's crazy to think about how does that perpetuate the generation after generation. And you know, as I think about AR, I was like, oh wow. This could reduce the reliance of tribal knowledge and maybe even advance the practice of medicine.

Jennifer Esposito: Yeah. Exactly, because it's tribal knowledge or art versus science. It's the same line of thinking. And the reality is it's because we can't necessarily... We might be able to explain it like one- to- one, but it's because we really can't objectively articulate why one thing is actually better than another. Or why did you have a good outcome in this surgery? The surgeon can tell you what they did, but that may actually not really be why. To the extent that you can either quantify that or build that database that stacks everything up over time that says," Yes, this is why." Then you can have these scenarios where maybe somebody that's not as experienced or is more of a generalist with the augmentation, can actually perform at a very similar level to somebody that is, for whatever reason, considered top of their game. So I think that opens up a lot of interesting possibilities. Even just in the broader sense of expanding access to care in markets and environments where you just don't have the people. You just don't have the experts or the support that is needed. This is something that can really expand that expertise wherever it needs to be.

Jon Speer: For sure.

Jennifer Esposito: There's really no geographic boundary anymore.

Jon Speer: Well, you hear often about how we have a shortage of nursing folks and shortage of physicians and I think that issue is probably even more exacerbated in certain parts of the world too. I think that's really interesting. And if I'm understanding what Magic Leap is, I'm sure you have some products that have a very targeted audience and market and things of that nature, but what I'm kind of picking up is this is really just more of a platform technology that you want to get in the hands of healthcare providers and kind of let them figure out opportunities where this could enhance their practice. I might have connected dots that weren't supposed to be, but how do you see or what do you want people to take home about Magic Leap and your technology?

Jennifer Esposito: I think you kind of nailed it. We really are a platform. We're a hardware and a software platform. My team and I focus on trying to... We have the general platform that's available to everyone and then we're also building a little bit of a layer there that facilitate some of the things that are more specific to the health industry, but really what we as an organization are trying to do is to bring that platform to the potential partners in the ecosystem that would then bring their own products and solutions to market that are built on in the technology. Because we really think that there are obviously lots of incumbents in this space, whether you're looking at big medical device manufacturers or an ecosystem of small to medium size startups that are playing in a lot of the digital health, digital diagnostic space that can adopt our platform and leverage it and then bring it to market. And that's what our team is really all about is enabling that ecosystem of potential partners to take advantage of our technology so that there is a very wide portfolio of solutions that run on Magic Leap in the health space. And like I mentioned, that runs a range all the way from very high- end use cases for things like surgery. I call them sometimes extreme- use cases, because I think they push the edges of the technology. And then this really interesting space that we've seen a lot of traction in already, which is what I call digital therapeutics and digital diagnostics where our platform is the foundation of some sort of replacement for some sort of existing diagnostic test. So we've got a really lot of interesting things going on in that space with partners that are doing very novel things. So that's why I think the way you characterized where we fit in the whole story there is right on.

Jon Speer: Yeah. I have, I guess, a couple more thoughts before we wrap up our conversation today. The first thought... And I think they kind of build on each other. One of the things that I think is really exciting about the healthcare sector and again, more close to my home, is the medical device industry. There's definitely a convergence of tech. The lines are very blurry about what is and is not a medical device these days. And you've got things like Apple Watches and things that people are wearing on their wrists and apps that might be on their phone and whatnot that might be now part of my everyday life that are technically under that umbrella as a medical device. And in some respects that could be scary, but as a consumer it's kind of exciting at the same time that it's blurry. So just seeing that convergence of how all these things are starting to just morph into one- and- the- same is kind of cool. And to learn a little bit more about Magic Leap, I just see that yet as another cool addition to the space.

Jennifer Esposito: No, absolutely. I think you're absolutely right. On the regulatory front that in some ways it's uncharted territory, but I think that the ecosystem is doing all of the right things to explore how to tackle these issues. Certainly, you have the examples that you gave of things that already exist in the world that have a secondary purpose or can be leveraged. And then there's some of the stuff that I was talking about, which is much more specific and we don't necessarily have an equivalent way to access it. So the way we currently do it, I'm on a subgroup with MDIC where we're looking at things, like how do you really define image quality, and things like that. Display quality in the context of augmented reality. So we're looking at what exists today in the 2D world that we can use as foundation or at least a starting point. And also, to point out, what you do in 2D isn't necessarily easily measured in the same way in 3D and you might have to have some different frameworks. So I think there's a lot of really great work going on to talk about this topic and to get the right standards and discussions going early, as opposed to later. And really, I think what I like seeing is that everybody's involved. You have a mix of people who are in academics or in clinical work and you have a lot of the hardware and the software manufacturers also engaging to come together at this early point to find the right path forward with the FDA, with MDIC, with whatever these organizations might be.

Jon Speer: Yeah. And I think the regulatory aspect of it does complicate things. The historical regulatory framework complicate things. But to your point, I see FDA I think is working really, really hard to being a little bit more flexible with tech that's in AI and AR and that sort thing. So yeah, that's encouraging, because the rules from the old... From years ago or from more traditional technology or medical devices that doesn't necessarily fit the model today. I'll use another example from my world, that there's a set of regulations for software known as Part 11.

Jennifer Esposito: Oh yeah.

Jon Speer: But those were written in 1996. It's 2021. That doesn't fit today and that framework hasn't been updated. Now I know FDA's got it on their radar screen, but I think in this space of AI, machine learning, AR, I do applaud FDA's efforts to be more cognizant of, or aware of, or open to enabling the technology rather than forcing it in a square peg through, around hole.

Jennifer Esposito: Exactly. I think it's clear that they see the value and I think, on the AI front, was kind of that early precursor work that needed to be done to say," Hey software as a medical device", or" How do you kind of have these preapprovals so that you can move more quickly as new algorithms come through?" So I think we're lucky that some of that work was done over the last five or six years coming through with AI. This, I think just changing the old paradigm in general.

Jon Speer: For sure. And then the last thought that's on my mind, it's kind of piggybacking on the time a moment ago about everything converging from a tech standpoint. I think with that... I mean there have been home use medical devices for a long, long time. And generally speaking, when a device is a home- use type product, it does complicate things a bit because you have maybe a person using it that's not clinically trained user. Just might be you or me. And we think it should operate like our cell phone, but this product has completely different functions or buttons or whatever the case may be. And then it's also in an environment that's not controlled. I probably don't keep the temperature of my house, maybe the same as what you keep yours, but if it were in more of a hospital or an OR setting we have more controls in place. So anytime you get into that home- use, it does sometimes, or oftentimes, complicate usability. And as I've been thinking today, it's like, oh wow. Companies that are developing products where, and I think almost every device has usability is important for almost every medical device, but I almost see a role of AR could be leveraged in some way as part of usability studies. Every company that does usability work, they're gathering data and information about the use of their product. It seems like if there was a way to incorporate AR as part of that process that we would learn a lot more about our product and improve upon that.

Jennifer Esposito: Any place where you kind of want to better understand how things are happening, especially how people are moving around in an environment. I think you're going to... You're going to be able to leverage a lot of that from AR and that's one of the things we are working on is one of the features that we see a lot of our potential digital diagnostic partners leveraging is eye tracking. And there's a lot you can do with that. Everything from vision and ophthalmology, but you know, just things related to brain health and understanding where people are actually looking. So there's a lot of really interesting information that can be gleaned from that. And then the other area is movement, because we're able to tell things like gait and balance and things like that and put that together. So we're seeing some of these clinical use cases that are attached to some sort of clinical diagnostic or therapeutic, also being very applicable in a much more generic setting that isn't clinical. How does a worker move? And how do you understand sports performance in an athlete? So these things are really interesting because I think they have a much more broad application potential coming out of the deep work that gets done on the clinical side.

Jon Speer: Yeah. It's cool. Well, I've enjoyed our conversation, but before we wrap the discussion up today, is there anything else that you think is really important that something that you want to share with listeners either about AR or Magic Leap or anything of that nature?

Jennifer Esposito: What I said at the beginning. I think it's really important for people to recognize that a lot of these use cases that we've all sort of heard about are actually very much possible in the short term. So it's important that we get that context out there. There'll be more to come from our side to show people some of the things that our partners have been doing with us over the last year or so. And I think that's really going to give people a good sense of what's possible today and what are we going to do over the next two to five years as we move forward with the technology. But I'm really excited about the technology and the use cases and I think in healthcare and health in particular, we have a lot of opportunity to tackle some of the bigger challenges that are out there in a way that we haven't before.

Jon Speer: Yeah, absolutely. Jennifer, I appreciate it. Thank you so much. I've enjoyed our conversation today. Folks, Jennifer Esposito. She leads the Health business unit at Magic Leap. You can learn more about Magic Leap. It's very simple. Go to www.magicleap.com. All one word. Well while we're wrapping things up today, I want to remind folks about Greenlight Guru. Greenlight Guru is the only medical device success platform design exclusively at only for medical device industry by actual medical device professionals. Yes, I know I just said medical device a whole bunch of times, but within the Greenlight Guru software platform, we have workflows to help, you manage your design and development activities, risk management documents and records, including electronic review and approval and revision control. We have workflows that help you manage your CAPAs, your quality, other quality events, complaints and nonconformances. And it's all in a single platform. A single source of truth. We're helping medical device companies all over the world get products market, obtain ISO certification, survive, and actually thrive during audits and inspections as well. So if you'd like to learn more about the success platform from Greenlight Guru it's very simple. Go to www.greenlight.guru to read all about all of our products and services. And if you'd like to have a conversation we'd love to chat you and see if our products and services can meet your needs and your requirements. Again, that's www.greenlight.guru. And always thank you for listening to the Global Medical Device Podcast. It's because of you that this is the number one podcast in the medical device industry. So continue to spread the word, share this with your friends and colleagues and looking forward to the next conversation that we get to share with you on the podcast. Until then this is your host and founder at Greenlight Guru, Jon Speer. And you have been listening to the Global Medical Device Podcast.

ABOUT THE GLOBAL MEDICAL DEVICE PODCAST:

The Global Medical Device Podcast powered by Greenlight Guru is where today's brightest minds in the medical device industry go to get their most useful and actionable insider knowledge, direct from some of the world's leading medical device experts and companies.

Like this episode? Subscribe today on iTunes or Spotify.