Jonmichael Moy: Meet the company making haptics so you can experience the feeling of cutting flesh

Haptic feedback helps you to know when you’ve tapped a key on your phone. With a reassuring little pulse, it gives you the feeling of firing a round in Call of Duty with a short, sharp jolt, and it lets you experience the weight and pressure of lifting and cutting stomach tissue.

We’re huge fans of applications of haptic feedback, giving you an extra sensory dimension in virtual reality experiences, so when we heard that there was a surgical VR experience at the Global Education and Skills Forum (GESF) that we were attending in Dubai, it was with a sense of palpable excitement that we headed to the Fundamental VR concession.

Primarily a surgical training tool, Fundamental VR has managed to create the most visceral haptic feedback we’ve ever experienced. Once you’re strapped into a VR headset, you are handed control wands that correlate to mechanical arms that appear in front of you on screen.

For the experience that we tried, the wands controlled a set of pincers and a cauterizing tool, but can be a scalpel, a bone chisel, a syringe, pretty much any tool that a surgeon will need to use during an operation.

The experience is device agnostic (we used a Windows Mixed Reality Headset) but Fundamental had an Oculus Rift and HTC Vive with it too and was happy to swap out to whichever device we wanted to try.

The experience itself is a very short, simple simulation. You’re put in the shoes of a surgeon, stood above a sheet-covered virtual patient, with a screen above an exposed fleshy belly with tools sticking out of it.

The screen shows the feed from the camera inserted into the torso, which is a view of the inside of a human body, just on the outside of the stomach.

Prepare yourself

You are tasked with grabbing and lifting the stomach lining, then cutting away a thin layer using the cauterizing tool by sliding it into a small hole and using heat to separate the section of the stomach lining.

It took us a couple of seconds to get the depth perception needed to grab the lining with our pincers, but when we did, the strangest thing happened and we felt the weight of the lining. It felt real. It didn’t feel like it was real, it felt real.

We’d never experienced anything like it, and we were halfway between awed and horrified. We had to lift the lining up to a point where it locked into position, and because we don’t have the deft touch of a surgeon we bungled it. The physical ‘pop’ of the lining slipping from our pincers genuinely made us gag.

Eventually, we got the lining in the right place, and a hole opened up for us to slip the cauterizing tool in. Yep. That happened. Fighting back some pretty intense nausea we cut the lining using the heat of the cauterizing tool and the session (thankfully) ended.

Talking with the maker

After taking a moment to compose ourself, we had a chat with Fundamental VR’s co-founder and CEO Richard Vincent to ask how it made such effective haptics, and what it plans to do with the technology.

“Surgery is about touch, and it’s about sight, with only one of them, it’s a game,” Richard said, “Building the haptic intelligence that allows us to map all the tools and tissues, and all the possible collisions between them took us about a year, and we’re now at a point where we can reliably deploy pretty much any tissue experience.”

And getting to that point took a lot of trial and error: “That’s a process of getting surgeons in. We’ve got a calibration tool that allows us to go ‘This is the point where you were inserting this into that location, did it feel right?’ and then we’ve got about seven different variables to adjust like how squishy it is, how much you have to push on it before it finally gives way or penetrates.”

And there was one real life doctor there, Dimitri Amiras, a consultant radiologist from the UK-based Imperial College NHS Trust. We took the opportunity to ask him whether the ‘gamification’ of surgery would reduce the emotional risk associated with a real life operation:

“The gamification will take the surgeon a step back from what he’s actually doing, and actually allow them to focus on the operation itself. If the surgeon’s thinking about the person’s kids waiting at home there’s no way they can do the operation. You can have empathy afterwards, but at that time you need to focus on the procedure.”

For an experience of a real life brain surgery in 360-degree video, check out this video made by Fundamental:

While the simulation experience we tried felt very real, it still looked very game-like, and we wondered how many of Richard’s team came from the game industry:

“The team’s made up of a lot of people from the gaming background because that’s where the skills in VR performance have really been refined over the last two years. The best place to find Unity or Unreal programmers with really good experience is gaming.”

At the moment the tool is being used to train would-be surgeons, but we can see obvious applications in consumer VR, and apparently we’re not the only ones. Richard told us he has been approached by a number of different industries for his algorithm. For the time being he’s keeping the focus purely surgical.