HoloLens goes to work—but easier developer tools are needed

Several decades into the desktop PC revolution, there are more than 2 billion workers around the world who don’t have a desk. These field workers maintain much of the infrastructure that keeps our world working. Over the years, we’ve given them laptops, tablets, and smartphones, but they’re still missing much of the information support they need.

That’s because even with touch screens and pen input, computers are designed to be used by people who can give them their entire attention—and both hands. If you need to use your hands to adjust the mechanisms in a fuel pump, an aircraft engine, or a lift motor, there’s no way you can use a computer or tablet to access manuals or ask for help. Maybe you can work one-handed with a smartphone, but even that is iffy for many jobs.

What if these workers had a hands-free way of accessing that information? That’s what Microsoft wants to remake its Windows 10-based HoloLens visor technology and associated augmented reality technology for: providing computer power to workers who can’t pick up a computer. That’s a more refined mission for the 18-month-old HoloLens, which Microsoft originally envisaged used for everything from immersive gaming to computer-assisted field work to 3D modeling to “fly through”-style exploration of anything from the human body to the Milky Way galaxy.

Bringing computing to field work is hardly a new problem. In fact, it’s one the tech industry has tried to solve many times. Even I’ve had a go at it, as back in the early 1990s, while working at a UK telecoms research lab, I was part of a team that was bidding for Fourth Framework EU research funding. Working with an Apple research lab, a semiconductor company, and a European military aircraft manufacturer, we were planning to develop a handheld device to support engineers on the flight line. Our proposed hardware would have used a secure wireless network to deliver maintenance manuals, giving aircraft engineers access to the information they needed, when they needed it.

HoloLens goes to work with new certifications, add-ons, and software

Devices like HoloLens make a lot of sense as field worker tools. They’re full-fledged computers that don’t need hands, using gaze and voice to handle interactions. You can take a HoloLens anywhere, and if you’ve got the right software and data installed, you don’t even need network connectivity. With network connectivity, you’ve got access to the whole internet, as well as a connection back to the office and to expert colleagues.

Microsoft recently announced new certifications for the HoloLens hardware that should make it easier to sell into field markets, allowing it to be used as eye protection. A 2018 hardware add-on will also let you use HoloLens with a hard hat, so engineers and architects can use it on site where head protection is needed.

Also coming in 2018 is a HoloLens version of the Skype for Business platform (including Microsoft Teams), for conferencing and messaging. Microsoft promises that it’ll be easy for you to quickly annotate the environment around a HoloLens, overlaying it on what you’re actually looking at.

If this new vision for HoloLens is going to be a success, it’s going to need more than Skype and Teams support. Although they’re a useful set of tools for connecting a worker with colleagues, they’re not at heart a way of delivering immersive, 3D information. That’s going to require new authoring tools that work as easily as PowerPoint, providing tutorials and content that overlay the real world, without requiring programming skills and learning how to work with 3D platforms like Unity.

3D authoring for augmented reality

So, it was interesting to talk to a team from industrial software maker PTC about the Thingworx Studio augmented reality authoring platform. It can take models from CAD applications, then provide a framework to quickly assemble 3D environments that can be anchored to physical markers. Models can be tied to animations, or to data from sensors.

In a demo, I used a HoloLens to explore an exploded view of a piece of industrial equipment, seeing how the various components fitted together, using HoloLens gestures to move from view to view. The same demo then walked me through replacing a component, overlaying accurate CAD images on the actual equipment, and showing me how to open it up and remove a failed part.

The application was pretty much standard for a HoloLens demo. But what got my attention was that the demo application wasn’t a custom piece of code, like so many others. Instead, it was in the Thingworx View runtime and had been created in on a PC without a line of code, using Thingworx Studio. Thingworx Studio looks and behaves like most modern CAD software, bringing in models and letting you build different views, adding PowerPoint-like transitions and basic scripts.

Tools like Thingworx are key to delivering on Microsoft’s field-worker strategy for augmented reality hardware like those based on HoloLens. Not every business can afford to build custom software for all their use cases, so having access to 3D authoring tools and runtimes is essential.

Such tools, along with a new generation of lower-cost hardware, should help get augmented reality into the hands of at least some of those 2 billion workers.