LifeinTECH

Over the past few weeks, I have been testing the Mirosoft HoloLens 2.

Released in November 2019, the HoloLens 2 is a mixed reality (MR) headset, which aims to produce environments and visualisations, where physical and digital objects co-exist and interact in real-time.

The mixed reality (MR) terminology can be confusing, especially when discussed alongside virtual reality (VR) and augmented reality (AR). The descriptions below outline the key differences.

• Virtual reality (VR) fully immerses the user in an artificial digital environment.
• Augmented reality (AR) overlays virtual objects on the real world.
• Mixed reality (MR) goes beyond augmented reality, anchoring virtual objects on the real world.

As highlighted in the articles “Virtual Reality” and “Oculus Quest”, computer-generated simulations have matured significantly over the past five years. We have not yet achieved the fidelity of the Star Trek Holodeck or Star War Holograms, but these science fiction concepts no longer feel unachievable.

When the HoloLens was first introduced in 2016, Microsoft failed to position the product effectively, setting a false expectation that it would be made available as a consumer product, similar to Virtual Reality Headsets (e.g. Oculus and Kinect.

Thankfully, Microsoft has rectified this confusion with HoloLens 2, clearly positioning the product for enterprise businesses, supporting use cases such as remote support, 3D modelling, training, hands-free procedures, etc.

As an example, the video below highlights HoloLens 2 being used alongside Dynamics 365 Remote Assist for hardware remote support.

Over recent months, the applicability of these use cases has been amplified, recognising the global restrictions being applied due to the COVID-19 pandemic.

In this article, I plan to share a brief insight into my testing (results to following in a future article), as well as my initial impressions of the HoloLens 2 hardware and software.

Testing

I have been partnering with World Wide Technology (WWT) to test two use cases, which have been documented in the article “Virtual Migration Strategy with Microsoft HoloLens 2”. A summary of the use cases can be found below.

Deployments and Site Migrations

• Leverage the HoloLens 2 to assist physical hardware deployments and site migrations, specifically applicable to R&D and Manufacturing sites.
• Instead of having a large number of expert resources on-site, a smaller number of local resources (known as “smart hands”) leverage HoloLens 2 to execute the required processes.
• Local resources use the HoloLens 2 and Dynamics 365 Guides, which delivers a “digital playbook”, providing a fully annotated step-by-step guide.
• Expert resources provide remote consultancy and support via Dynamics 365 Remote Assist, including real-time voice, video and annotation.

Regulated Testing and Validation

• As part of a regulated industry, the qualification process includes visual validation of the IQ (installation qualification) and PQ (performance qualification) testing.
• Evidence must be produced and reviewed by a quality assurance resource, confirming the system configuration/installation has been completed in a controlled fashion.
• Local resources use the HoloLens 2 and Dynamics 365 Remote Assist to connect with remote quality assurance resources.
• The HoloLens 2 cameras could be used to record all testing, including human interactions with annotations. The recording can be retained as documented evidence of the testing process.

The hope is that the HoloLens 2 and associated services can help improve the quality and efficiency of these use cases, reducing the need for on-site resources (travel and costs), whilst improving speed to value and providing a greater level of agility via real-time collaboration.

Hardware

At first glance, the compute, storage and network capabilities of the HoloLens 2 are fairly underwhelming, essentially equivalent to a mid-range smartphone.

• SoC: Qualcomm Snapdragon 850
• Memory: 4GB LPDDR4x System DRAM
• Storage: 64GB UFS 2.1
• Connectivity: Wi-Fi 5 (802.11ac 2x2), Bluetooth 5, USB Type-C

The Qualcomm Snapdragon 850 Mobile Compute Platform is an ARM-based SoC designed for Windows laptops. It is essentially a higher clocked version (3.0Ghz vs. 2.8GHz) of the popular Snapdragon 845 Mobile Platform, manufactured using 10nm LPP FinFET at TSMC.

Thankfully, the HoloLens 2 is more than just a smartphone that you mount on your head. It also includes a unique display, a wide range of sensors, as well as a custom multiprocessor known as the Holographic Processing Unit (HPU).

It is the HPU that makes the HoloLens 2 unique, removing the need for a cumbersome breakout box and cables, which would severely limit real-world adoption.

The HPU is responsible for processing the information coming from the sensors, including the custom time-of-flight depth sensor, head-tracking cameras, inertial measurement unit (IMU) and infrared camera.

HoloLens 2

As previously stated, the HoloLens 2 is not targetted at the consumer market. However, the packaging is certainly premium quality, that would not look out of place being sold at an Apple Store. This care and attention regarding the “unboxing experience” is unusual for an enterprise product, likely highlighting that the HoloLens 2 is still highly specialist (limited audience).

Inside the box, the HoloLens 2 includes a high-quality travel case (very similar to the optional Oculus Quest Travel Case), which stores and protects the headset itself, as well as the USB-C power adapter.

The HoloLens 2 headset is very well built, with premium materials and tight tolerances. The product feels rigid and dense, which promotes the quality, but also practical considering the target market.

In the Virtual Reality world, the “halo” head strap (found on the PSVR and Oculus Rift S) is considered the most comfortable. However, Microsoft has chosen to implement a traditional velcro headband, with the addition of an “adjustment wheel” at the rear, which can be used to quickly and easily tighten/loosen the headband itself.

This “hybrid” design is very comfortable and should be suitable for almost any head size (including my five-year-old son, as shown in a later photo).

At 566g, the HoloLens 2 weighs about the same as the Oculus Quest (571g), but the weight is more evenly distributed, with the battery being located at the back of the headset. This delivers a very balanced product, making it more viable for prolonged usage.

Wearing the HoloLens 2 is surprisingly comfortable, although not subtle (you cannot hide the fact you are wearing the headset). Therefore, it is likely a “point in time” device, only worn when completing a specific task.

When viewed from the front, The HoloLens 2 looks a little like wearing a large bicycle helmet and cycling sunglasses.

Thanks to the open top and sides, the HoloLens 2 does not feel constrained or claustrophobic, which can be a common criticism of Virtual Reality headsets.

Although the lenses are transparent, the embedded display can obscure vision. As a result, Microsoft has implemented a flip mechanism, allowing the lenses to be moved from the field of view. This process awkwardly shifts the weight of the headset but is still useful for quick conversations outside of mixed reality.

Overall, I am incredibly impressed with the HoloLens 2 hardware. Although the technology is relatively new and likely targeting a very specific audience, these limitations are not reflected in the design and/or build quality.

In short, Microsoft has clearly invested the time to deliver a high-quality “production-ready” product, instead of hiding behind common nomenclature such as “Developer Edition”, which is often used as an excuse for reduced quality and/or missing features.

Software

The HoloLens 2 runs Windows Holographic OS, which is part of the Windows 10 family.

Like all other versions of Windows 10, it includes a Start menu, which can be triggered via a one-handed gesture (pinching motion) or by gazing at your wrist, which triggers the Windows logo to appear.

Alongside the Start menu, other common Windows concepts are sprinkled throughout the operating system, specifically the Application Launcher, Edge Browser, Mail, Calendar, Store, etc. The HoloLens 2 is also fully AutoPilot and InTune compatible, meaning the device can be managed centrally like any other Windows 10 endpoint. These management features, alongside a valid Microsoft Account, helps to ensure the setup process is quick and simple, providing instant access to user applications, data, etc.

Recognising the lack of a standard input device (e.g. Mouse and keyboard), user interaction is achieved via “Gestures”, “Gaze”, “Hand Ray” and voice.

• Gestures allow the user to interact with virtual objects using their hands and fingers.
• Gaze allows the user to interact with virtual objects by looking (gazing) at specific items (e.g. Application, Buttons, etc.)
• The Hand Ray presents the user with a “laser pointer”, enabling gestures to be used on virtual objects that are anchored out of direct reach.
• The HoloLens 2 includes a microphone and speaker array, therefore can accept voice commands as an input (similar to voice assistants such as Amazon Alexa).

Gestures, gaze and voice can be used simultaneously, delivering a reasonably intuitive experience, with the hand ray reducing the need for physical movement (a common challenge in mixed reality, where virtual objects are anchored in the real world).

Similar to when multi-touch was first introduced, the new gestures do have a learning curve. For example, to “air tap”, the user must:

1. Create a loose fist, pointing the index finger straight up.
2. Tap the finger down.
3. Quickly raise it back up again.

Navigating the operating system and applications is fairly simple, primarily accomplished via air taps. For example, the image below is my five-year-old son interacting with Holo Lens 2. I had previously hidden multiple virtual dinosaurs around our house, which he then had to locate using mixed reality. Thanks to the advanced anchoring of virtual objects, the virtual dinosaurs look and feel part of the physical world. His reaction to seeing the dinosaurs hiding around our house was priceless and demonstrated the “magic” of this technology.

At the time of writing, there are 351 HoloLens 2 compatible applications in the Microsoft Store, highlights include 3D Viewer, Galaxy Explorer, Graffiti 3D, HoloBlocks, HoloTour, Robo Raid, HoloAnatomy, etc. Unsurprisingly, the majority of the applications are still very experimental, offering a glimpse into the “art of the possible”, but rarely adding tangible value.

Unfortunately, activities that require traditional typed input are far more cumbersome. Although Windows Holographic OS includes a virtual keyboard, the lack of any tactile feedback makes the process of typing very slow and prone to error. To make matters worse, the requirement to hold your arms up without a palm rest is surprisingly difficult, which amplifies the issues. I assume, as the application ecosystem matures, other interaction mechanisms, such as voice, will be favoured.

Unsurprisingly, HoloLens 2 is at its best when it is operating in true mixed reality mode (virtual objects anchored on the real world), not when attempting to mimic traditional productivity activities. If you need to write a document or respond to e-mail, your laptop is still a far better choice.

The video below was taken directly from HoloLens 2 and highlights me interacting with the operating system and a few applications. I have purposely not edited, enhanced or refined the video, hoping to highlight the good, the bad and the ugly. You will see me (more than once) fail to interact with a virtual object on the first attempt.

As you can see, the virtual overlay and anchoring of virtual objects is incredibly impressive and certainly beyond any other mixed reality or augmented reality device I have ever used. However, you can also see how the interactions can be cumbersome, which can break the illusion, resulting in frustration.

Conclusion

In conclusion, I am very impressed by the HoloLens 2 and excited to see how the product evolves. I am pleased Microsoft remain willing to invest in these innovative technologies, especially considering past failures and accepting that the viable market is very small (at least today).

Microsoft should also be commended for the quality of the product, which feels “production-ready” from the moment you pick-up the box. The hardware feels refined and robust (ready for business) and while the software needs some refinement, it is surprisingly intuitive and usable.

Finally, I am pleased that Microsoft targetted and achieved basic impact protection requirements (ANSI Z87.1, CSA Z94.3 and EN 166), making the product viable for my test use cases. Over the coming weeks, I will conclude my testing, therefore I look forward to seeing how HoloLens 2 performs, as well as the reception it receives from the users. I’ll be sure to post an update with the results.