Maker Spaces FabLab NExT Lab Robotics Lab Print Room and Loans

Microsoft Hololens

This article introduces the Microsoft HoloLens and how it works.

Microsoft HoloLens

The HoloLens is an optical head-mounted display that composites virtual content with the users field of view by rendering to a transparent stereoscopic light field display. The illusion of virtual content appearing fixed in place (registered to the physical environment) is achieved through a combination of 3D scan data created with an infrared depth camera and “inside out tracking” using feature pixels from RGB video. These cameras detect user gestures and hand location, and a 6DOF sensor on the device captures head position and orientation to infer the wearer’s gaze. Users interact with virtual content using a combination of gestures, gaze and voice commands.

The head-mounted display of the HoloLens has several advantages over AR and VR experiences viewed through the video screen of a phone or tablet. It frees up your hands to interact with your physical environment or digital content in a natural way. The position and depth of holograms is also easier to infer from the see through display than from mobile cameras with different focal lengths to your eyes.

How the HoloLens AR/MR Works

The HoloLens uses inside-out tracking to calculate the changes in the position of your headset as you move around in your physical environment. Inside-out tracking works by using a pair of cameras (in the case of the HoloLens, these are infrared depth cameras) to observe features (points of high contrast) in your environment, and triangulates the position of your head by determining the offset of these features in each camera. This position is combined with data from the inertial motion unit (IMU) that provides accurate measurements of the roll, pitch and yaw of the headset

Technical Specifications

Optics

  • See-through holographic lenses (waveguides)

  • 2 HD 16:9 Light Engines

  • Holographic Resolution of 2.3M total light points

  • Holographic Density of >2.5k radiants (light points per radian)

Sensors

  • 1 IMU

  • 4 Environment-Understanding Cameras

  • 1 Depth Camera

  • 1 2MP photo/HD video Camera

  • Mixed Reality Capture

  • 4 Microphones

  • 1 Ambient Light Sensor

Input / Output / Connectivity

  • Built-in stereo speakers

  • 3.5 mm Audio Jack

  • Volume up/down buttons

  • Brightness up/down buttons

  • Power button

  • Battery Status LED's

  • Wi-Fi 802.11ac

  • Micro USB 2.0

  • Bluetooth 4.1 LE

Power

  • 2-3 Hours active use

  • 2 weeks standby

Processors

  • Intel 32bit processor with TPM

  • Microsoft Holographic Processing Unit (HPU 1.0)

Memory

  • 64GB Flash Storage

  • 2GB Ram

Loaning a HoloLens

  • MSD has 10 HoloLens

  • Students and staff can book through RefTab in 1hr, 2hr, 3hr and 6hr intervals from the loans desk

  • Special arrangements must be made for larger workshops

Set up

The Microsoft HoloLens is a head-mounted display. An adjustable wheel is located at the back of the device.

The HoloLens Kit contains:

  • HoloLens Device

  • Clicker

  • Carrying Case

  • Microfiber cloth

  • Charger

  • Micro-USB 2.0 cable

Device Support

The HoloLens features power, brightness and volume controls, as well as speakers, microphones and an adjustable wheel.

  1. Power Button + Battery Status LEDs

  2. Volume up/down + Brightness up/down

  3. Built-in speakers and Microphones

  4. Adjustable wheel

Fitting

Use the following steps to fit the HoloLens:

  • Rotate the headband up and slide it back

  • Turn the adjustment wheel to extend the headband

  • Place the HoloLens over your head and slide the visor back until the nose-pads are resting comfortably on your nose

  • Make sure the device sits comfortably on your forehead, just below your hairline, then tighten the adjustment wheel at the back for a snug fit

Calibration

Calibration should be run for each person using the HoloLens during each use.

Open HoloLens Calibration app in the Home Screen. This app is used to calculate inter-pupillary distance (the distance in mm between centers of pupils of the eyes).

  1. Bloom to open Home Screen

  2. Launch Calibration app

  3. Follow Cortana's Instructions

Saving User Profiles

Saving user profiles is not yet supported by the Microsoft HoloLens

Voice, Gaze and Gestures

Voice Support

You can access voice support by saying ' Hey Cortana'. Cortana can take photos, record video or open and close applications.

HoloLens also interacts with spatial sounds. It synthesises sound so you can hear holograms from anywhere in the room, even if they are behind you.

Gaze tracking

Glance navigation is shown in the center of your field of view as a white dot. It can be controlled by moving and tilting your head.

The gaze is usually used for precise interaction compared to hand gestures.

Gesture Input

There is a 'Learn Gestures' app from the windows home screen. Basic gestures shown below.

Hands must be within the 'Gesture Frame' (roughly from nose to waist and between the shoulders) when performing movements.

The main HoloLens gestures are BLOOM and TAP.

  1. Bloom is the "home" gesture only

  2. Tap/Air Tap is performed by positioning your finger in the ready position and then tapping down to select,

Gestures are detected by short range depth camera, therefore any fingers obscured by your hand will not be recognised.

Windows Device Portal

Device portal allows you to manage your device over WiFi or USB. Common uses of the device portal are to:

  1. Live-stream video to a computer or projector

  2. Access and download saved images and video from the device

  3. Manually set calibration data (IPD) without re-running the calibration app

To access the device portal, a few steps must be taken, which are outlined in further detail below

  1. Enabling developer mode

  2. Accessing the device portal from a browser

  3. Creating credentials

Enabling Developer Mode and Device Portal

  1. On your HoloLens, place the Settings tile

  2. Select Update and Security

  3. Select the For Developers option in the side bar

  4. Turn on Developer Mode

  5. Scroll down and enable Device Portal

  6. Click the Remove icon in the corner of the Settings tile to save and exit

Accessing the Device Portal

  1. Ensure that your HoloLens and computer are connected to the same WiFi network

  2. On your HoloLens, check your IPV4 Address. You can find this under:

    1. Settings > Network & Internet > Wi-Fi > Advanced Options

  3. In an internet browser, type the IPV4 address and press enter

  4. Depending on your browser, you will receive a different prompt to proceed with the connection. In Edge, click on Details, then Go to Webpage. In Chrome, click on Advanced, then Proceed. See notes below on accepting security certificates.

  5. If you have problems connecting to device portal, see this troubleshooting guide for solutions to common problems.

Creating Credentials

When accessing device portal for the first time, you will be asked to create credentials

It is important that you remember these credentials as you may be required to reset the device if they are lost. Also consider creating credentials that can be shared with other users of the device, so that they too can share their experiences.

  1. On your computer, after completing the "Accessing the Device Portal", request a pin from your device. It will be shown on the HoloLens

  2. Enter the pin, and set your username and password

  3. Click Pair

  4. Log in with your credentials on your computer.

Configuring common options

If you are only using your device in secure locations, you may wish to consider disabling SSL security, which will simply avoid a certificate error when connecting to your device. Alternatively, you can download and install the security certificate

Further Reading:

Using the Windows Device Portal

Live Streaming

The HoloLens can live-stream first person view to any other device (computers, phones, etc.) on the same network.

To allow this function, device portal must be configured.

Accessing the Stream

The video stream can be accessed:

  1. Via browser (Chrome offers best performance in this regard) by entering the IP address. Either:

    1. Connect directly by clicking on the connected headset in the Fologram application, or

    2. Enter the IP address of the headset, enter the device portal credentials, and click on Mixed Reality Capture (or go directly to "http://<your-ip>/Mixed%20Reality%20Capture.html")

  2. Via the HoloLens app for Windows 10

Via Fologram for Rhino

  1. Launch Fologram on the HoloLens

  2. Connect your headset to Fologram for Rhino

  3. Click on your device icon in the connected devices panel

  4. Login using your device portal login.

Don't know your device portal login? At the login screen, change the address in your browser to point to

http://<your-ip>/devicepair.html

Where <your-ip> is the ip address of your HoloLens device. This will display a code on your headset and you can then reset login details. Once in device portal, click on Views -> Mixed reality capture to get to the stream panel. From here you can live preview your device or record your session. Some things to remember:

Considerations for Live Streaming

  1. Only one application (including the live-stream) can access the camera at a time. This means that some operations (such as scanning for QR codes, using image trackers) will not be available whilst live streaming. However you can always turn live streaming back on once you have performed these actions.

  2. There is always some delay in the stream. This can vary from less than a second to up to 30 seconds. If you experience significant delays, you may wish to stop and start the stream.

  3. Live streaming will reduce the performance of the headset, as it requires some processing power.

  4. Live streaming uses a reasonable amount of bandwidth. If you are using it on a congested network, you may face performance issues with your stream and with applications that require a continuous network connection. If you encounter these issues, you may wish to reduce the quality of the stream.

Constraints and Considerations of mixed reality capture.

The following articles outline considerations and constraints when using mixed reality capture.

Performance considerations whilst using mixed reality capture

Restarting, Updating, Recharging

Update HoloLens

HoloLens periodically checks for system updates so you don't have to. When an update is available, it will be automatically downloaded and installed the next time your device is plugged in and connected to the Internet but isn't in use.

To manually check for updates, go to Settings > Update & Security > Check for updates. If the Settings app says your device is up to date, you have all the updates that are currently available.

Go back to a previous version

In some cases, you might want to go back to a previous version of the HoloLens software. You can do this by using the Windows Device Recovery Tool to reset your HoloLens to the earlier version.

Note: Going back to an earlier version will delete your personal files and settings.

To go back to a previous version:

  1. Make sure you don't have any phones or Windows devices plugged in to your PC.

  2. On your PC, download the Windows Device Recovery Tool (WDRT).

  3. Download the HoloLens Anniversary Update recovery package.

  4. When the download is finished, open File Explorer > Downloads. Right-click the zipped folder you just downloaded, and select Extract all... > Extract to unzip it.

  5. Connect your HoloLens to your PC using the micro-USB cable it came with. (Even if you've been using other cables to connect your HoloLens, this one works best.)

  6. The tool will automatically detect your HoloLens. Select the Microsoft HoloLens tile.

  7. On the next screen, select Manual package selection and choose the installation file contained in the folder you unzipped in step 4. (Look for a file with the extension .ffu.)

  8. Select Install software, and follow the instructions to finish installing.

If the tool doesn't detect your HoloLens, try restarting your PC. If that doesn't work, select My device was not detected, choose Microsoft HoloLens, and follow the instructions.

Restart or Recover your HoloLens

If your HoloLens isn’t running well or is unresponsive, try the following things:

First, try restarting the device: say, "Hey Cortana, restart the device."

If you’re still having problems, press the power button for 4 seconds, until all of the battery indicators fade out. Wait 1 minute, then press the power button again to turn on the device.

If neither of those things works, force restart the device. Hold down the power button for 10 seconds. Release it and wait 30 seconds, then press the power button again to turn on the device.

Reset or recover your HoloLens

If restarting your HoloLens doesn’t help, another option is to reset it. If resetting it doesn’t fix the problem, the Windows Device Recovery Tool can help you recover your device.

Resetting or recovering your HoloLens will erase all of your personal data, including apps, games, photos, and settings. You won’t be able to restore a backup once the reset is complete.

Reset

Resetting your HoloLens keeps the version of the Windows Holographic software that’s installed on it and returns everything else to factory settings.

To reset your HoloLens, go to Settings > Update > Reset and select Reset device. The battery will need to have at least 40 percent charge left to reset.

Recover using the Windows Device Recovery Tool

Before you use this tool, see if restarting or resetting your HoloLens fixes the problem. The recovery process may take some time, and the latest version of the Windows Holographic software approved for your HoloLens will be installed.

To use the tool, you’ll need a computer running Windows 10 or later, with at least 4 GB of free storage space. Please note that you can’t run this tool on a virtual machine.

To recover your HoloLens:

  1. Download and install the Windows Device Recovery Tool on your computer. Download now.

  2. Connect your HoloLens to your computer using the Micro USB cable that came with your HoloLens.

  3. Run the Windows Device Recovery Tool and follow the instructions.

If your HoloLens isn’t automatically detected, select My device was not detected and follow the instructions to put your device into recovery mode.

Spaces

The following article outline tips for setting and mapping spaces.

Limitations with Spaces

It is not uncommon when connecting a second device to experience some issues. The article below goes through some troubleshooting strategies that may help.

HoloLens Clicker

The clicker was designed specifically for HoloLens and gives you another way to interact with holograms. It comes with HoloLens, in a separate box. Use it in place of hand gestures to select, scroll, move, and resize.

Pair and charge the clicker

To pair the clicker with your HoloLens, see Pair Bluetooth devices.

When the clicker battery is low, the battery indicator will blink amber. Plug the Micro USB cable into a USB power supply to charge the device.

Indicator lights

Here's what the lights on the clicker mean.

Blinking white. The clicker is in pairing mode. Fast-blinking white. Pairing was successful. Solid white. The clicker is charging. Blinking amber. The battery is low. Solid amber. The clicker ran into an error and you'll need to restart it. While pressing the pairing button, click and hold for 15 seconds.

Hold the clicker

To put on the clicker, slide the loop over your ring or middle finger with the Micro USB port toward your wrist. Rest your thumb in the indentation.

Clicker gestures

Clicker gestures are small wrist rotations, not the larger movements used for HoloLens hand gestures. And HoloLens will recognize your gestures and clicks even if the clicker is outside the gesture frame, so you can hold the clicker in the position that's most comfortable for you​.

Select. To select a hologram, button, or other element, gaze at it, then click.

Click and hold. Click and hold your thumb down on the button to do some of the same things you would with tap and hold, like move or resize a hologram.

PreviousPlatforms (Hardware)NextMobile

Last updated