FINEXUS finger tracking in VR

I stumbled upon a very good project that has been undertaken by the people at the University of Washington. This project involves tracking the movement of fingertips using co-existing magnetic fields. This will allow the user to grab objects more precisely, maybe even tie virtual shoelaces! I have been myself wondering if anybody has used magnetic apparatus to induce gravity for VR environments and now it seems there might some traction on this.

I would like to talk about this briefly for my student choice presentation.

These are some relevant links for this talk:

https://ubicomplab.cs.washington.edu/publications/finexus/

http://www.keyuc.com/research/Finexus/

 

With the resurgence of head-mounted displays for virtual reality, users need new input devices that can accurately track their hands and fingers in motion. Finexus is a multipoint tracking system using magnetic field sensing for such a purpose. By instrumenting the fingertips with electromagnets, the system is able to track fine fingertip movements in real time using only four magnetometers. To keep the system robust to noise, they operate each electromagnet at a different frequency and leverage bandpass filters in order to distinguish signals attributed to individual sensing points for localization. They develop a novel algorithm to efficiently calculate multiple electromagnets’ 3D positions from corresponding field strengths.

In order to address many of the shortcomings in existing hand-tracking solutions, they have designed Finexus, a 3D input device capable of tracking multiple fingertips using magnetic field (MF) sensing. The system continuously tracks fine finger movements, in real time, with electromagnets mounted on the backs of the fingers (see Figure 1 and the video figure). In order to enable multipoint tracking, they drive electromagnets using alternating current (AC) operating at distinct frequencies. By applying a bandpass filter centered at the corresponding frequency, the system is able to extract the magnetic fields generated from each individual electromagnet and remove unwanted noise (e.g., the Earth’s magnetic field and ambient electrical noise). They have designed a custom hardware and develop a novel algorithm to efficiently calculate the 3D position of multiple electromagnets from measurements of their magnetic fields. Because MF sensing is non-line-of-sight (NLOS), this system avoids the issue of occlusion inherent to optical-based tracking devices. Moreover, the MF sensing solution presented in this work does not suffer from drift, the fundamental limitation of IMUs.

Screen Shot 2017-11-13 at 4.55.18 PM
Picture taken from official site of Finexus
Screen Shot 2017-11-13 at 4.57.59 PM
Magnetic field decomposition and spatial ambiguity. (Left) The magnetic field H can be decomposed into two orthogonal vectors, Hr and Hθ. (Right) All points on a concentric circle (orange spots) have the same field strength. With unknown sensor orientations, these positions may have the same sensor reading, causing spatial ambiguity.

The System:

To localize electromagnets, Finexus leverages techniques similar to those used by the Global Positioning System (GPS). Intuitively, the system first calculates the distance between the electromagnet and four magnetic sensors, and then uses trilateration to identify the electromagnet’s 3D position. Instead of solving three unknown orientations for the 2D projection, they convert the magnetic field space into a beacon-like system and evolve an efficient algorithm for positioning multiple electromagnets. Below, they present the requisite background information and discuss the challenges inherent to a magnetic field-based tracking system . they then detail our methods for addressing these challenges to enable continuous multipoint 3D input; these include a predefined coordinate system, a band-passed filter , and custom hardware

Multi-Point tracking:

Finexus is able to simultaneously track multiple fingertips. Instead of using a permanent magnet, they leverage AC-driven electromagnets operating at different frequencies. By applying bandpass filters centered at these frequencies, our system can extract and differentiate the magnetic field emanating from individual electromagnets. The bandpass filter is a 6th order finite impulse response (FIR) filter with the 3 dB cutoff at +2 and -2 Hz from the center frequency. The data rate from the PCB is 320 samples/second, so the usable bandwidth is half of this, 160 Hz. Within this bandwidth, they drive our five electromagnets at 70 Hz, 85 Hz, 100 Hz, 115 Hz, and 125 Hz. The frequency selection approach taken in this work uses the widest possible bands while avoiding 60 Hz noise (and harmonics) emitted from surrounding electronic devices or the nearby power line infrastructure.

 

The Algorithm:

The localization algorithm can be summarized in the following 5 steps:

1. Read the magnetic vector H from magnetic sensors and keep it in a sliding window of size 160

2. For each axis in H: Apply the bandpass filter to extract magnetic fields emitted from individual electromagnets o Extract the envelope of the filtered data using the Hilbert transform

3. Calculate the L-2 norm of H (Eq. 4) 4. Substitute the variables r and θ in Equation

4. with x, y, z (Eq. 5 to Eq. 12)

5. Solve for the electromagnet’s 3D position (x, y, z) and render the results

Disclaimer: All data on this page is taken from the official page of Finexus and from the white paper : “Finexus: Tracking Precise Motions of Multiple Fingertips Using Magnetic Sensing” by Ke-Yu Chen , Shwetak Patel , Sean Keller.

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Create a website or blog at WordPress.com

Up ↑

%d bloggers like this: