Magnetic smart skins for augmented reality

Augmented reality gadgets are becoming common for our information intensive society assisting us to acquire and process the data. Although impressive in the realization and demonstrations, the obvious drawback of the state-of-the-art augmented and virtual reality devices relying on optical detection systems is their bulkiness, energy inefficiency and the stringent requirement for an operator to be at the line of sight of the device.
We envision that prospective augmented reality systems will strongly benefit from the recent developments in compliant on-skin electronics [1-3]. The fabrication of highly conformable gadgets requires the realization of the electronic replica of the exteroceptive sensory system of humans as well as calls for the acquiring new perception skills beyond those prescribed by the evolution. The representative example of the missing exteroceptive sense of humans is the magnetoception, which allows some of the mammals but not humans perceiving the location in space or directions based on the detection of magnetic fields. The first crucial step towards the realization of this vision was accomplished with the development of interactive magnetosensitive skins [4-6]. The key enabler for this technology is the shapeable [7] –namely, flexible [5,6], stretchable [8,9] and imperceptible [4]– magnetic field sensorics.
Here, we present the first on-skin gadgets, which replicate our natural proprioceptive sensory ability of detecting the motion. The technology is put forth to realize distributed arrays of magnetic field sensors on ultra-thin polymeric foils. Relying on this magnetically enabled electronic proprioception, we visualize the bodily motion and demonstrate the touchless manipulation of virtual objects for augmented reality systems.
Those highly conformable interactive devices possess great potential to extend the portfolio of tasks, which can be performed in virtual or augmented reality. The integration of gadgets in imperceptible electronic skins will open not only exciting possibilities for business or gaming industry but is also beneficial for safety and security applications, where the somatic manipulation of objects, e.g. turning regulation knobs located in a restricted environment is undesirable or even prohibited.

1. J. A. Rogers et al. Nature 477, 45 (2011).
2. S. Bauer et al., Adv. Mater. 26, 149 (2014).
3. M. Kaltenbrunner et al., Nature 499, 458 (2013).
4. M. Melzer et al., Nature Commun. 6, 6080 (2015).
5. M. Melzer et al., Adv. Mater. 27, 1274 (2015).
6. N. Münzenrieder et al., Adv. Electron. Mater. 2, 1600188 (2016).
7. D. Makarov et al., Appl. Phys. Rev. 3, 011101 (2016).
8. M. Melzer et al., Nano Lett. 11, 2522 (2011).
9. M. Melzer et al., Adv. Mater. 27, 1333 (2015).