A bimodal soft electronic skin for tactile and touchless interaction in real time


A bimodal soft electronic skin for tactile and touchless interaction in real time

Ge, J.; Wang, X.; Drack, M.; Volkov, O.; Liang, M.; Canon Bermudez, G. S.; Wang, C.; Zhou, S.; Faßbender, J.; Kaltenbrunner, M.; Makarov, D.

The transformative emergence of smart electronics, human-friendly robotics and supplemented or virtual reality will revolutionize the interplay with our surrounding. The complexity that is involved in the manipulation of objects in these emerging technologies is dramatically increased, which calls for electronic skins (e-skin) that can conduct tactile and touchless sensing events in a simultaneous and unambiguous way. Integrating multiple functions in a single sensing unit offers the most promising path towards simple, scalable and intuitive-to-use e-skin architectures. However, by now, this path has always been hindered by the confusing overlap of signals from different stimuli.
Here, we put forward the field of soft, flexible electronics by developing a compliant magnetic microelectromechanical platform (m-MEMS), which is able to transduce both tactile (via mechanical pressure) and touchless (via magnetic field) stimulations simultaneously and discriminate them in real time [1]. For the first time, the electric signals from tactile and touchless interactions are intrinsically separated into two different regions, allowing the m-MEMS, a single sensor unit, to unambiguously distinguish the two modes without knowing the signal history.
Owing to its intrinsic magnetic functionality, our complaint m-MEMS platform is able to discriminate magnetic vs. non-magnetic objects already upon touchless interaction. With this intrinsic selectivity, we address the long-standing problem in the field of touchless interaction – namely, the issue of interference with objects, which are irrelevant or even disturbing the interaction process. In addition, the interaction process is programmable. The sensitivity of the two interaction modes could be tuned by adjusting the magnetic field of the objects able to meet the requirements of different interaction tasks.
By using tactile and touchless sensing functions simultaneously, our m-MEMS e-skins enable complex interactions with a magnetically functionalized physical object that is supplemented with content data appearing in the virtual reality. We demonstrated data selection and manipulation with our m-MEMS e-skins leading to the realization of a multi-choice for augmented reality through three dimensional (3D) touch. Beyond the field of augmented reality, our m-MEMS will bring great benefits for healthcare, e.g. to ease surgery operations and manipulation of medical equipment, as well as for humanoid robots to overcome the challenging task of grasping.

[1] J. Ge, X. Wang, M. Drack, O. Volkov, M. Liang, G. S. Cañón Bermúdez, R. Illing, C. Wang, S. Zhou, J. Fassbender, M. Kaltenbrunner, and D. Makarov. A bimodal soft electronic skin for tactile and touchless interaction in real time. Nature Communications 10, 4405 (2019).

Keywords: flexible electronics; shapeable magnetoelectronics

  • Lecture (Conference) (Online presentation)
    2020 MRS Fall Meeting, 02.12.2020, Boston, USA

Permalink: https://www.hzdr.de/publications/Publ-32022
Publ.-Id: 32022