A 10.8 cm x 10.8 cm robotic skin capable of jointly detecting applied force and proximity of approaching objects is constructed by embedding IR proximity sensors into a sheet of clear PDMS rubber. A small microcontroller (ATmega2560) queries an 8x8 array of sensors over I2C at a rate of 20Hz. A classifier is trained to recognize affective touch gestures using a window of measurement frames. Using frames from the gesture data, the skin can also learn to discriminate between approaching hands and obstacles, and inform the host robot to respond appropriately (i.e., avoid or allow contact). The skin behavior can be simulated using a stochastic model, providing a tool to determine safe behavior conditions and failure probabilities.
Dana Hughes, John Lammie and Nikolaus Correll, “A Robotic Skin for collision Avoidance and Affective Touch Recognition,” RA-L, 2018. [PDF]
A smart tire capable of learning to identify the type of terrain (e.g, grass, concrete, gravel, etc.) a vehicle is driving on is constructed by attaching thin, flexible piezoelectric sensors to the interior face of a rubber tire. The generated signal represents changes in local deformation of the tire at the location of the sensor. A small, WiFi capable platform (e.g., Intel Edison, ESP32) is attached to the wheel. Terrain is predicted from a 160 ms window of samples using a small CNN, with low-bandwidth terrain estimates transmitted over WiFi.
Switchback is a novel e-textile input device capable of detecting tapping and sliding gestures. The RF-based sensing approach allows for swatches to be designed based on a multitude of microwave transmission lines (coax, microstrip, stripline, etc.), and requires only a single input port to detect a continuum of finger positions. Switchback is designed to pair with Bluetooth enabled devices (e.g., smartphone) and provide an unobtrusive means of control (e.g., start, stop and adjust volume of music). A circuit board consisting of the microwave reflectometer, microcontroller and Bluetooth components is roughly one square inch, and senses touches, identifies gesture, and issues commands over Bluetooth.
Dana Hughes, Halley Profita, Sarah Radzihovsky and Nikolaus Correll, “Intelligent RF-Based Gesture Input Devices Implemented using e-Textiles,” Sensors, 2017.[PDF]
Dana Hughes, Halley Profita and Nikolaus Correll, “Switchback: An On-Body RF-Based Gesture Input Device,” ISWC, 2014.[PDF]
This paper proposes a modular approach to classification using convolutional neural networks. Individual CNN modules are collocated with a local group of sensors, and are used to perform feature extraction from local sensor data. The output of each module is aggregated to perform final classification. Human activity recognition using wearable sensors is used to demonstrate this approach; this choice of task is motivated by the need for wearable sensing and computing components to be small, light and low powered, as well as the natural hierarchy of sensor distribution on the human body. We demonstrate that the required computing resources per element for a distributed, modular CNN is significantly lower than a centralized, monolithic architecture, while the reduction in classification accuracy is minimal.
Dana Hughes and Nikolaus Correll, “Distributed Convolutional Neural Networks for Human Activity Recognition in Wearable Robotics,” DARS, 2016.
Motivated by the Pascinian corpuscle in human skin, we constructed a skin using silicone rubber with a network of sensing and computing nodes, each node using a small microphone to detect local vibrations. The skin is capable of detecting contact due to objects rubbed against it, localize the point of contact, and identify the texture of the material making contact. The skin performs a significant amount of computation in-material, converting high-bandwidth vibration signals to low-bandwidth information on contact location and texture class. An amorphous algorithm is used to determine the subset of nodes in the network which have detected contact, and to perform localization to determine the position of the contact. An individual node is elected to identify the texture using a logistic regression classifier. The algorithm is designed to be fully distributed, requires only local communication, and is agnostic to position of individual nodes in the skin.
Dana Hughes and Nikolaus Correll, “Texture Recognition and Localization in Amorphous Robotic Skin,” Bioinspiration & Biomimetics, 2015. [PDF]
Dana Hughes and Nikolaus Correll, “A Soft, Amorphous Skin that can Sense and Localize Texture,” ICRA, 2014. [PDF]