Exploring Audio and Kinetic Sensing on Earable Devices

We explore audio and kinetic sensing on earable devices with COTS form factor. For the study, we prototyped earbud devices with a 6-axis inertial measurement unit and a microphone. We systematically investigate the differential characteristics of the audio and inertial signals to assess its feasibility in human activity recognition. Our results demonstrate that earable devices have a superior signal-to-noise ratio under the influence of motion artefacts and are less susceptible to acoustic environment noise. We then present a set of activity primitives and corresponding signal process- ing pipeline to showcase the capabilities of earbud devices in converting accelerometer, gyroscope and audio signals into the targeted human activities with a mean accuracy reaching up to 88% in varying environmental conditions.