Natural Typing Recognition via Surface Electromyography

By using a computer keyboard as a finger recording device, we present the largest existing dataset for gesture recognition via surface EMG, and use deep learning to achieve over 90% character-level accuracy on reconstructing typed text entirely from measured muscle potentials. We prioritize the temporal structure of the EMG signal instead of the spatial structure of the electrode layout, using network architectures inspired from those used for realtime spoken language recognition. We present a realtime architecture for recognizing the rapid movements of computer typing, which occur at irregular intervals and often overlap in time. The size of our dataset also allows us to study gesture recognition after synthetically downgrading the spatial or temporal resolution, showing the system capabilities necessary for realtime gesture recognition.