Using Deep Data Augmentation Training to Address Software and Hardware Heterogeneities in Wearable and Smartphone Sensing Devices

Even small variations in mobile hardware and software can cause significant heterogeneity in the sensor data each device collects. For example, the microphone and accelerometer in different device models can respond differently to the same phenomena. Furthermore, factors like the computational load of, for instance a smartphone, can cause key behavior like sampling rates to fluctuate, further polluting the data. When sensing devices are deployed at large-scale, examples of sharply lower classification accuracy are being discovered due to what is collectively known as sensing system heterogeneity. In this work, we take an unconventional approach and argue against solving individual forms of heterogeneity (e.g., improving OS behavior, or the quality and uniformity of components). Instead, we propose and build classifiers that themselves are more tolerant to these issues by leveraging deep learning and a data argumentation training process. Neither augmentation nor deep learning have previously been attempted to cope with sensor heterogeneity - we systematically investigate how these two machine learning methodologies can be adapted to solve such problems, and identify when and where they are able to be successful. We find that this approach is able to reduce classifier errors in aggregate by 9% and 17% for a range of inertial and audio based mobile classifiers.