Towards Everyday Life Logging: Complex Activity Recognition with Smartwatch Data Fusion

Abstract

Human Activity Recognition (HAR) has garnered significant attention in both research and industry, particularly within the realms of healthcare and elderly life support. The shift towards remote work has exacerbated health risks associated with reduced physical activity and prolonged sedentary behavior, underscoring the critical need for advanced HAR research. Unlike existing studies that predominantly focus on simple, repetitive human activities such as sitting and standing, this research adopts a novel approach by holistically reconsidering human activity, and subdividing behaviors into simple and complex activities. For example, office work is a complex activity, comprising repetitive atomic activities such as writing, reading, and computer usage, often intermixed with irrelevant atomic activities like drinking.To accurately identify such intricate activities and their durations in real-life scenarios, we propose a pipeline that features convenient measurement using sensors embedded in a smartwatch and a lightweight inference algorithm based on a custom spatiotemporal-edge-weighted Graph Neural Network. This data-driven framework effectively captures and interprets the nuances of complex activities. Through a rigorous validation process that includes 5-fold, 2-fold, and reversed 5-fold cross-validation (CV), the results demonstrate the pipeline's proficiency in representing and recognizing these activities, achieving an accuracy of 0.93 and an F1 score of 0.87 in our complex activity dataset, and an accuracy of 0.96 and an F1 score of 0.96 in the open MHEALTH dataset under reversed 5-fold CV. These findings underscore the potential of this approach in addressing contemporary health challenges posed by changing work environments.

Keywords: Human Activity Recognition, wearable sensors fusion, graph neural network, Mobile computing