Hardegger, Michael, Roggen, Daniel, Calatroni, Alberto and Tröster, Gerhard (2016) S-SMART: a unified Bayesian framework for Simultaneous Semantic Mapping, Activity Recognition and Tracking. ACM Transactions on Intelligent Systems and Technology, 7 (3). p. 34. ISSN 2157-6904

	PDF (Accepted version, but the camera ready will contain slight chnges.) - Accepted Version Restricted to SRO admin only Download (2MB)
	PDF - Published Version Restricted to SRO admin only Download (2MB)

Abstract

The machine recognition of user trajectories and activities is fundamental to devise context-aware applications for support and monitoring in daily life. So far, tracking and activity recognition were mostly considered as orthogonal problems, which limits the richness of possible context inference. In this work, we introduce the novel unified computational and representational framework S-SMART that simultaneously models the environment state (semantic mapping), localizes the user within this map (tracking), and recognizes interactions with the environment (activity recognition). Thus, S-SMART identifies which activities the user executes where (e.g., turning a handle next to a window), and reflects the outcome of these actions by updating the world model (e.g., the window is now open). This in turn conditions the future possibility of executing actions at specific places (e.g., closing the window is likely to be the next action at this location).
S-SMART works in a self-contained manner, and iteratively builds the semantic map from wearable sensors only. This enables the seamless deployment to new environments. We characterize S-SMART in an experimental dataset with people performing hand actions as part of their usual routines at home and in office buildings. The framework combines dead reckoning from a footworn motion sensor with template-matching-based action recognition, identifying objects in the environment (windows, doors, water taps, phones, etc.) and tracking their state (open/closed, etc.). In real-life recordings with up to 23 action classes, S-SMART consistently outperforms independent systems for positioning and activity recognition, and constructs accurate semantic maps. This environment representation enables novel applications that build upon information about the arrangement and state of the user’s surroundings. For example, it may be possible to remind elderly people of a window that they left open before leaving the house, or of a plant they did not water yet, using solely wearable sensors.

Item Type:	Article
Keywords:	Wearable computing, mobile computing, localisation, SLAM, mapping, activity recognition, context recognition
Schools and Departments:	School of Engineering and Informatics > Engineering and Design
Subjects:	T Technology > T Technology (General) > T0010 Communication of technical information T Technology > TK Electrical engineering. Electronics Nuclear engineering > TK7800 Electronics > TK7885 Computer engineering. Computer hardware
Related URLs:	Publisher
Depositing User:	Daniel Roggen
Date Deposited:	14 Sep 2015 09:47
Last Modified:	12 Sep 2017 03:41
URI:	http://srodev.sussex.ac.uk/id/eprint/56648

View download statistics for this item

📧 Request an update