Learning to recognize human activities using soft labels

Ninghang Hu, Gwenn Englebienne, Zhongyu Lou, Ben Kröse

Research output: Contribution to journalArticleAcademicpeer-review

11 Citations (Scopus)


Human activity recognition system is of great importance in robot-care scenarios. Typically, training such a system requires activity labels to be both completely and accurately annotated. In this paper, we go beyond such restriction and propose a learning method that allow labels to be incomplete and uncertain. We introduce the idea of soft labels which allows annotators to assign multiple, and weighted labels to data segments. This is very useful in many situations, e.g., when the labels are uncertain, when part of the labels are missing, or when multiple annotators assign inconsistent labels. We formulate the activity recognition task as a sequential labeling problem. Latent variables are embedded in the model in order to exploit sub-level semantics for better estimation. We propose a max-margin framework which incorporate soft labels for learning the model parameters. The model is evaluated on two challenging datasets. To simulate the uncertainty in data annotation, we randomly change the labels for transition segments. The results show significant improvement over the state-of-the-art approach.
Original languageEnglish
Pages (from-to)1973-1984
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Issue number10
Publication statusPublished - 16 Oct 2016


Dive into the research topics of 'Learning to recognize human activities using soft labels'. Together they form a unique fingerprint.

Cite this