Research Material




Observation of Human Emotions






Concepts from Research Works




  • Nearest Pose Retrieval Based on K-d tree

Since HERB has 24 degrees of freedom which create a high-dimensional pose space, if each joint orientation is represented as one dimension and thus a pose is represented as a vector.

There are times when we want to retrieve poses that meet certain requirements, for instance, we want to reach a nearest rest pose, which is a starting pose to many animations, to prepare for next step animations. Such search may take very long as the pose library gets huge, if we perform a linear search by brutal force. The sheer dimensions could cause the curse of dimensionality.


24dovThe 24 dimensional joint orientation data creates a high-dimensional pose space

Using KDTree’s Nearest neighbor search, we could potentially minimize the time cost of retrieving the nearest pose, even if we have a huge pose library.A k-d tree (short for k-dimensional tree) is a space-partitioning data structure for organizing points in a k-dimensional space. k-d trees are a useful data structure for several applications, such as searches involving a multidimensional search key (e.g. range searches and nearest neighbor searches). k-d trees are a special case of binary space partitioning trees.The k-d tree is a binary tree in which every node is a k-dimensional point.


Graphical representation of a 3-d tree ( a KDTree when k=3)

Nearest neighbor search (NNS), also known as proximity search, similarity search or closest point search, is an optimization problem for finding closest (or most similar) points. Closeness is typically expressed in terms of a dissimilarity function: The less similar are the objects the larger are the function values. Formally, the nearest-neighbor (NN) search problem is defined as follows: given a set S of points in a space M and a query point q ∈ M, find the closest point in S to q. Donald Knuth in vol. 3 of The Art of Computer Programming (1973) called it the post-office problem, referring to an application of assigning to a residence the nearest post office. A direct generalization of this problem is a k-NN search, where we need to find the k closest points.

Most commonly M is a metric space and dissimilarity is expressed as a distance metric, which is symmetric and satisfies the triangle inequality. Even more common, M is taken to be the d-dimensional vector space where dissimilarity is measured using the Euclidean distance, Manhattan distance or other distance metric. However, the dissimilarity function can be arbitrary. One example are asymmetric Bregman divergences, for which the triangle inequality does not hold.

Reference: wikipedia




  • The uncanny valley


The uncanny valley and its implications to HERB

Since complex and natural social interaction between humans and artificial agents necessitates the display of rich emotions in order to be believable, socially relevant, and accepted, and to generate the natural emotional responses that humans show in the context of social interaction, such as engagement or empathy. Whereas some robots use faces to display (simplified) emotional expressions, for other robots such as Nao or out robot HERB, body language is the best medium available given their inability to convey facial expressions.


from: Emotional body language displayed by artificial agents

ACM Transactions on Interactive Intelligent Systems (TiiS) – Special Issue on Affective Interaction in Natural Environments archive
Volume 2 Issue 1, March 2012
Article No. 2
ACM New York, NY, USA




  • The six basic emotions, upper body expression, and blending with neutral pose

Six basic emotions have great implications on HERB. On ETC bowtie‟s HERB project, we try to translate the six basic emotions anger, disgust, fear, happiness, sadness, and surprise into HERB movements. Although HERB doesn‟t have facial expression and has limited lower body movements, The papers studied partial occlusion of part of the body, and corresponding effects for emotion perceptions. For each of the six emotions, they chose the two clips with the highest recognition rates. They then occluded different parts of the body: the head motion (or NH for “No Head motion”), the lower body motion (NL), and the upper body motion (NU). The unaltered motion is labeled OR for “original”. We did not alter the root motion for any of the conditions to avoid very unnatural motions that could affect the ratings in unintended ways.

from:The effect of posture and dynamics on the perception of emotion

SAP ’13 Proceedings of the ACM Symposium on Applied Perception
Pages 91-98
ACM New York, NY, USA ©2013
ISBN: 978-1-4503-2262-1 doi>10.1145/2492494.2492500



  • Inspirations from Guy Hoffman’s Acting lamp, jazz improv robot and more

–Robotic desk lamp: A collaborative lighting assistant which serves as a non-anthropomorphic robotic platform, using a cognitive architecture for robotic teammates based on the neuro-psychological principles of perceptual symbols and simulation, with the aim of attaining increased fluency in human-robot teams. An instantiation of this architecture was implemented on a robotic desk lamp, performing in a human-robot collaborative task.



Related paper:

HRI ’08 Proceedings of the 3rd ACM/IEEE international conference on Human robot interaction
Pages 1-8
ACM New York, NY, USA ©2008
ISBN: 978-1-60558-017-3doi>10.1145/1349822.1349824

–The world’s first human-robot rendition of Duke Jordan’s “Jordu”, for human piano and robot marimba. Also, the stage debut of Shimon, Georgia Tech’s new robotic marimba player. Using a novel gesture-based improvisation and motor control system, the robot listens to the human play and tries to match its beat, sync, and themes. It uses choreographic gestures to generate both motion and sound.

–A robotic speaker device designed for joint listening and embodied performance of the music played on it. The robot generates smoothed real-time beat-synchronized dance moves, uses nonverbal gestures for common ground, and can make and maintain eye-contact.

Related Paper:

HRI ’13 Proceedings of the 8th ACM/IEEE international conference on Human-robot interaction
Pages 317-324
IEEE Press Piscataway, NJ, USA ©2013
ISBN: 978-1-4673-3055-8


List of Documents for download in pdf

Research Paper Digests



Nearest Pose Retrieval Based on K-d tree