Gesture Research

/Gesture Research
Gesture Research 2018-01-22T20:08:28+00:00

Investigating multimodal alignment amongst gesture, prosody, and discourse in collaboration with Dr. Stefanie Shattuck-Hufnagel, Speech Communication Group, RLE, MIT.

Building a modular gesture labeling system to fit multiple gesture research needs with potential to expand into automated labeling. This requires teasing apart the kinematic dimensions from the meaning of gestures in coding schemas. One of our goals is to identify these dimensions and make sure they are mutually exclusive. A second goal is to add quantitative measures to qualitative attributes of speech-accompanying gestures. Overall, this looks at the broad span of multimodal alignment across speech, prosody, and discourse.

Publications

Shattuck-Hufnagel, S., Ren, A., Mathew, M., Yuen, I., Demuth, K. (2016) Non-referential gestures in adult and child speech: Are they prosodic?. Proc. Speech Prosody 2016, 836-839. (link)

Shattuck-Hufnagel, S., Ren, A., (2012) Preliminaries to a Kinematics of Gestural Accents. Presented at International Society for Gesture Studies 6 San Diego.

Shattuck-Hufnagel, S., Ren, A., Tauscher, E. (2010) Are torso movements during speech timed with intonational phrases? Presented at Speech Prosody 2010.

Tools for Researchers

I’m putting together a set of handy tools for researchers that I used on a daily basis. Right now there’s a simple t-test/chi-square test, but soon there will be more on data visualizations and research poster templates!