Forschungsprojekte der Natural Media Group

Neuro Peirce

We investigate the neurocognitive underpinnings of co-speech gesture perception. For this aim we employed the well-established cognitive categories of the influential American logician and semiotician Charles Sanders Peirce.

Website


eCosmos

Complex multimodal bodies of data such as combined speech and gesture data represent a new frontier in humanities research demanding novel approaches to the mathematical modeling, mining and quantification of these data.

Website


3D heat maps of multimodal travel planning

This approach combines qualitative and quantitative methods to analyze audio, video and motion-capture data in correlation with with prepositional/adverbial phrases.

Website


Frames and mental maps as mediating structures

This study explores how coverbal gestures may function as physically articulated entry points to frames and mental maps.

Website


Felt qualities of experience in gestural enactments

Gestural re-/presentations of sound experience were analyzed in terms of spatial and temporal features.

Website


Social neuroscience of multimodal communication

To investigate the neurocognitive underpinnings of communication, a fMRI experiment was conducted with a movie sequence (Lola Rennt) as a semi-naturalistic stimulus.

Website


Head gestures in dialogue

A methodology employing motion-capture technology was developed to investigate the forms and functions of head gestures.

Website


Mapping and visualizing 3D gesture space

Based on data stemming from semi-structured interviews, a new method to map and visualize speakers' three-dimensional gesture spaces was developed.

Website


Description and appreciation of abstract art

Gestures in description of paintings from memory with respect to their meanings and message.

Website


Gesture and Architecture

Architecture students pursued their own artistic projects, using the MoCap system to e.g. create three-dimensional sketches through gestures.

Website


Influence of noise on gesture production

This study followed up on work by S. Harrison in factory surroundings that indicated an influence of environmental conditions (e.g. noise) on gesture production.

Website