NeuroPeirce
Gesture as test ground for the social neuroscience of Peirce's universal categories
NeuroPeirce is the continuation of the Natural Media & Engineering project. It spans three faculties, i.e. the faculties of Philosophy, Medicine, as well as Mathematics, Informatics & Natural Sciences and bridges concepts well established in the history of science with cutting-edge research methodologies and technologies. As Natural Media as a whole, NeuroPeirce integrates diverse disciplines - such as linguistics, semiotics, psychology, neuroscience, and computer science - to investigate the dynamic complexity of dis-/embodied cognition and multimodal communication. While building on technological advances and empirical results achieved in the first phase of Natural Media, the focus of NeuroPeirce is on developing a new research paradigm for the social neuroscience of multi-channel interaction.
Employing advanced motion capture technology and innovative brain-imaging, the aim of this sub-project is to test the neuro-cognitive representation of fundamental cognitive categories as introduced by the influential American logician and semiotician Charles Sanders Peirce. Peirce assumed a set of universal categories to underlie all processes of perception, reasoning, and communication: firstness, e.g. quality, potentiality of meaning, secondness, e.g. facts, contextualized meaning, and thirdness, e.g. habits, patterns, rules. He further proposed a better known typology of signs, e.g. icon, index, symbol and different kinds of interpretants, i.e. meaningful responses in the attending mind. Gestures are particularly apt at testing the validity of these categories, because they are not as highly codified as spoken and written language. The dynamic, visuo-spatial mediality of gestures affords a broad range of forms and functions. Compared to thirdness-confined linguistic symbols, gestures may exhibit all of Peirce's universal categories to greater and more varying degrees.
On the basis of video and motion capture data obtained in the HumTec Natural Media Lab, a series of functional magnetic resonance imaging (fMRI) experiments will be conducted at the Department of Psychiatry, Psychotherapy and Psychosomatics in the University Hospital as well as in the Jara-Brain consortium. These studies will be designed to disentangle cognitive and neural systems differentially involved in processing gestural stimuli embodying Peirce's categories. Finally, multimodal manifestations of the universal categories will be related to psychopathological dimensions. The assessment in clinical biographical interviews will elicit gesture production, allowing for a systematic association with psychological dimensions such as affect, memory, and social cognition. The aim is the development of motion capture stimuli for clinical research and therapies, e.g., with applications for schizophrenia, depression, and autism, as well as for behavioral studies in cognitive psychology. Advancing the integration of semiotic theory and psychopathology, it will be of great interest to disciplines building on Peirce's ideas: e.g., Mathematics, Biology, Linguistics, Semiotics, Informatics, and Cognitive Science. Improved analysis of the different levels of communication will promote new interaction technologies, especially in the expanding domains of new media and gesture-based interfaces in human-computer interaction.
Integrating this new subproject with the ongoing research activities, the Natural Media Group will be able to further enhance its gesture laboratory and substantially contribute to the growing field of interdisciplinary gesture research as well as to the domains of COMMUNICATION, HEALTH and TECHNOLOGY - as stated in the excellence initiative's institutional strategy of RWTH Aachen University. Through an increased cooperation between the humanities, neuroscience and technology, it will help solidify the growing infrastructure for interdisciplinary research at RWTH Aachen, linking expertise and resources at HumTec, the University Hospital, JARA, the Jülich-Aachen Research Alliance; Translational Brain Research, and the Fraunhofer Institute for Information Technology (FIT). The research will be carried out in a team consisting of doctoral students and Natural Media faculty.