Methods

Cognitive modeling is a central tool in all projects in our lab. It involves building a mathematical model of the cognitive processes that lead to an observable behavior on a particular task. Once defined, such cognitive models can be a very useful tool for model simulations, fitting behavioral data and generating predictions for new observations. In addition, cognitive models define specific internal variable for describing the dynamics of cognitive operations and these variables can be used in model-based fMRI (LINK TO fMRI below) and EEG (LINK TO EEG hyperscanning below) analysis to detect a neural signature of the underlying computations. These internal variables (e.g. expected values, prediction error, beliefs etc.) generate much more specific predictions for neural activity than simple experimental contrast and can therefore constrain models to large degree and provide stronger evidence for a neural basis of that cognitive model.

Cognitive Modeling Academy Hamburg

We offer an annual introductory workshop that provides step-by-step instructions for conducting a cognitive modeling study and introductions to two major cognitive modeling frameworks: reinforcement learning theory and sequential sampling models.

Deep learning has revolutionized the field of Artificial Intelligence (AI) and the advances in computational resources to solve deep learning problems have promoted their deployments in many different scientific and economic scenarios. One particularly interesting aspect from a neuroscientific perspective are the deep analogies between the mechanisms by which a deep learning network and the brain operate. Representational Similarity Analyses provide an analysis tool by which similarities between layer activations in a DL network and fMRI activation patterns can be directly compared. We have recently embarked on this avenue by highlighting the similarities between a supervised DL network for emotion recognition in audio-visual movies of facial expressions and vocalizations (LINK TO Research/crossmodal/) and brain activation patterns in participants exposed to the same emotional stimuli. These analyses confirmed the similarities of different arms of the DL network and the modality-specific activations in the brain.

Investigations of the neural signatures social interactions can be elegantly accomplished using EEG hyperscanning, which refers to the concurrent measurements in two interacting participants. It is thus possible to detect synchronization of activity between two different regions in a single brain, but also between two different brains. Such between-brain coherence is hypothesized to be a specific signature of social interaction that involves mentalizing about other people’s beliefs and intentions. The use of hyperscanning depends on the scientific question. Investigating the construction of mental models does not require hyperscanning as the processes are all taking place in a single brain and through the use of computational models of ToM we can detect specific variables that can uniquely signify a mental model of another person (LINK TO Research/IFBT). Furthermore, if the goal is to investigate the processing of another person’s actions (e.g. behavior-to-brain coupling), the other person’s actions simply become inputs to the brain and no hyperscanning is needed. However, emergent phenomena arising from synchronized inter-brain activity that could be grounded in concurrent mental activity (e.g. both players think about the respective other person’s strategies), then hyperscanning is necessarily required

In our lab, we use the BioSemi ActiveTwo system for our EEG hyperscanning projects (e.g. Intreactive False Belief Task and Interactive Tiger Task (LINK TO Research/IBFT, Research/Tiger Task, Research/TCG) in combination with a Cedrus StimTracker and 3D camera (Structure Sensor) for recording exact electrode position, which can then be combined with anatomical MRIs for the use of a forward head model that benefits between-subject registration.

fMRI has become the de facto standard method for cognitive neuroscience. It relies on the regional cerebral blood flow and measures the field distortions in a strong magnetic field that are induced by oxygenated and desoxygenated hemoglobin (the so-called BOLD signal, a measure of local energy consumption by firing neurons). Even though the neural activity is closely linked to the BOLD signal, the fact that the BOLD signal takes several seconds to rise renders a poor temporal resolution of fMRI measures. This is, however, compensated by a fairly good spatial resolution, which in recent years has become more and more important with the advent of multivariate analysis techniques (like MVPA and RSA) that exploits the information embedded in a regional pattern of BOLD activity.

More recently, we have begun (LINK TO Research/Representational Changes after PE and Belief Updating) to explore fast (and ultra-fast) fMRI sequences, which provides a more accurate measurement of the shape of the BOLD signal (although they do not overcome the problem of a low-pass filtered BOLD response. Nevertheless, a fast fMRI acquisition enables us to investigate fairly rapid representational changes in the course of learning.

The Institute for Systems Neuroscience manages and maintains a research-dedicated Siemens Prisma MRI, which is fully equipped for many different experimental stimulation (visual, auditory, olfactory, pain) with accessories to measure peripheral physiological responses *heart rate, respiration, skin conductance, eye-tracking) and even combined EEG/fMRI measurements.

During TMS stimulation a strong electromagnetic coil is placed on the scalp over a particular brain region. This stimulation temporarily (and reversibly) lowers the excitability of the underlying neural tissue (up to about 1.5 to 2 cm deep), thus effectively inactivating the brain regions. Participants then usually perform an experimental task and their performance is compared to that during the stimulation of a control region or to participants receiving a sham stimulation that feels identical to the real stimulation.

Compared to fMRI/EEG, the causal inference from TMS stimulation is much stronger. However, neural time series as in fMRi and EEG are usually not obtained. We have recently used TMS in a follow-up study in the Social Influence project , during which we stimulated bilateral TPJ to induce a change in representing social information observed in others and their effect on one’s own decision to switch or stay with the first choice.

In the lab we use a MagStim R30 amplifier to generate TMS pulses that are transmitted through coil onto the participant’s head. Neuronavigation, i.e. stimulating the target brain region based on an anatomical MRI) is achieved through the BrainSight software.

Eye-tracking can be used to obtain a moment-by-moment estimation of the participant’s attentional focus by using scanpath analyses on the fixation data from the eye-tracker. In addition, the pupillary dilation response (PDR) provides a measure of the state of arousal of the participant, which can be indicative of a significant event (e.g. an aversive US). We have used the latter in a project on second-order conditioning to detect arousal responses to stimuli that were in fact never directly paired with an aversive outcome. Furthermore, we analyzed PDRs responses in Tacit Communication Game (LINK TO Research/ToM/TCG) and found that the receiver of a communicative message produces and arousal response at the point of the message that indicates his goal state.