- Information embedded in fine-scale cortical topographies
Our research models information embedded in cortical responses and connectivity as high-dimensional information space. Cortical patterns of activity and connectivity have a fine-grained structure that is idiosyncratic, requiring computational models to decode the information and investigate how that information is shared and how it differs across brains.
- Neural decoding using multivariate pattern classification (MVPC)
Patterns of cortical activity reflect population responses, filtered to the spatial granularity of fMRI, that encode information, such as percepts, memories, thoughts, knowledge, and emotions. This information can be decoded with machine learning pattern classifiers (Haxby et al. 2001, 2014), such as nearest-neighbor classifiers, linear discriminants, and support vector machines.
- Modeling information as representational and connectivity geometries
A cortical field can be modeled as a high-dimensional information space. Patterns of activity and connectivity encode related information, and the relationships among those patterns create a geometry. Whereas decoding distinguishes the information content of patterns, information geometry models their inter-relationships, thus providing a new window on the information content and function of a cortical field. Cortical fields are linked to form processing systems or pathways, and information processing enabled by communication across fields in a system can be modeled as changes in information geometry (Connolly et al. 2012, 2015; Guntupalli et al. 2016, 2017; Visconti di Oleggio Castello et al. 2017, 2021).
- Modeling shared information with hyperalignment
The fine-grained patterns that encode information are idiosyncratic. Therefore, MVPC between subjects yields very low accuracies because anatomical alignment cannot resolve these fine-grained idiosyncrasies. We created hyperalignment to resolve these idiosyncrasies (Haxby et al. 2011, 2020).. Hyperalignment aligns the information content in cortical fields by projecting idiosyncratic information spaces based on misaligned cortical vertices into a high-dimensional model information space. Hyperalignment affords between-subject MVPC that can be more accurate than within-subject MVPC by virtue of a larger multisubject dataset for training the classifier. Surprisingly, hyperalignment also increases intersubject correlation of information geometries because it resamples vertices into and out of a cortical field to increase cross-subject consistency of information content. We have developed hyperalignment algorithms based on cortical response patterns (response hyperalignment, RHA, Haxby et al. 2011; Guntupalli et al. 2016 ), on cortical connectivity patterns (connectivity hyperalignment, CHA, Guntupalli et al. 2018), on both response and connectivity patterns (hybrid hyperalignment, H2A, Busch et al. 2021) and an algorithm that allows some warping of information geometry (warp hyperalignment, WHA, Feilong et al. 2023).
- Estimating functional topographies using hyperalignment
The individual transformation matrices produced by hyperalignment contain information about how shared information is idiosyncratically distributed in an individual brain. A response or topographic pattern in the model space can be projected into an individual’s cortical anatomy using the transform of that individual’s transformation matrix. We have shown that we can estimated known functional topographies in individual brains using functional localizer data from other brains and a transformation matrix based on movie-viewing data. These estimates capture individual topographies for visual category selectivity and retinotopy with high fidelity matching the precision of estimates based on a subject’s own localizer data. Using CHA, we can estimate topographies based transformation matrices calculated across different movie-viewing datasets (Haxby et al. 2011; Guntupalli et al. 2016; Jiahui et al. 2020, 2022)
- Modeling individual differences
We investigate individual differences in cortical functional architecture at the level of fine-grained topographies. Because shared information is embedded in idiosyncratic topographies, these idiosyncrascies must be resolved with hyperalignment to reveal differences in information content (Feilong et al. 2018, 2021). We have developed a new algorithm, the Individual Neural Tuning model, that calculates transformation matrices with WHA, which allows warping of information geometry. Consequently, information about both individual differences in topographic organization and individual differences in information geometry is contained in the transformation matrices, and the model matrix reflects stimulus information that is mostly stripped of individual variation (Feilong et al. 2023).
- Visual neuroscience and person perception
Our research has focused on visual neuroscience with a particular interest in high level vision for faces and actions and the representation of person knowledge. (Haxby et al. 1994, 2000, 2020; Gobbini & Haxby, 2007; Nastase et al. 2017; Visconti di Oleggio Castello et al. 2021)
- Naturalistic stimuli and open neuroscience
We use naturalistic stimuli (mainly movies) to collect fMRI data that serves as the basis for hyperalignment and modeling individual differences. Naturalistic stimuli sample stimulus space much more broadly than controlled stimuli, and engage many cognitive systems in parallel for visual and auditory perception, language, social cognition and story understanding, thus providing a richer basis for modeling the functional architecture of the brain.
We are longstanding, committed advocates of open neuroscience for sharing of data, stimuli, and software. We work closely with Yarik Hanchenko, creator of the DataLad platform for data sharing, director of the Center for Open Neuroscience, and, incidentally, an alumnus of the lab.