Research currently being validated highlights an original project: reconstructing quality videos from brain imaging data based on artificial intelligence. “MinD-Video” allows us to better understand our cognitive process.
You will also be interested
[EN VIDÉO] An artificial intelligence that adjusts the angle at which the photo is taken Interactive point-based manipulation of the Generative Image Collector. © DragAN Project
In order to better understand our cognitive mechanisms, researchers are trying to reconstruct human vision from the activities of our brain. Although recent research has already had great success in reconstructing static images from noninvasive brain recordings, there has been a lack of work on recovering continuous visual experiences as videos.
The “MinD-Video” project enables high-quality video reconstruction from brain recordings. The study undergoing peer review can be found on the pre-publication platform Arxiv. Based on AI, the new model gradually learns spatio-temporal information from videos, and this from brain signals (from continuous fMRI scanner data thanks to masked brain modeling).
A biologically plausible and interpretable model
The result offers video images very close to reality, showing once again the impressive capabilities of AI. ” Analysis of attention revealed correspondence with the visual cortex and higher cognitive networks, suggesting that our model is biologically plausible and interpretable. “, note the researchers.
Their analysis highlights in particular the essential role of the visual cortex in the processing of spatio-temporal visual information. Moreover, their encoder works in hierarchical layers: the initial layers focus on structural information, while the deeper layers are geared towards learning more abstract visual features.
