Using a technology called a Brain Interaction Transformer (BIT), the new Brain-IT, presented by academics, can read a person's brain waves and interpret the signal as a visual image using A.I.
What the computer does, in simple terms, is scan the 3D pixels – voxels – of the brain using fMRI, and interpret this data to predict the image.
This is a technology which has been worked on for some time, however, and scientists have struggled for a number of reasons. Unlike humans, an AI needs a large amount of data to learn from.
People also interpret data through both the factual information (what color is it?) and the semantic. In fact the semantic information is used as a kind of key to lock onto in the brain, before the rough layout of the image is translated using a 'diffusion process'.
The process is not the first – others such as MindTurner, MindEye2, and NeuroVLA have all offered alternatives, but none have been as accurate.
This is especially true if the system has only had an hour of training. The usual goal is 40 hours.
This study, led by Roman Beliy of the Weizmann Institute of Science, built on other academic works as far back as 2013, when it was identified that different aspects of the cognition of objects took place in connected but different areas of the brain.
The testing was performed using the Natural Scenes Dataset (NSD), which is one of the few datasets available. It is made up of 73,000 pairs of images (one real, one an fMRI scan made while the subject was looking at it). There are only around 1,000 images that were shown to all 8 test subjects; many of the images (around 9,000 each) couldn't be compared.
The upshot of this research, though, is that the AI was trained to predict what it would see on the fMRI scan from a photographic image as well as interpret an fMRI image of the brain to understand what might be seen.
While the final results are still not perfect, they are far closer to the original image than previous models, incorporating additional details.
From a photography perspective, it seems perfectly possible to read an image from the brain using this kind of technology.
Accuracy wouldn't be guaranteed by optical paths in the same way that true photography is, but it seems very plausible that a version of your memory could simply be scanned from your brain and printed out.
And if that can happen, why bother with a camera at all? It might be more accurate, but you'll probably prefer your memory!
The technology is also just looking through the skin and into the brain to see what pictures are floating around in your head. It could, for example, be worn while asleep (or worse, put onto someone while asleep) to capture the images from dreams. That has more than a few privacy concerns!
It's also been suggested to me that OCD sufferers might not appreciate having their visual thoughts brought out and shared.
But this is a technology we might have to face up to and there are clearly far broader implications than helping some people with very specific forms of blindness regain some form of sight.
The paper was published earlier this year but work continues. At the time only about a couple of seconds of data was collected for each sample – but one of the scientists, Michal Irani, even said that while decoding video was challenging because the images change so frequently. Nonetheless he suggested recording dreams was a future possibility.
