September 17, 2005

Extracting Video from the Brain Neurons can be decoded into a reconstructed image, claims Dr. Stanley, Associate Professor in the Division of Engineering and Applied Sciences at Harvard. In other words, you can "jack into" a brain and extract video.

This picture shows an example of a comparison between the actual and the reconstructed images from a cat's brain: The decoding algorithm is said to be simple since each point in space can be reconstructed at high resolutions from 6 to 8 pairs of cells. The more neurons are tracked, the higher the quality of the reconstruction. This is rather old, it's from 1999, here's the pdf, but it was news to me and couldn't find it in a search.

posted by Scary Corn Imp 18 years ago
  • Arrrgh- wish they'd stick to human brains. Say: this would make good movie material! I see Robin Williams as the lead, editing said videos.
  • That's pretty impressive.
  • The LGN: your brain's "video in" jack. What I want is not video, but audio, hooked to a recorder. I could then play back what I had once heard and thus settle any number of differences with my wife. Yes, ma'am, I heard you the first time.
  • I wonder if you could record dreams?
  • Ah, imagine being able to watch your own and other's dreams and experiences... the ultimate addiction. As well as the ultimate nightmare, surveillance and enforcement agencies-wise. At least when you're the one with the electrodes on.
  • Hang on, what if you could record someones death experience? Oh yeah.. there was that Walken movie about that..
  • It's only a bit more time till they jack in and "project" experiences instead of recording them.
  • I wonder what William Gibson thinks about this?
  • This reminds me of the film Until the End of the World (1991), in which an old man in Australia invents a device which allows people to record what they see. He begins to use it to record his dreams. His daughter (I think it was his daughter) becomes addicted, convinced that she will figure it all out if she can only interpret her dream.
  • As a musician, I would be really interested in the ability to extract audio from the brain. When I was young, I used to hear elaborate symphonic pieces in my head and I was very frustrated because I didn't have a clue as to how to notate them. As an adult, I can now notate them but because notation is so tedious and time consuming, the ideas don't stick long enough in memory to be recorded as accurately as they were originally conceived. This is a common frustration for many of us. For composers and improvisers, the ideal is (and I stress this with my students) has always been to remove as much of the "middleman" between imagination and performance, as things like technique (or more precisely, the lack of it) theoretical training (bad if one relies on it too much at the expense of pure invention) and/or "finger patterns" (i.e. cliches and learned "licks") etc tend to distort the pure idea that inspiration was kind enough to bestow upon you. When one composes, the original idea you started with usually gets morphed into something not quite as great as the original thought, (unless you're like Beethoven who started with fairly lame ideas but beat them mercilessly into shape until they became something much more transcendent.) I remember reading a science fiction story whose title and author I can't recall now, (Ellison?) where in the future people would come to hear concerts given by musical geniuses that had no need to learn instruments. They just had a bank of synthesizers (or whatever) jacked directly into their brain and their imagination "played" the music. Thereby the greatest artists were the ones with the freest imaginations. Given this advance with video, it seems like it is not so far fetched an idea after all. Bring on the future! (except for Peak Oil, Theocratic wars etc).
  • This reminds me of the film Until the End of the World (1991) Also The Lathe of Heaven, possibly Ursula K. Le Guin's finest novel, where a psychiatrist records his patient's dreams (well it was about a lot more than that, but the dream recording bit figured prominently.)
  • Those are some freaky pictures. Interesting post, I had no idea. Would like to know how this study has gone in the last six years.
  • And if my thought's dreams Could be seen They'd probably put my head In a guillotine -Bob Dylan
  • I remember the movie "Strange Days" with Angela Basset and Ralph Fiennes. Oh, I do definitely see a market r for snuff material, and first account experiences of soldiers and the like. Calderon de la Barca might also have something to say.
  • Also, Cold Lazarus.
  • Freeeeaaky. I wonder how they took the "real" pictures from the cat's perspective. Do they have a "kitty cam" mounted on the cat's head? (maybe I need to rtfa) And I really remember Until the End of the World because "Blood of Eden" by Peter Gabriel was on the soundtrack.
  • Trippy. But I hope it's not so much like Until the End of the World, because that flick sucked.
  • In a way, this is not surprising: I believe a 'map' of the activity of the retina is propagated quite a long way into the brain with only minor topological distortion. In effect, you're just using the eye as a camera. If you're doing it with electrodes, wouldn't the brain in question be somewhat minced by the large numbers needed for decent images (one electrode per pixel?). Since perception doesn't actually consist of us sitting in our heads watching a video feed from our eyes, the screen images would also. I presume, be no closer to our actual subjective experience than those of a conventional camera mounted on a pair of glasses, and wouldn't (I think) capture dreams or imagination.
  • With advances in miniturisation, the problem of mincing the brain won't be such an issue in future decades. We'll have molecular chips. In fact, no link, but I seem to remember reading an article a while back about just such a technology being developed; a mesh-like tiny circuit net that was suitable to directly interface with neural tissue. As far as the other points, I agree, but after all this here appears to be the equivalent of Edison's first "Watson, come here" phone message or whoever it was, or the first phonograph recording. The technology will be perfected whilst understanding of the brain's neural processing also advances. It is an inevitability that future human generations will become partly computer. It's just a matter of time.
  • "the problem of mincing the brain won't be such an issue in future decades" hehe One hopes.
  • Funny, the image from the cat's brain looks more like another cat than a human.
  • This technology has Freedom™ written all over it.
  • ...make it go!.........make it work!
  • goetter - as a person who has spent quite a bit of time poking around in the geniculate complex (mostly the intergeniculate leaflet, but still retinorecipient: ain't behavioral neuroscience great?) I have to say: Holy crap. Video reconstructed from a brain. Why have I not heard of this? Fish tick: Sad but true, cats have great visual systems and are a traditional model for any type of vision research. Guinea pigs are used for auditory research, pigs as human body stand-ins, and monkeys generally for human brain stand-ins. Me, I stuck with rats. Working on cats isn't really my thing. The Kitten Room in the animal colony building though - hot damn! A whole room full of kittens - what could be more fun? (Used for spinal research, not vision: About half the kittens were born with a degenerative disease that caused paralysis of the hind limbs later in life; the research aimed to find the cause and a cure for this progressive disease, and the healthy kittens are generally adopted out.) Give this a few more years and we'll have live feeds: "Caution: This store monitored by security cats!" (Not that cats would make great security cameras - 90% of each recoded video would be an image of the inside of a cat's eyelids...)