Introduction

so we've been talking tonight a lot about how to use machines and so i would like to take the final slot tonight i know that we're standing between you guys and pizza um but i wanted to take the final slot tonight to kind of flip things on its head and actually talk a little bit about how machines and developments in ai can actually teach us more about ourselves as humans and how our brains work

Meet the Speakers

So I'm Eleanor. I'm a tech journalist and also a writer, and I've brought up a very special guest tonight with me, someone who's been exploring this very question through many angles, art, memory, and we're going to touch on all of those tonight.

Holly, do you want to introduce yourself? Thank you so much for having me. So my name is Olivia Henoff. I'm a research scientist at Google DeepMind.

The Intersection of AI and Neuroscience

At Google, I mainly build AI systems that try to perceive the world around them, try to remember events that can be happening far in the past. systems that can assist us in our everyday lives. But I'm actually coming at this from a neuroscience background. So my PhD was in neuroscience, where I was actually investigating how human brains actually enable us to do all these same things.

And I will say that that transition from neuroscience to AI is a pretty classic one. People oftentimes figure out something interesting about the brain and then kind of try to build better machines. as a result. But I think what I'm really excited about these days is what it would look like to go the other direction, as Eleanor was saying. So, you know, how can we use this kind of tremendous progress that we're seeing in AI to actually kind of shed light on some of the foundations of our own personal experience?

AI's Lessons on Human Perception

So what has AI and developments in AI thus far managed to teach us about human perception? Where are we right now? Yeah, so I think maybe to get at that question, I might kind of turn the clock back a little bit, maybe even all the way back to the 19th century, and kind of trying to see, okay, well, what did kind of, you know, the old school AI, you know, the kind of simplest version of AI that you could imagine, tell us about human perception?

And I think that goes back to I think it's like right around the turn of the century, maybe 1805 or something like that, when Jung and Helmholtz actually make an incredible discovery, which is that human vision actually only perceives three colors. And this is despite the fact that, you know, the light that's coming at us through our eyes actually has many more colors than that. It's extremely rich.

And yet when you ask humans to kind of match, for example, light that's coming at them with, you know, three little knobs, it turns out that it perfectly recapitulates their experience. And that really simple realization actually has profound implications for then rendering visual experiences. Because that means that actually a color TV only needs to create these three simple colors and in doing so will kind of generate a visual experience which is indistinguishable actually from the one that we would have had had we actually been there in that scene. And so that 19th century AI is actually, I think, fairly indicative of what current AI can actually tell us about visual perception today and actually what that means for our everyday experience.

So of course, these models have made a little bit of progress in the meantime. We're not just explaining color vision, we can also explain the ways in which We perceive objects and people, their relations, their interactions. But really, these things are going to be similarly used actually to kind of generate visual experiences. So oftentimes, you might be able to say like, okay, well, I can ask the model to give me an image, for example, that's going to have certain properties. And in doing so, then I'll actually create a visual experience that will be very similar for the machine as well.

Artistic Exploration of AI and Perception

So I know you've also looked and come at this question from the angle of art and how art can kind of shed light on this. What do you see as the connection between arts and this question of AI and human perception? So I think it's a really good kind of application of sorts, actually, because it's kind of putting these tools in the hands of extremely creative people, and they can really kind of tell us just, you know, how much can be done with this technology.

So actually, maybe if you want to pull up the next slide, I think we can have like a kind of nice little example. Yeah, so this I think is actually a really interesting paper that was kind of put out by Daniel Gang et al fairly recently. And it's kind of telling us, you know, it's kind of giving us a little glimpse actually of what kind of AI can tell us about the human mind. In that here we're asking this model to generate various visual illusions.

So it's kind of, we're asking it, okay, give me an image that looks like, for example, Albert Einstein. under one particular view, but also Marilyn Monroe under a different view. And that simple prompt is going to be sufficient to then let the AI kind of go in and mine the space of all possible images and then find relevant matches to that. And so this is still in the realm of visual illusions.

This isn't a feature in a film, although we're very much, we're fast making progress in that direction. But I think it kind of gives a flavor of the sort of generative experiences that we might start to see. a single sort of handcrafted video, for example, we can actually have a sort of visual experience or in fact, you know, any kind of media experience that would really be optimized and kind of fine-tuned with a particular experience in mind.

Decoding Brain Activity Through AI

And maybe we can go to the next one and talk a little bit about the weak exhibit as well. I think that's a good example too. Yes, so yeah, I'm a really big fan of this exhibit because I think it also fast-forwards into higher level, this time not just visual functions but also cognitive functions.

So I'll just maybe give a little bit of context. So in this case, the artists and their collaborators are using an artificial intelligence in order to decode brain activity. So you have someone who's in a scanner, from that scanner, you can measure actually the various patterns of activity inside their brain.

And then we feed those patterns of activity and we kind of ask the AI to reconstruct what they were seeing when they were in the scanner. And then we can see, okay, well to what extent is this visual imagery representative of what they were seeing? And what I find so fascinating about it is that these reconstructions are fairly approximate, very imperfect in a lot of different ways, but I think they actually reveal something fairly profound about how we perceive and remember the environment also.

So oftentimes, we can even see that introspectively. If we think about our memories of the past, remember in exquisite detail actually, some very specific elements and then other ones will actually be very blurry, will have kind of compressed a lot of that information down to just maybe a few bits and that's kind of reflected in this visual piece actually where you have some bits that are stable and other ones that are kind of flickering in and out.

And I love how he also talks about, the artist talks about in the video, about how you can be creative just by thinking almost, right? That's so crazy. Love that.

Practical Applications and Ethical Considerations

So beyond, okay, great that we can explore human perception through art and AI, which can give us some interesting exhibitions and interesting experiences, but is there any actual kind of like practical applications for this beyond just let's go to the Serpentine Gallery and look at a nice exhibition? Yeah, yeah, exactly. If this were only relevant for the goers of this gallery, it would be a little unfortunate. So yeah, I think there's actually a lot more potential at stake here.

AI in Understanding Human Emotions

And I think this is particularly the case when we start moving beyond just simple visual functions and actually going up to these higher-level cognitive functions, like, for example, actually building mathematical models of our memory of the past. And through that, we might actually get at the very foundations of our emotional lives, actually. So if we have a very precise quantitative description of the associations that we're forming between the present and past experiences, then we'll probably have made a lot of progress actually in understanding our reaction to a kind of new event, because maybe we'll, current experience will elicit memories of past experiences.

Those past experiences will have a particular valence. It might be strongly positive or negative. And it's basically through the associations that we've made between the past and the present that we'll then have a very specific emotional reaction to what's going on right now.

Ethical Implications of AI's Influence on Emotions

And so that's really fascinating almost in itself, but actually has very deep implications, like you were saying, for therapeutic practice, actually, because oftentimes therapy is going to get at those associations and kind of try to allow you to revisit them, maybe rework them, kind of turn them into a positive light. And so you could imagine, actually, if you play this tape forward a little bit, if you have these very strong quantitative descriptions of these associations that we're making, and then you have someone, for example, that's suffering from some sort of traumatic experience, maybe they have symptoms of PTSD or things like that, well, maybe this model actually tells you, okay, well, how would you design a treatment plan? In the same way that we were designing generative experiences to make people have certain emotions, here you could have a sort of generative or kind of fully designed, automated, treatment that would actually allow you to intervene on those associations and hopefully free them from some of these associations.

But then I guess the opposite can also be true, right? Because if you're, you know, acting on human emotions and you're influencing how we can feel things with such precision, right, that obviously can be used for not so nice purposes. Yeah, so yeah, I'm sure a lot of people might be kind of worried about that these days, actually, when they see all these amazing generative media that are coming out. And particularly, if you then combine that with some of the scandals that we've seen, for example, with Cambridge Analytica that was kind of finding that actually, given enough information, then you can actually start targeting these ads very, very specifically.

And, in fact, the potency of these media starts to become a little troublesome. You wonder, like, okay, well, how sound is our democracy, actually, if we can actually kind of target these ads, you know, push people on a particular end of the political spectrum or things like that. I think on that note, I think the answer isn't really so much technical as it is societal, actually. And I think we see examples of this in the past.

Societal Impact and the Future of Information Privacy

Historically like for example around the the Industrial Revolution where you there too you have this huge technological onset and then you have all sorts of societal norms that kind of follow suit so you know that's it's only after the advent of you know factories and electricity and things like that that we actually start to see a lot of progress in you know workers rights and things like that and And my sense is that we're going to see, you know, something similar actually. So, you know, these sort of generative, you know, media or, you know, these extremely targeted ads are really only so potent because of all of the information that we're shedding about ourselves when we go and interact online, interact on social media and things like that. And so my sense is that we're going to start to be, you know, a lot more mature with respect to this technology and maybe a lot more mature with respect to, you know, how we treat our information and actually kind of sort of treat it with the same sort of sanctity as we do kind of our personal physical selves and maybe if we think of our informational selves we might kind of start taking more care and the hope is that then that will kind of allow us to mitigate these nefarious applications and just benefit from the good ones.

AI's Role in Enhancing Human Connection

I think kind of coming back to the ways in which this can push us forward as humans and as humanity as well, I wanted to come to the next slide. That we were talking about how actually some of these experiences that can help us learn more about human perception can also help us in this world that we're living in that's so polarized, right? Where we're being kind of attacked by fake news and things that are on the internet. It can actually kind of feel oneness and it can be a way to access that feeling of oneness with not just other humans, but with other species and with nature and kind of build that rapport to help us go to the next level of society.

Yeah, absolutely. So I think, yeah, maybe if I could just comment on the piece for a second. I mean, first off, so yeah, for all of you machine learning practitioners in the room, you'll kind of recognize some really cool latent space traversals using some kind of neat generative model. I think the other kind of maybe kind of passing point that everyone will have appreciated is that

you know in six years this technology has really progressed by by leaps and bounds um but i think it really gets at what you're what you're describing actually that kind of in being able to kind of interpolate actually between these you know seemingly very very different visual shapes you know maybe even different you know species of plants um in the rest of the exhibit they'll also kind of start interpolating between you know different animals for example you kind of seamlessly go from you know from a bear to a koala and so on and it kind of tells you or it kind of maybe gives you this kind of very deep sort of visceral sense actually that these things that we think of as kind of very distinct, actually, are maybe just kind of slight parameter changes away from each other, actually. And these might be kind of differences in degree, but that are kind of all governed by the same overarching mechanism. And I think that maybe kind of gets back to this point about these sort of mathematical models of our personal experience, actually.

And what is the value in that? We're saying that, OK, well, there are various interventions. We can build products about that around them that are going to be very exciting. They're probably going to be therapeutic practices that might be kind of more data driven in this way.

But I think there's almost something kind of fairly profound and philosophical about it if we say that like, well, actually, our personal experience, you know, as as people, you know, as rich and exciting as it is actually might be governed by these kind of very simple kind of underlying principles, actually. And those principles might be responsible for a lot of our reactions, which are, you know, of course, going to be different especially when we're talking about, you know, maybe fairly divisive subjects, like you said. But, you know, the mechanics behind those responses might be actually fairly common, you know, amongst us.

And maybe if we kind of get at those foundations, then it will be a little bit easier, for example, to see where the other person is coming from or, you know, understand that, okay, maybe they have this very different reaction, but I can kind of understand the mechanics at play and then kind of better understand where they're coming from. I kind of get the sense that we're talking about, you know, when astronauts go up to space and they see the world from space, they talk a lot about feeling for the first time that like, oh, wow, like my problems are really not worth anything, you know? And having that perspective, it's a little bit like that aha moment where you need that distance to feel, yeah, almost like a connection to something that's bigger than us, that's bigger than humans. Yeah.

The Awe-Inspiring Aspect of AI Research

Have you had that experience in your work? Yeah, absolutely. So I think, I mean, I think that feeling of awe actually is kind of what's driven, I'd say, I mean, all of my, you know, research practice.

So I think it even goes back to just this, yeah, maybe even this one kind of physics class, actually, I think in high school where I still remember, you know, there's a description of a BMX biker who's going down the hill. And, you know, even though it's just this like, you know, extremely intense kind of visceral experience, there are actually these really simple, kind of clean and elegant mathematical laws that are actually governing, you know, the entire experience. And so now, you know, and that's just classical mechanics, you know, let alone, you know, intelligence that we're talking about.

And so I think I kind of regularly have that impression, especially in building these systems and knowing just like just how complex it is to, you know, build AI that can you know, accurately perceive the scene, accurately remember past events, you know, that will understand, you know, what's relevant, what to pay attention to, and things like that. And then you see, you know, all of this stuff playing out in your everyday experience. You know, all you need to do is, I don't know, get on your bike, for example, and cycle down the street.

And, you know, just in order to do something that simple, your brain is kind of affecting this, you know, incredible magic under the hood, actually, that's kind of deploying these extremely sophisticated computations, you know, understanding, you know, who's going where, oh, do I have time to, you know, past this person or in a break in front of this pedestrian and things like that and so it just feels like you know as we as we kind of elucidate actually the foundations of our our kind of you know mental lives we can actually just kind of witness the the kind of extreme complexity and beauty actually that's going on every day of our lives so what are next steps where do we need to go from here to to delve even deeper into how we perceive the world as humans

The Future of AI: Challenges and Opportunities

Yeah, so I think actually today we have an amazing opportunity, actually, because we are witnessing this sort of, I think, paradigm shift in this kind of technological progress. And so I think we sort of, I think we're faced with two dual challenges, actually, two things that you've maybe touched on already. I mean, the first one, I think, is really being kind of I think taking a sober look on how these technologies are going to be applied and actually kind of preemptively mitigating any kind of risks.

And I think for that, and I think we really need to just pay attention to the people that are going to be using these technologies and understand, okay, well, what is their experience going to be like when they use this particular thing? How might they be affected? How might their beliefs be changed? What are the social dynamics that are going to emerge from using this technology?

I think we're all extremely excited and I think for good reason about the technological progress, but I think really kind of focusing our research also on the humans who are going to be interacting with this, I think is really exciting. And then I think maybe on the more positive side also, I think we can say like, okay, well, how do we maximally leverage actually this technology for human fulfillment and human joy. And I think we're kind of seeing this, you know, amazing kind of leap forward on the technological side.

Do we have time for one question, Josh? Anyone want to ask a question? We've got one over there. Hand up over there.

Questions from the Audience

How are these systems Yeah, yeah, totally. So I think it's a more general point about self-play, really. And I think we've seen a lot of really interesting applications of self-play.

So it's one thing to kind of think of all the different corner cases, for example, all the different failure modes of an AI system generally, and LLMs in particular. And so you can try to use sort of human evaluators in order to kind of plug all those gaps. But actually using AIs themselves actually as sort of adversarial example generators is actually extremely potent because you will actually ask the AI to kind of find the weaknesses of the other one. And then in doing so, you kind of automatically generate these failure modes, ask the main AI to kind of plug those gaps and so on and so forth.

And that kind of ideally virtuous cycle actually of like self-improvement has been deployed to great effects, like for example, in the AlphaGo system. One more question.

Oh. Yeah. Yeah, yeah, absolutely. No, I think it's a fascinating question that I think is a very, very hot topic right now.

So I think the question is about unified architectures ultimately. So, you know, we know that the brain has various functions that are assigned to different parts of the brain. For example, you'll have visual cortex, auditory cortex and things like that. But the question is, you know, is are they all kind of using the same unified architecture or actually have they been specialized to varying degrees, you know, let's say through evolution and things like that? and you can sort of see the parallel trend in AI actually.

So basically, ever since the advent of the transformer, we've actually seen ever more unification. So we had this architecture be developed in natural language processing, and then it basically turned out that it could be deployed across use cases. And it's just by feeding it different types of data that you actually get the sort of functional specification that you were describing. So I think the kind of leading hypothesis is that that's actually consistent with what's happening in the brain also.

So cortex, for example, is just this hugely malleable kind of mass of neurons. And you see it, for example, with people that have very different experiences. Like, for example, maybe someone who was blind at birth, for example, the entire part of the brain that would normally be used for visual processing actually then gets reconverted for auditory processing. And sure enough, then they have these like exquisite senses of hearing and things like that.

So I think we're actually, if anything, seeing like a larger alignment actually between biological and artificial intelligence. Okay, thanks.

Exploring the Sense of Touch in AI

We're seeing lots of progress in text generation, audio, video. How, like where does touch... factor into all of this? And what are the opportunities? What are the questions and challenges that maybe need to be overcome?

And how are you thinking about that potentially? Yeah, I mean, I think, yeah, touch is a really fascinating one. Actually, it kind of goes back to the previous question in that, you know, that typically is processed on a much more visceral level. Actually, it's kind of, you know, far from the sort of cognitive experience that we have a vision or audition or things like that. I think typically touch has lagged behind just because of technological limitations.

So it's a lot harder to collect, you know, large touch data sets, for example, than it is to just scrape the web of data. of images and then obviously training models on them also. My sense is that there are similar opportunities associated to them. So I think if we were to solve that technological challenge, then in the same way that we've kind of had this unified treatment of all other modalities, we'll probably see very rapid progress for touch also. And there too, maybe coupled with 3D printing, for example, you could imagine fully tailored sort of tactile experiences as well.

Okay, so you're both going to be around for questions afterwards. Well, a round of applause to both Eleanor and Olivier.