The Brain Is Just Specialized Agents Talking To Each Other — Dr. Jeff Beck

[00:00:00] Speaker 1: geometric deep learning is a big part of like is a big part of the stack if for no other reason than when we talk about like modeling the physical world that means like incorporating the symmetries that exist in the physical world it's like we're highly motivated to employ a lot of those methods [00:00:14] Speaker 2: and techniques but is the world written in code or do you mean exploiting the regularities in the code that seem to have some exploiting the regularities no it's like look we it things [00:00:24] Speaker 1: are it is the world is translation invariant the world is like rotation well not really because there's gravity but like in principle you know there is a principle axis but it's certainly rotationally invariant in the xy plane yeah um and if you if you want to have a good model of the world as it actually is it should incorporate those features of course you can discover it you know in a brute forcey way but the mathematician in me really wants to build build the symmetries in and fortunately we've got a lot of great tools that were developed over the last several years that can do that what's your view on agency if i'm being you know like an fep purist i have to sort of say like oh well there's no difference between an you know an agent and an object in in a very real way or at least there's nothing structurally distinct between what how we model an agent and how we model an object it's really just a question of of degrees right and agent is is a really sophisticated object right it has internal states that represent things over very long time scales um you know uh it has uh sophisticated policies that are context dependent which is basically saying really long time scales [00:01:32] Speaker 2: again um and things like that yeah you know um there's the kind of the philosophical highbrow notion of agency that we introduce notions of um intentionality and self-causation and things like that i mean the really no-nonsense version of an agency is it's just it's just a thing which acts and performs some kind of computation and i guess you could almost model anything as an agent you know yeah well so if if if [00:02:02] Speaker 1: if your definition of an agent is something that executes a policy then anything is an agent right a rock is an agent right every everything has you know it's an input a policy is an input output relationship when many people talk about agents they they're adding a few they're adding um a few additional elements that i think have a lot to do with how the policy is computed right so for example when we think of how the difference between like us and like like really like amoebas we we often cite things like planning counterfactual reasoning goal oriented behavior right we're specifying things that that um have that that are specific mean that that are all related to how it is we compute our policies right they're latent variables that represent policies um that are uh you know that are compatible with like well reinforcement learning right and um and that's the defining characteristic of an agent but you could very easily just sort of say like from an outside perspective if you can't look at how someone or something is doing the computations if the only thing you observe is the policy right does that mean that you can never conclude that something's an agent and i would say no right you'd still like to be able to conclude that this is an agent even though the only thing i ever get to measure is its policy but do you think we should have some notion of the strength of an agent the strength of an agent or how is this like a measure of agency is that what you mean yes or yeah so i mean i think you could use like notions of like transfer entropy and things like that in order to estimate like the timetable over which something is incorporating information or the degree to which it's taken into it it exhibits a context dependent behavior and things like that and that would be a pretty good measure now is it normative no it's not it's it's a but it is a measure and you could use things like that but at that point you're really just talking again about policy sophistication right not does it have a reward [00:04:03] Speaker 2: function like is it actually executing planning yeah i mean certainly intuitively agents to me seem to be kind of causally disconnected because they're planning into the future they are not impulse response machines they're not just you know part of the mass of things going on around them they are just [00:04:22] Speaker 1: obviously disconnected from the locality so here the trick is is that okay so i've got this agent and i know exactly what it does right it takes and it takes into account information um it rolls out future you know internally it rolls out a whole bunch of like future uh consequences of of various different actions or plans that it could take it selects the best one and then it executes it right so all of those variables all of those variables that were that occurred inside right from the outside perspective it just looked like a function transformation right it's i don't unless i unless i'm somehow going in and recording and somehow demonstrating the fact that the manner in which it is calculating its policy you know like involved doing those rollouts right i wouldn't be able to show that it's actually doing those rollouts i would just be able to conclude it has a really sophisticated policy so can you conclude that something isn't is is so so the question is how do you identify something that is actually doing planning i think that's a really hard question as opposed to having an incredibly [00:05:23] Speaker 2: sophisticated policy i think my intuition is if it feels to me that a function a simple input output mapping can't be an agent and and in a way this is related to what we were talking about with grounding you know it seems that when things are physically embedded in the world then they're more likely to be agents this functionalist idea that's just a bit of computer code running on a machine it kind of feels like that can't be an agent it does so suppose i coded it up so it was doing all [00:05:52] Speaker 1: of that planning it's like gets its input so some crazy like massive monte carlo tree search picks the best policy possible and then executes it now you don't observe any of that right because you know what's going on you could say oh well it's it's clearly like executing you know this is it's doing planning and counterfactual reasoning it's going on like look there it is because you coded it so you know it's doing it but if you're looking at it from the outside right it you know if you don't know what's happening inside it's going you know all you have access to is oh here's the action that it that it that it did given this long series of inputs and so it's it's really hard to identify what you you know something as an agent per se from the outside you kind of have to know what's going on inside this by the way is why i don't think that like you know can you know these sort of prediction based approaches to like ai um are necessary you know you could sort of say well it's not really doing anything even remotely agentic unless it's executing and doing planning and counterfactual reasons so like your chess program is is like oh clearly it's doing some planning and counterfactual reasoning because you know it's doing it but um but it but you couldn't like right yeah i could describe the [00:07:07] Speaker 2: exact same set of behaviors just with the policy function i i think the counterfactual thing is is an important feature here because we could take something which was conscious or something which had agency and we could just take a trace of the actual path which was found and now we've just got this a reductio and absurdum but you know now we now we've just got a computational trace and that thing clearly has now lost whatever agency or consciousness it had so there's something about considering all [00:07:32] Speaker 1: of the possibilities yeah yeah i think so in my mind that is the fundamental feature of of of an agent like if you can show that it's engaged in planning counterfactual reasoning and then it's definitely an agent my my argument is just simply that that's hard to do unless you crack it open and see what's going on inside now you could take a a pragmatic view and say well if the simplest computational model of the behavior model it as if it was doing planning and counterfactual reasoning then you can draw an implicit conclusion that oh yes well i may as well say it's an agent and that's kind of the approach that i've taken so like one of the things that comes out of the physics discovery algorithm is that you apply it to agents and what do you get well you get a model now bear in mind i called them all objects before and i didn't change anything to make it special to an actual agent right but what i do have the ability to do because of the model is i can look at the internal states associated with that object that i want to call an agent and look at how sophisticated it is right and that degree of sophistication is what allows me to say oh well i'm going to go ahead and say that like and i like the whole idea it's a great idea like let's have a metric right and i'm sure it would be something that would effectively be like transfer entropy or something like that but we have this metric on like well how sophisticated were the internal states that were necessary in order to generate this output and if it's above some threshold we'll call it an agent i don't like thresholds but you know we just sort of say a degree of agency a degree of sophistication and coming back to dennett's [00:09:01] Speaker 2: intentional stance so this is that you know there is um a level of representation which serves as a useful explanation even though it's not actually you know the the microscopic causal graph and maybe we can agree that no agent can possibly be the cause of its own actions but when there is a degree of planning sophistication for you know macroscopically it's as if it's the cause of its [00:09:24] Speaker 1: own actions yes and that's why this as if phrase comes up a lot right i mean this it's important to remember that like no matter how clever your model is and no matter how clever your approach is and how clever the words are that you use to describe it um a lot of this stuff is is is as if right this is this is the best model right it's not that it's not this is why like i i repeat this over and over again grind it into the students right is that that you know science is about like prediction and data compression and like nothing else and the same thing is going on here right you you'll never you know just looking at behavior you'll never know for sure in any meaningful way like whether or not it's it's just doing a function transformation or whether it's engaged in planning and counterfactual reasoning but if your best model of it if you sort of say well i tried to model as a function transformation but god damn it had a lot of parameters right but then i tried to model it as something that was just doing monte carlo tree search on the inside and giving the answer and that had like you know 40 parameters and it's like well that's the model i'm going to go with and now i'm going to call it an [00:10:27] Speaker 2: agent if we had a physical agent in the real world that was doing all of this planning and so on would that have some kind of primacy to the computer simulation of agents that were doing all of this [00:10:37] Speaker 1: planning oh is this is this like uh if i uploaded my brain onto a computer and didn't connect it to the world would it still be thinking even though it's like doing all of those things is that the idea [00:10:46] Speaker 2: here or am i like that works so yeah let's say a high fidelity computer simulation of jeff would [00:10:52] Speaker 1: would jeff be an agent no oh i wasn't expecting to say that because i'm the agent and if you uh uploaded no i don't know um so if you is do a high fidelity computer simulation and you put it in my body then i think i would have to say it's an agent yeah right if it's doing exactly the same kind i mean this is like the standard is doing exactly the same calculations from from a purely like phenomenal audio perspective it's like it's the same it's indistinguishable okay so agents need to be physical so i do believe that an agent needs to be physical absolutely i don't believe you know i believe you can have a model of agency and not have an agent right you know you can put that model in a computer and run it and make predictions as to what an agent would do you and it might even be 100 correct but i still wouldn't call it an agent but again this is like getting into philosophy and like philosophy frustrates the bayesian because philosophy is not probabilistic right philosophy is really about drawing clear lines and distinctions and in my world those don't really exist right there's everything has an error bar you know all of there isn't a clear delineation between you know uh you know an object and an agent it's really you know in from this modeling perspective it's really just a question of degrees and philosophy is terrible at handling questions of degree my friend keith he he's a big [00:12:14] Speaker 2: fan of um computability theory and and he thinks that an agent is basically you know like a type of computation and it has access to ambient state and it can take action and there's this kind of like cybernetic loop and for him the strength of the agency in the system is the compute type that the thing is doing right so if it's if it's a finite state automata then it's a weak agent if it's a turing machine it's a strong agent yeah it's the degree of sophistication of the compute pretty much yeah does that ring true [00:12:46] Speaker 1: to you i mean that if if you were gonna if you forced me like you know at the point of a gun to put a measure on agency it'd probably look a lot like that yes jeff let's talk about energy-based models [00:12:58] Speaker 2: sure so um yan lecun he had a monograph out i think in 2006 talking about this oh yeah talking [00:13:03] Speaker 1: about this for a long time oh yeah when you fit your neural network to data you know via gradient descent right then you have written an energy function in weight space and you are fall and you're following it to its energetic minimum you know the the advantage of using an energy-based uh taking an energy-based approach as opposed to taking say a straight up like function approximation approach is that an energy-based model comes with something that's kind of like an inductive prior right it it basically you know energy-based models you know if you're doing function approximation you're basically saying there's any mapping from x to y x is my inputs wise but any mapping is out there i just want to figure out what it is right now in an you know in an energy-based model right you're you're you're you're effectively placing constraints on what that input output relationship can be i like thinking about the distinction between an energy-based model and a traditional sort of feed forward neural network um has to do with where your cost function is applied right so in a in a traditional neural network you take in your inputs you got your outputs and the cost function is just a function of the inputs and the outputs and the only thing that you're optimizing is the weights in an energy-based model there's another thing that that your cost function operates on and that's something one of the internal states of your model and as a result like in order to figure out what the best you know the the best approach is right you actually have to do two minimizations one that that finds the energetic minimum associated with the the part of the cost function that operates on the internal states like the hidden nodes of your network right and then one that is the prediction that is your like effective prediction error um this is this is very much consistent with the approach that a bayesian would take right you have a you have a prior probability distribution which gives you an energy function over every single latent variable in your model and you are optimizing with respect to all of them so you take a probabilistic approach good examples of this are like a variational autoencoder a variational autoencoder i think is the is the best example of the most commonly used energy-based model out there why because you have an encoder network you have a decoder network right and your cost function is based on the difference between inputs and outputs right so that's just like a that's fine that's still a regular but it also is how how gaussian and it well it depends on what flavor of va but you also have some uh some some part of your cost function um is a function of the actual rep internal representation right in a traditional vae it's it's how gaussian is you want that internal representation to be as gaussian as possible um if it's a vq vae then it's like mixture of gaussians but it's still like a cost function that is applied on the internal states as well as on the inputs and outputs very cool [00:15:39] Speaker 2: so a vae is is a fairly canonical example of an energy-based model and what you were saying about the i mean you know the whole dl world is obsessed with test time inference at the moment and in a way that that is a step towards what you're talking about so yeah you're treating us yeah you're treating [00:15:53] Speaker 1: some of the weights of your model right i mean well yeah you're treating some of the weights of your model as if they're latent variables right because when you when you show a new input right you're allowed to change some of the weights without looking at the output right and so what are you doing well you're treating the weights as latents now i think that like which makes it a great trick in my opinion it's like oh great like yeah they're they're they're they're moving in the direction of energy-based models i love it the only thing i don't like about test time training is the vast majority of the training that is done so in a traditional energy-based model you always find the minimum with respect to the latent variables right these extra weights that you know which in this case which in the case of test time training is the you know the subset of weights that you're allowed to change during you know during test time um when you do the training for a traditional energy-based model you're allowed to make those changes right throughout the entire course of training the way that we're often doing test time training these days is we just do regular old neural network learning like we don't do and and then and then and then finally when it comes to when we get to the deployment phase then we suddenly turn on right these additional latents which are basically some of the weights of the network and we do additional an additional bit of learning at that point this seems monument now again not an expert here right but this seems unwise to me and the reason it seems unwise is because you didn't train the original network with that on right you trained it as in a completely supervised way yes now i'm sure that people are aware of this and it's been addressed in the literature but i'm not not personally aware of that i don't think that's how it's used in practice super we should also [00:17:30] Speaker 2: introduce this term transduction so my definition of transduction is that you're actually doing search or optimization as a function of the test samples like i interviewed clement bonnet he had a vae on on arc you know searching latent spaces and he actually um searched through the decoder as a function of the test sample yeah and because these models they are maximum likelihood estimators right which means they're always giving you a kind of smoothed out average and there's so much information in the test sample let's just riff on the relationship between energy-based models and and bayesian inference so of course they have this advantage that you don't need to do this very expensive intractable normalization [00:18:09] Speaker 1: yes yes tell me about that my take on it is is that an energy-based model and a bayesian model have a lot in common right in many ways like energy i mean well literally in physics right energy is like log probably energy is log probability now of course there's the normalization you know factor that you don't need to worry about if you're just doing if you're just minimizing energy and so the difference between uh you know like which is sort of like you know in a bayesian framework that's like saying well you know i'm not actually going to treat some of these latent variables in a probabilistic way i'm just going to do maximum or map estimation on some of my variables and just be okay with that and that's one way to interpret the relationship between an energy-based model and a properly bayesian model there's there's a happy medium here though right and the happy medium is you can still treat it as if it's you know you know you don't have to just minimize the energy function but you can calculate the curvature down there too do a laplace approximation and call yourself a bayesian again right yes there is more computation involved but we've got a lot of great tricks for making that totally tractable [00:19:12] Speaker 2: what's the relationship between the free energy in the free energy principle and the energy and [00:19:16] Speaker 1: energy-based models uh regularization term i think is the short answer right um no so so uh there's interesting uh and and if you're being very very very pedantic the difference between an energy-based you know minimizing energy and minimizing free energy is that free energy has this additional entropy penalty term now if you're just doing maximum likelihood estimation if you're minimizing your energy function with respect to some particular just we'll just pretend we're only one variable and i'm just going to like get a point estimate and call it a day do like you know some kind of map estimation to get to get that that one thing there's not that big of a difference right because you're you're not there is no probability distribution over the latent that allows you to compute that regularization term but that's the only difference it's it's are you regularizing or not is i think [00:20:05] Speaker 2: the easiest way to think about it so lacuna is a big advocate of jeppa so these joint embedding prediction architectures using this non-contrastive learning where essentially the the learning objective is is comparing the um the the latents of observed and unobserved parts of the space this is an [00:20:22] Speaker 1: architectural design well what is okay so what does jeffa stand for it's it is it it's joint embedding prediction architecture there we go so what's the joint embedding bit about well the joint embedding bit about is is you know is well i'm going to take my inputs i'm going to take my outputs and i'm going to embed them in some space right and then i'm going to learn a prediction between the two embeddings and that's a great idea it's a great idea because it has some of the flavor of what we would like to get out of our models like we're not interested in predicting every in many situations i should be very particular about this in many situations we're not interested in predicting every single pixel on the image we want to get you know maybe something that's a little more gestalt a little more high level a little more conceptual understanding of what's going on and so emphasizing the goal of predicting every single pixel which is what's typically done in generative modeling right now you know might lose some of the power the abstractive power of some of the networks and so like let's do it so so the whole point of jeppa as i understand it i'm sure there are other points um is that uh is that you're going to take you're going to you're going to compress your inputs and compress your outputs and then do all the learning in this compressed space love it right science is about prediction and data compression let's make that compression explicit on the front end and the back end the downside of this approach is that is it is it it doesn't work out of the box right because it's very easy to find a compression or an embedding of the inputs and an embedding of the outputs for which prediction is perfect which is to basically make both of them zero and so you have to do some other things other tricks [00:21:53] Speaker 2: need to be employed in order to make it work yes yes i remember lacune was talking about this so there was there's the the traditional contrastive method which is from it's kind of hinton's idea apparently yeah like the negative sampling and whatnot and and that's very expensive because you actually have to do lots and lots of sampling and this non non-contrastive thing yeah this is this by [00:22:13] Speaker 1: the way is what he should have won the nobel prize for right in my opinion yes because the the whole point of of of of the wake sleep algorithm and contrasted divergence was that oh it's actually biologically plausible right it was a it was it was an end run around the need to do back prop and [00:22:30] Speaker 2: that's what made it so clever and interesting in my opinion lacune is a big fan of this non-contrastive thing where you work in the the latent space there are many different algorithms that do this we had a whole load of shows all about non-contrastive learning there's things like v craig and byol and barlow twins and there's there's an entire thread of research all around that and in many different ways what they're trying to do is avoid this motor collapse problem that you're talking about and they use different forms of regularization there's an old school way of accomplishing the same thing [00:22:59] Speaker 1: and that is that is to to um do all of your is it's it's called pre-processing right and this is this is something that a lot of people do you take your data and in fact we do this all the time with with with like vision language models right so we want to do we want to use an lom and we want to predict images so what do we do well the first thing we have to do is tokenize the image right and so what do we do we run a va that we do the pre-processing and we do it by the pre-processing step is completely independent right from the actual algorithm that's going to be the be be tasked with solving the problem of interest um and you know that's not something that we necessarily have to stick with right it would be very nice if there was a way uh if there was a way of like again well jointly we're getting right back to jeff again what we'd like to do is we'd like to choose our pre-processing algorithm in a manner that that you know you know not a priori not do it first we'd like to choose the pre-processor that works the best in in this space and i think that that's the ultimate motivation for a lot of this work is it's like what's the right embedding one of my favorite tricks like of course i you know i pre-process the vas all the time in fact it's when you know the second every time someone hands me a new neural data set the first thing i do i'm you can i'm not ashamed to admit i run pca on and pass it through a vae and then sort of take a look right it's the first thing you do with your data because it gives you a good idea of what the signal to noise ratio is in the data set itself yes and then i yeah and then what do i do i subsequently do most of my analysis right in that discovered embedding space um and there's i i i don't see a huge problem with that from a purely pragmatic perspective but it it's certainly cleaner right to to have a single algorithm and approach and not just be stringing these sort of things together in an ad hoc way there's you know when when doing pca pca is a really great example of this there's a failure mode for principal component analysis which is actually really common in neural data because principal component analysis basis well where's the most variability okay i'm gonna worry about that and then all the stuff that's not varying very much i'm just gonna throw it away right it's just like you know dimensions in which there's low variability are not important well it turns out that in neural data the dimensions in which there's very little variability are some of the most important dimensions and so pre-processing with pca runs a risk of throwing out the most valuable information in your data set yes and so there's a lot of wisdom in in in jointly right pre in in jointly fitting your pre-processing model as well as your inference [00:25:44] Speaker 2: and prediction model i mean on this subject of not throwing things away and jepper and non-contrastive learning it's part of this bigger field of self-supervised learning and we want to learn representations that maintain fidelity and richness and lacune's hypothesis is that when you do something like supervised learning with you know some particular downstream task in mind the neural network gets wise and what it does is it kind of discards all of the the long tail stuff that aren't relevant for that particular task so when you train these models what you're trying to do is sort of maintain enough ambiguity so that it it compresses the information but it also maintains enough fidelity to work broadly [00:26:23] Speaker 1: for different things yes and that that and that is a laudable goal right and and i certainly share it right the last thing you want to do is i mean you know fortunately like networks are so big we don't really run the risk of of like uh overfitting so as much as we used to um but the last thing you want to do is throw is is is train your network to toss information that you might need down the road um that said like the vast majority of what you know the brain does just like these neural networks is decide what information is currently task irrelevant but that's all the more reason to do things in a self-supervised or unsupervised way right because you're basically not telling it this is the importance you know you're not telling it like what's all task relevant and task irrelevant so um i interviewed uh chalet about [00:27:09] Speaker 2: the version two of the arc challenge and one thing that struck me is i think of intelligence as being multi-dimensional so version one got saturated the arc was actually really amazing because it's the only intelligence benchmark that has survived for five years before being defeated you know since the advent of these thinking models it has been defeated very quickly but they're working on version three and there'll be version four there'll be version five will there always just be something left over that [00:27:38] Speaker 1: sounds like another philosophical so yes is my answer there will always be there will always be something left over in the sense that like you know you know we we have this this has been the trajectory things have been going for a really long time right it's sort of like we get algorithms that do amazing new cool things and then someone comes along and says yeah but it can't build me it can't pull a rabbit out of a hat right and then and then of course what does someone do they oh they they figure out the new training protocol slightly different architecture or they just train it to pull rabbits out of hats and then suddenly it can and then someone proposes a new challenge and a new challenge and a new challenge and it's always this game of like one-upsmanship so the question becomes well what's the point at which there are no more new challenges and i'm not entirely certain we're ever going to get there right um it may very well be the case that we get you know these sort of algorithms that are capable of replicating the complete suite of human behaviors and then someone will come up with some criticism like yeah but it's not really doing x it's just faking it right this is just the direction things go [00:28:40] Speaker 2: because people really do think they're important yeah do you think that the concept of recursive [00:28:45] Speaker 1: self-improving intelligence is a valid one yes i do think that is so so i think that one of the most critical missing elements right now is some form of continual learning right you at the end of the day you really want an algorithm that that doesn't just learn on the training that on the training set and then just gets deployed you want something that that that runs around in the world and comes across things that it doesn't understand right and then is able to incorporate to build you know append its model in some sense right so this is like the this you know and there are some approaches to do it's all based on like bayesian non-parametric and dearly process priors and stuff like that where you you sort of see something that's surprising or unique or different something you didn't expect and it causes you to say i need to turn learning on because i got to figure this out that is an absolutely critical element that we need to be developing we are developing that it turns out that that's one of the nice things about this sort of object-centered physics discovery thing is because it's object-centered if it comes across a new situation that it does not understand it is capable of instantiating a completely brand [00:29:49] Speaker 2: new object just to explain this new situation continually learning agents can acquire new knowledge autonomously and and the whole you know the whole thing just learns more knowledge but intelligence intelligence feels different it it feels like in in the system that we've been describing the intelligence is the way we're implementing the you know the bayesian updates and and you know actually building the algorithms could could the systems on their own metaprogram themselves and develop better algorithms or something like that that's a very good question [00:30:22] Speaker 1: something that would be closer to true artificial intelligence than what we currently have would be capable of building models on the fly to deal with new situations to taking things that it knows about right and combining them in new and different ways um there are approaches that have some of that aspect to it like g flow nets from like benji and stuff is like is like a great example of something that at least in principle is a generative model of generative models right it's sort of like oh like you know i might actually need a new node like it's time to create a new latent variable because like like the current set's just not cutting the mustard anymore those are things that that that i think are hallmarks of of true intelligence i don't want to ever make the statement as soon as it's got that it's truly intelligent i will never ever ever say that um but i do think that that is a a critical component that that needs to be present right is the ability to generate new models on the fly to deal with novel situations and data um most of that you know um you know as well as the ability to um uh combine old models previous models in new and interesting ways this is actually how the brain evolved right we started out with like um you know really simple brains and there were different regions and they solved sort of different problems and what eventually happened as we evolved is is that these different regions of the brain learned to communicate with each other in new ways and through that communication acquired new abilities right and then eventually evolved into in you know you know new capabilities and things like that right i often like to point out to the i think olfaction is like the the sense that's not studied nearly enough it's an incredibly old part of the brain and arguably right it's the it's the first part of the brain that evolved the ability to do proper like associative processing right odor the odor unlike visual space right where there's translation symmetries and and all that sort of stuff and things are smooth olfactory space that does not exist right it's it's really really really combinatorial and complicated and the part of the brain that evolved to solve the olfactory problem arguably is the part that evolved into our frontal cortex don't quote me on that there's a lot of disagreement there that's just my take um but it certainly has a lot of the features that we associate with associative cortex right it is it wow i just said it got like six uses six three different uses of the word associate in that sentence but but i think you see what i mean right it it um it was all about like taking old capabilities right combining combining you know simple models and modules to create something that was more complex and then over time right so so that was what made the brain work right it was all about taking little things that worked and combining them in new and different ways in order to evolve you know effectively an emergent you know emergent properties emergent you know computational abilities and an emergent understanding of the world in which we live and i do think that like what what you know if when we get to the point where we start really saying oh this is actually truly intelligent it's going to have that feature it's going to have the ability to have a it's going to have a modular description of the world and it's going to have the ability to to combine those modules in a way that creates a more sophisticated understanding it's like legos right i can you know the the lego bricks all connect in certain ways and i can build like all sorts of new and amazing things that were never built before right out of them that's the capability that we have and that's the essence of like creativity it's why i refer to systems engineering as like the thing we really want our our ai models [00:33:58] Speaker 2: to be able to do collective intelligence is a bit different we we have this plasticity right we can adapt our behavior day by day we might see some kind of meta learning or some kind of change in our organization dynamics you know maybe some agents will specialize and it might be an existence proof of this kind of recursive you know super intelligence that we're talking about yeah i do i i think that's [00:34:20] Speaker 1: absolutely correct right is that you know so the specialization is great in fact i would argue that specialization is how we got all of this right and this was i'm pointing at london in case you there was some confusion there um right it was it was really about you know the interconnected highly specialized intelligences that are people and their ability to learn how to to work together that that you know gave rise to the technological revolution the brain is the same way right it's in my view it's highly specialized little modules or agents that are capable of of of of um being repurposed reused capable of communicating with one another in order to solve really complicated problems but there's always a benefit to specialization i don't believe in like like agi agi seems like a bit of a misnomer to me what we really want is not artificial general intelligence we want collect we want collective [00:35:15] Speaker 2: specialized intelligences what about scientific discovery do you think that we could you know what would the world look like when we could discover new drugs we could discover new knowledge [00:35:25] Speaker 1: in science you know right now the way that we're doing that is is um largely focused on summarizing vast troves of data and looking for correlations that are present in it um i think the next major milestone uh in this trajectory is is experimental design right not just oh well here's here's some correlations you you may not have seen because they're really small and this is what computers are good at they're really good at identifying small but highly relevant correlations um and uh the next step of course is is constructing a system that tests these hypotheses explicitly right and generates the experiments that will identify like that will fill in the gaps of our knowledge and all of this i believe can in fact be automated in a very sensible way i i you know i don't see any like major obstacles to automating empirical inquiry other than we probably want to place some safety constraints when we start letting them work when we start letting the ais run the labs right because you never know it's sort of you always have this ai was like well you know the most effective experiment to determine if this is correct is to set off a nuke and that that would be bad yes right so pure empirical inquiry right does run risk like that but i think that that's not not not the biggest issue i think what we need to do is we just had need to have a nice concise framework for saying like oh look you know like i'll give you an example so we had that we we had this um a problem that popped up a while back a gentleman we were talking to us is um is you've got these law you know you've got these robots and the robots see something that's never seen before and internet you know so a robot is like running around it comes across like a beach ball never seen a beach ball in its entire life and what you'd like is you'd like the robot to know how to figure out that it's a beach ball and to figure out what its properties are and if you tell the robot like like if you see something new just stop right you're kind of then that's that's no good right what you really want to do is you want to figure out a relatively non-invasive procedure for the robot to like poke do what a child would do what does a kid do when they see a beach ball right they run up and they poke it and they say oh right yeah and then it moved and and it actually learned it actually experiments with its environment for the purposes of identifying the properties of the objects that exist in it um now i do think we probably want to test this out virtually before it's deployed in the real world because you never know it might very well be that the optimal experiment experiment is to run up and kick it as hard as you possibly can um and we certainly want to avoid that but like something along those lines something you know a robot that is able to test the theories that it has about how things work in an online way and learn from those results in an online way is definitely [00:38:03] Speaker 2: part of the goal looking forwards what do you think the future will look like when we have more autonomous ai's among us a lot of people worry about infield enfeeblement loss of control you know it making us dumb all of this kind of stuff i do i do worry about ai making us dumb right i mean [00:38:20] Speaker 1: offloading offloading your thinking onto a machine which is something that that that ai allows is is is is is a potentially a big problem i i don't really want to have a situation where humans are reduced to like value they're just to reduce to like value function selectors they're just basically going oh no i don't like that outcome like do this instead i do want to see a future where where we have an ai that actually improves our understanding of the world and simply automating everything runs the risk that you specified right it runs the risk of people becoming couch potatoes that just watch tv and occasionally say like yeah you know these chips are no good um uh that seems like a bad outcome to me um i worry less about that i think than some because people are remarkably adaptable right i mean i you know you have all these arguments about like oh you know this new technology comes along and it's going to completely destroy this way of life and you know and that's going to be awful for people and it is maybe in the short term um you know i think of like tractors right or just go back how many hundred years you have to go back when like 99 of people were involved in agriculture and now it's like what two right i consider that a solid improvement right because it allowed the rest of us to it allowed us to do a bunch of other things that we find more satisfying that are more interesting it allowed us to like you know like i can read you know spend some time reading a book don't have to labor in the fields all day um that's the future that i sort of see and that's the future that i hope for is that is is one in which you know all of these artificial agents running around and doing things autonomously um are there to to free us up to pursue more interesting more you know you know to improve ourselves in in in in in more interesting ways but at the end of the day it's just another technology you know at least initially it'll just be another technology like the tractor now 100 years from now who knows what will the value of work be if the ais can do everything and there's nothing left for us to do i don't think that will ever be the case that the ais can do everything like i said the future i worry about is one where like it's you know the the sole role of people is like sitting around like making sure the ais aren't aren't going rogue and and and things like that um which i don't consider a good outcome i would really like to see human improvement you know i i envision a future of i don't know this like cybernetic transhumanism if i'm going to go sci-fi on this right where where you know the technology and us evolve together in a way that's beneficial for both that's the goal um you know are there these dystopic possibilities where like oh well what are humans in a world where well what are they what are what are humans in a world where everything can be done by a robot yeah you know that's that's a good question and that's and at the end of the day right they end up just becoming like reward function selectors right they end up just sort of saying oh i don't like this and i do like that and they're basically you know i mean you end up with this is another nightmare scenario i don't like talking about these dystopian futures because honestly i think people are too clever and i think people are too motivated and people are too interested in how the world really works and people are too interested in actually understanding things that they will never stop that they that ai will become a partner not an adversary or a crutch and that's that's that's what i think will happen because that's but that's a statement more about my belief about humans than it is about my belief about the development of ai you know i am a techno optimist if you will not a not a pessimist i i believe that we will find a way to adapt to an ever-changing world as we have done for millions of years including one that includes technology that alleviates most of [00:42:07] Speaker 2: our labors on that there's an ai literacy thing because ai has moved so quickly now that certainly my parents don't understand anything about it but by the same token policy makers don't understand anything about it and there are people saying ai is going to kill everyone and there's people making negative arguments as people making positive arguments so there's a bit of a fog of war now because there are so many people saying different things about ai how should they make sense of all of this [00:42:32] Speaker 1: we are now well outside my area of expertise so i'm just going to say that before i say anything else um ai is developing very quickly but i am much more concerned about what people will do with the new technology than i am with what the technology will do all by itself i don't have the this big concern about i don't really believe that like you know skynet's going to take over the internet's going to suddenly become conscious and kill us all right um in part because you know ai is not that advanced but also because we are telling a we you know we are still in the position where we specify the goals of the system and that will likely continue for a very long time and it will always be the case that these systems you know will can be you know are subject to review we will always keep an eye on them they will always at least initially be be released in relatively restricted domains and where we're where we're test where we're keeping a close eye on what it is that they are and are not doing so i don't worry too much about like the going rogue i worry a lot more about somebody building you know it's sort of like a virus which we already have to deal with like somebody builds like some insane virus and like takes down the internet i'm more worried about malicious human actors than i'm malicious ai actors because at the end of the day all of these algorithms they simply do what they are told right we train them we tell them here's your objective function as long as we are specifying the objective function and we understand the objective function we're probably going to be okay i think the safest way to deal with ai concerns is to tell people hey look this ai is just doing what we told it to we we you know we set it up to make really good predictions and to achieve these outcomes now is it dangerous to like specify these outcomes without being very very very careful yes it is right that's this is the whole like hey skynet and world hunger and it kills all humans that that's a that is that that is a real possibility but whose fault was that the fault was the person who like was very very naively specified their goals there are in fact relatively straightforward ways to specify the the reward function that that don't run that risk nearly as badly and the best one is so are you familiar with like maximum entropy inverse reinforcement learning i like to call it active inference because it's really similar um and so there what you're doing is you're basically observing someone's policy and then you're trying to do a maximum entropy um model you're doing maximum model on the reward function itself um at the end of the day what ends up happening when you do this is this is why it's like basically just like active inference you get a reward func and so you have some you know organism or whatever and you're trying to do this for it and and it's got some stationary distribution over actions and outcomes right it's inputs and outputs of a stationary distribution that becomes your reward function like not directly there's some math involved but basically your reward function is a function of the steady state distributions over actions and outcomes so we could do this right we could take the car we could take the current manner in which humans are making decisions and we could write down right what's the stationary what is the current estimate of the stationary distribution of our actions and outcomes so this would include things like everyone's getting you know this number of people are going hungry this you know and and you know all the stats that describe like the inputs and outputs to our policy make you know to our policy distribution um and then we could just ask an ai your reward function is the one that results in the same outcome that we currently have right on average and it would execute it and it would and and to the extent that it works right it it it would it would ultimately result in a in an ai algorithm that just sort of is like mimicking human behavior right or at least achieving the same outcome that we were achieving before now here's the safe way to like improve the situation you don't say end world hunger right you perturb that distribution over outcomes right and just just over outcomes a little bit and then you evaluate the consequences right it's it's all you're doing you make these little changes in the reward in an empirically estimated reward function right rather than just sort of specifying one by hand because that's the dangerous thing jeff thank you so much for joining us that's my pleasure amazing