# Lec 9 | MIT 9.00SC Introduction to Psychology, Spring 2011

**Channel:** MIT OpenCourseWare
**Source:** https://www.youtube.com/watch?v=SBrCPDC21f4
**Transcript page:** https://www.withtranscript.ai/video/SBrCPDC21f4

## Chapters

- 0:01 — Chapter 1 (0:01)
- 7:02 — Chapter 2 (7:02)
- 14:27 — Chapter 3 (14:27)
- 21:07 — Chapter 4 (21:07)
- 28:54 — Chapter 5 (28:54)
- 37:00 — Chapter 6 (37:00)
- 41:23 — Chapter 7 (41:23)

## Transcript

**[0:01]:** the following content is provided under a Creative Commons license your support will help MIT open courseware continue to offer highquality educational resources for free to make a donation or view additional materials from hundreds of MIT courses visit MIT opencourseware at ocw.mit.edu

**[0:32]:** okay uh good afternoon um so today we're going to talk about learning so uh what can you do now that you couldn't do as an

**[0:49]:** infant very simply talk that's a good one language that's pretty good right I mean huge anything else that you couldn't do yeah yeah we don't actually know what the internal mental life of an infant like but it doesn't seem like they remember many very specific things about their lives you do but you know it's a lot more you know you know a ton of stuff a lot of facts right things about the world values you hold about your family your culture the country or countries you've grown up in you just know a ton compared to an infant and there's only two ways that you know this it's either in your genes when you're born or you're learned it through life and so we're going to talk today about how you know scientific psychology tries to approach learning and then a remarkable thing about learning is we often think well we're learning we're learning lessons or

**[1:40]:** whatever we're learning material but the important part of learning the way we think about it is that it allows us to predict the future on the basis of the past right we want to learn lessons in life to know what's desirable what's dangerous what's a smart way to do things so we're more effective we choose what we want to do better in the future to learn from the past and predict the future it em use our word with meaning for an infant not much means much right they just look around who knows what anything means words gestures facial expressions uh there's not much meaning of the world out there for you the world is full of meaning history social interactions everything that you're thinking about so how do we acquire the meanings of things in the world that matter to us and it's also had an interesting kind of uh give and take with how

**[2:26]:** we learn about learning in scientific psychology sort of his Al I'll just touch a tiny bit on that and we'll talk about three aspects of learning today uh classical conditioning operant conditioning and then various ways in which uh uh there are limits to conditioning thank you Tyler so let's off you know with the sort of the one of the most famous figures in all of conditioning or learning Ivan Pavlov who actually won a Nobel Prize not for his studies of learning but for his sort of groundbreaking studies for the reflexes for the reflexes of digestion he worked out a lot of the fundamentals of of how in mammals your stomach breaks down food and then how uh food in the mouth provokes specific salivation that begins the process by which food is decomposed uh and and then digested in your stomach the fundamentals of really how we live right Eat to Live um and he

**[3:15]:** was very interested in salivation reflexes what is it that drives that first bit of breaking down food in your mouth with saliva and uh Pavlov was a to was a pretty intense researcher as you might be used to at MIT uh there's a famous story where a graduate student came somewhat late to the lab and said but Professor there's a Revolution going on with shooting in the streets this is in Russia and he said what difference does it make where when you've got work to do in the laboratory next time there's a revolution get up earlier okay so uh so anyway you know not an easygoing uh supervisor um and he would do experiments a little rough in dogs like cut the esophagus the path by which food goes from the mouth to the stomach to understand the role of the esophagus and he found that even in those animals when he placed food in

**[4:01]:** the dog's mouth he expected as a fundamental biologist studying how digestion works that nothing would happen but he noticed Not only would the dog salivate even though the food will never make it to the stomach but furthermore the stomach juices the gastric juices that break down food also were released within the stomach even though the food would never get there and then he got interested in how it is that these things are driven if they're not driven simply as a direct response to the arrival of the food how is it that uh the the mayor sight of the food or the sight of the person bringing the food would drive the Salvation um and what he called psychic secretions you know because he was a biologist so he expected basically you put the food in the mouth there's some Chemistry Between the food and salvation and here he's here he's finding just seeing food just

**[4:46]:** seeing the person who would bring the food was driving these fundamental biological uh functions so this is a kind of experiment that he would set up where the dog would be there he be measuring the amount of Salvation in some way or another um and in these kinds of experiments and I'll I'll go through this language a couple times uh in conditioning they've come up with a sort of vocabulary to describe the pieces of things that go into this kind of learning so there's an unconditioned stimulus which is food I mean you have you start with that you want to eat food and there's an uncondition response which is you salivate when you see the food or if it's in your mouth that's that's the beginning then you're going to add what's initially a meaningless stimulus in this case and it could be anything but in this case we'll say it's a bell you ring

**[5:27]:** a bell before the food comes and you're going to get to a conditioned response the animal will say well the Bell predicts the food and I'll salivate to the Bell itself even though I'll never eat the bell I learned that it predicts the food is coming and so you've created a new Association you learned something new about the world that a bell signals something that drives salvation so that the two things are are sort of occurring next to each other in time or what Aristotle talked about as the law of contiguity the animal learns that the Bell predicts the food coming and so here's the basic idea at first you know the tone does not drive salivation right there's no reason to salivate when you hear tone by itself but as the animal discovers that the tone predicts the food the tone predicts the food the prone predicts the food then all of a sudden

**[6:09]:** uh the animal starts to learn that simply the tone alone will drive the Salvation now this stimulus is conditioned and you have a conditioned response that wasn't there to start with it's it's very simple right you learn that two are associated and now you you uh begin to have a biological response to the tone Alone um and typically what happens is here's a learning curve here's how often perhaps you salivate and across trials you get you learn more and more of that and you finally Plateau at some level of performance so a neat thing about Pavlov is although there weren't a lot of movies at his time there is a little bit of film of him and his actual dogs so here's paavo's actual dogs

**[7:02]:** so um so we said conditioning is going to do two magical things right it's going to let us predict the future on the basis of the past and this very simple experiment now the Bell predicts the food coming up and the Salvation And it Imus the world with meaning in a sense the Bell now means here's dinner right the Bell is not just a bell it's something with meaning and significance these are other example another example of classical conditioning and it's got some sophisticated properties you could go well balloons popping water in your face food for dinner that's pretty basic stuff but here's some sophisticated properties of these as learning mechanisms so one was already talked about Extinction that if the Bell continues to ring and no food arrives the learning gets weakened weakened and disappears so you can learn something and you can unlearn something because it's

**[7:56]:** no longer an effective predictor of the future generalization is kind of cute also so imagine the the the frequency of the stimulus is this just the specific frequency of the tone well if the tone is a little bit different you know a little bit a little bit different you might still think food is coming right so it's not that you learn the precise thing uh you'll be a little you'll still respond if it's a pretty similar tone and as the tone gets more and more different your response gets weaker and weaker right it's that smart generalization I stay pretty close to the tone I go for it I get further away H not so sure okay it's not all or none it's reasonably graded uh and sensitive to the specifics of the situation there's also one fascinating thing um which is here you acquire a condition response is extinguished because you're only hearing that Bell

**[8:46]:** and there's no more food coming but when you come back in and uh you'll often have a kind of preserved recovery function of that CS it's like it's lurking in you somewhere okay it's like I have a mental note you know that this this Bell might mean food okay even though it was extinguished uh it's still pretty interesting stimulus so you'd have some easily spontaneous recovery of these kinds of learned uh uh conditioning it can also be sophisticated in other ways and let me give you a couple of examples um uh you can learn to discriminate between two things we've only talked about one thing so far Bell food but what if you what if you have a black patch for example that predicts food coming and a gray one that predicts it's not coming you can distinguish between those two you can get second order conditioning let me show you that um the dog

**[9:33]:** learns that the Bell predicts food now there's no food presented for a while but there's a black patch presented that you see presented with the tone because you know the tone predicts food the animal figures well maybe this is a pretty good hint that food is coming and after a while it begins to salivate to this one even though it was never directly paired with the food itself right so it's learning that if you know it's a Transit of inference that it's making you know if the Bell predicts it and the square goes with the bell at some point I'll start cating for the square because that sounds like a pretty good candidate for predicting the future so that's pretty sophisticated too you don't even have food in this part but it's even more sophisticated than that so uh look at these two things and you can think about I'm going to let you think

**[10:17]:** about this for a moment here are two groups of animals or could be people would work just the same way as um so here's the condition stimulus and the unconditioned stimulus for example the Bell and the food the Bell and the food the Bell and the food one group only gets Bell food Bell food kinds of pairings the other group gets the same pairings but in between they'll have a random uh food reward two random Bells random food random Bell now the pairings are identical that's shown in Orange this group will learn better and faster more strongly than this group even though they've had the identical number of pairings why will this group learn more slowly why I will the even though they have an identical number of times they got the tone and the food the tone and the food yeah is there a little bit of Extinction between it's an it's an idea

**[11:07]:** that is's an Extinction between I think that's a good idea so that that's a very good answer Extinction between because when when you get this uh C us alone you're extent you're fighting the learning but it's telling you the system is smart right and because what it's kind of doing is saying look ah CS alone not always a reliable predictor so I'll learn it but I'm not going to believe it that's so strong right and it's basically because you're you're sticking Extinction in between those right you're also having some trials with the food that aren't predicted either right so it's like the the animal or you are learning what's a really excellent signal and if there's other stuff in there that gives the signal not the same predictive power for the future you you learn more slowly and less certainly that relationship okay so that's pretty smart too here's another way in which it's smart

**[11:54]:** um and I'll tell you the phenomena and you think about it for why you think you know learn learning might work this way in animals and people so pretend there's a tone that predicts food just like we've been talking about all the way through food stops and now you get a tone and a light a tone and a light a tone and a light the tone you know predicts is food you're getting this part again without food being presented and now you get the light alone okay or everything's the same but there isn't the initial learning uh with the tone okay and then in the end you're tested for the light and what happens is there's less conditioning to the light here than here even though you've seen a light an exact even number of times why is the conditioning to the light weaker here why is it weaker here than

**[12:48]:** here this is an information problem for the brain to solve about the truth of the world okay well our interpretation is this tone food tone food you go good I got it easy right now you get tone and light together for a while and you go okay I know the tone is really important the light I don't know the tone was perfectly good the light seems redundant informationally redundant okay so now when it's a light alone uh you don't have so much conditioning because you didn't care about the light so much the food had all the information you needed the light was informationally redundant right it was Superfluous here you only got trained with a tone and the light so you never made a bet on what's the critical thing they both predicted evenly so when the light comes by itself that was always something that seemed like a pretty good signal equally good to

**[13:40]:** the tone so that's very sophisticated that the animal learning it or the human learning it is deciding what's the information that's Diagnostic and what's the information that's redundant and it's conditioning across that right so it's it's not just very simple reflexes it's sophisticated learning and here's um okay so now I'm going to switch gears so we're going to have a demonstration that involves the opposite of fearful things like spraying water and uh popping balloons uh but I can't tell you too much about this but I can tell you it's it'll be pretty Pleasant okay but I need somebody's willing to step out of the room for a moment uh to do this okay thanks yeah yeah you can safely take off your jacket now

**[14:27]:** okay and uh what's your name I'm Sam Sam okay try not to listen if if possible okay so so you guys are going to help me perform instrumental conditioning and then we'll discuss what that is uh our job is to have Sam come over here and pick this up okay um your job is as Sam comes in for every action he takes a step this way you know you played this game as a kid or arm going in the right direction applaud if you he's doing something that gets him closer to the goal of picking up the brain and withhold your applause if he's doing something that's taking him away from that is that okay all right okay so we're going to discuss uh uh uh What instrumental conditioning is and then we're going to slightly think philosophically about whether this is how you learn to do everything you get to choose who the Applause

**[15:23]:** comes from your parents your family your friends you know uh Medical schools and law schools and graduate schools you get to choose who the Applause come from but how much of our Lives is responding to the Applause that we want to that we care about so we'll start more simply less philosophically let's think about this the examples we talked so far in classical conditioning all build on what feels like a basic reflex salivating for food flinching for water right okay so if we ask somebody like why do you work hard there's not going to be a very good explanation in terms of those initial reflexes right okay you're not flinching you're not looking for food or or water and yet much of our life is involved in Endeavors that we learn where there's no primary reflex that we can see in the story and so operant or instrumental conditioning those two words are interchangeable uh

**[16:14]:** is the way that psychologists have tried to understand those forms of learning um and it began something like this and this is uh from Thorndike he put a cat into a puzzle box I'll show you what the box looks like or a sketch of it the cat had to unlatch a door by pulling a latch so here's Cat's In A Box like this to get out which cat wants to get out he or she has to push this which will push the latch now that's not obvious to a cat right wouldn't be necessarily obvious to a person but definitely not obvious to a cat um and the animal learned it by trial and error sort of like Sam did right and uh where is the where is the UN uncondition stimulus there's no salivation or any reward like that so Thorn proposed that you at people and and animals learn things by the consequence of

**[17:01]:** their response and the law of effect says the consequence of a response determines whether it is strengthened or weakened you can be rewarded that will strengthen the response you can have no reward that will weaken it you can be punished that will greatly weaken the response so here's the cat here's a an example of a cat at first it takes a long time for the cat to get out of there as it tries different things but with over repeated trials the cat can get out really really fast and how Thorndike you know by observation and thinking thought about it was at the beginning the animal wants to get out it has no idea how it gets out so it does everything it can it scratches at the bars it pushes at the ceilings it digs to the floor it howls it press and some moment it presses a lever and that's good right okay so

**[17:44]:** here's different things that the animal starts with thinking might be good ideas but because it discovers with repeated efforts repeated trials that pressing the lever works these get weakened this get strengthened so the consequence of a response determines whether it strengthened or weakened and now there's no salivating or reflex or anything like that you can learn then almost anything um so this was picked up a lot uh in the US which is John Watson is a famous name you'll see a kind of a haunting uh uh film of him performing this kinds of stuff with infants in a kind of a way that we would never allow ethically now it's not horrible but it's not it's pretty bad okay but it's a famous thing and you'll just get a feeling of the power of this uh behaviorism uh that if you study behavior instead of going in you know for things like uh unobservable thoughts

**[18:34]:** that you would just do observable actions and you wouldn't make inferences about the mind that you can't measure you make a stimulus you measure the response and that's it uh no fundamental differences between animals and humans and you make laws that describe the relationships between stimuli and the environment and physical behaviors that you can observe or responses uh in out in the world that an organism has so I'll come back to this a little bit but one of the ways he wanted to show anybody can learn anything because right now don't forget the way we set this up anybody can learn anything you can applaud anything right uh so that makes you know you could teach anybody anything by this perspective and he went to show that he could teach an infant who initially uh whose name was little Albert uh that was his cover name nobody actually knows his real name or what happened

**[19:19]:** to him um and now well nothing horrible happened to us first we know uh and now you would never do an experiment like this and this if you have this in your no this is a long version of movies taken of this actual sort of experiment that we wouldn't do now um and I'll show you a short version in a couple minutes where he he got an infant to become afraid of a a rabbit that the infant previously was not afraid of um and then the other huge name in this is BF Skinner who worked at Harvard uh uh consequence of a response again F you know creates the responses that we make in the world and him sense response is not just the laboratory one but things we do in the world uh with each other and to each other and so the classical conditioning the Cs elicits the CR the condition stimulus The

**[20:04]:** Bell elicits the condition response elevation but instrumental conditions CRS are emitted uh so you can do really anything that's in the range of what an organism can do and then he could show kind of impressively how you might imagine a chain of these things would lead to complex behaviors so he would teach uh pigeons and other animals to you know first to click to get a pellet then where to click it then to where to face before you click it and he would just keep adding little pieces shape the pieces up from the first step to the Second Step until you got pretty complex behaviors um okay so now I'm going to show you a slightly longer clip that has in it uh these kinds of experiments and little Albert it's kind of cool because in the sense that um

**[20:52]:** uh we we can actually see films of these sort of historical things which often we don't get to do in various science fields

**[21:07]:** so um let's talk about that just for one moment so we can talk about primary reinforcers like food thirst or pain things that go with life right pain not being injured food and thirst we can talk about secondary reinforcers that we know a lot about uh money attention of other people praise from people admissions to places you want to be admitted to Promotions when you want to get them there can be positive or negative rewards uh and I'm going to talk about one thing but just for a moment um you know the the behaviorists said anybody can learn anything right um we'll come back to that so that's that's been a big debate uh whether you know anything can be equally learned with any other thing and the other thing that's uh uh striking is you know uh they BF Skinner particularly said that we have an illusion of free will that all we are are

**[21:59]:** learning these things and we might as well you know get over our illusion at Free Will scientifically it's very hard for you know even the best scientific psychologist to tell you whether Free Will exists or not that's not something that's easy for us to begin to imagine how to measure uh but these kinds of learning things are very impressive in many ways and we'll talk about some of that um okay so let me talk to you about uh one way in which for example um uh casinos get people to come back often and other things in life okay without giving you money every time because then they wouldn't make money right so uh your book talks about a number of examples of this but I'll just pick one partial reinforcement so this means when do you get a in these animals when do you get a reward every time or only a third of the

**[22:47]:** time so of course you learn faster if you get a reward every time than if it's a third of the time you're not quite as impressed and excited but here's the interesting thing if you stop giving the reward these animals will go on for quite a while before they extinguish these animals will extinguish right away and stop responding why will these animals keep behaving for a time even though you've entirely stopped the rewards yeah well because since they only got it like 30% of the time anyway it seems plausible to them that it would go a couple more times without them getting it yeah since they only got the answer was was right was only got 30% time so they're going to you know the first one or two non payments right they're going to go well it's coming up next okay a couple more okay coming up soon you until they finally decide that

**[23:36]:** the whole thing is over right okay so uh so it's very interesting the different reinforcement schedules you know how they relate you might think well you always learn better if you reward more but in some cases if you want to get people to keep doing stuff without direct reward partial reinforcement is actually more powerful um okay now we're going to move into an animal a dog experiment and and then directly to sort of thinking about people in our own lives so this is a a famous famous experiment uh University of Pennsylvania from Seligman and this case he used dogs it's been done with many species in many circumstances and he put them in in a hammock so there's two dogs on two sides of a thing that they walked around and they don't see each other the dog a would periodically get a shock you know um and he could stop the shock by pushing

**[24:24]:** a panel near its nose okay as soon as the shock came he could turn its nose and stop the shock dog B on the other side of the divider would get exactly the same shock it would stop at exactly the moment that the first dog stopped it but dog B had no Direct Control okay equal number of shocks equal duration of pain of objective pain if you want to call it that way one animal has the capacity to stop it the other does not so that's the first part of the experiment in the second part of the experiment they're put into a shuttle box uh okay that looks it's just a box with a big divider in the middle and they're just hanging out there and then they hear a tone at first it means nothing to them um and then comes a a shock on the grid on the floor okay and at first

**[25:11]:** they don't know what to do uh but then after a while they they get smart like the cat and they jump over the divider to the other side and it's all okay so you can imagine that when they hear the tone pretty quickly what did they learn to do jump over the divider and stop the pain and here's their remarkable thing for the animals who had the experience that pushing the panel stopped the pain they learned that pretty quickly for the animals who could not stop the pain initially they don't learn how to escape when they could it's as if the animals in group b what they learned about life you say you it's painful and you can't do anything about it right the animals in group a learned pain happens but I can do something about it um so here's the animal you know first it doesn't know what to do it's getting the

**[26:02]:** shock but with trials the animal you know the animal learns to jump and jump so quickly that it doesn't have almost any problem here's the tone boom it's out of there okay it learns to avoid the pain but what happens to the animals who either had the escapable shock are the ones right the ones who could poke their nose they also learned this the ones who could jump over there's nothing preventing them from learning that they just H you know don't escape it's it's what elment called learned helplessness that the animals learn from the first experience their conclusion was pain happens and I can't do anything about it and then when they get in a second situation where they can do a lot about it they continue to feel helpless and not do something about it it's like a deep lesson about life like when do you think you can make your situation better and

**[26:48]:** when do you think there's nothing I can do about it it's just bad bad bad okay um and so there's ideas about that some of this might be related to people who struggle with depr depression but also other things in life so here's the idea from Seligman as an explanation of how we have resilience the idea is that we all you know get in sometimes in small doses sometimes in tragically large doses difficult things into our lives how do we explain those difficulties that have happened to us the setbacks we have do we think they're internal to us you know I'm the kind of person who has bad luck I'm the kind of person that these things happen to or is it external you know somebody gave me a test you know and what can I do about that but next week the test is over and life will be better okay is it Global

**[27:33]:** or specific does it just seem miserable all over or is it just you know one specific squirt in the face that this faculty member gave me okay is it stable or unstable are are bad things just seeming to happen all the time or is it one occasion and you can imagine that people who believe that bad things are internal Global and stable will be learn to be helpless everything seems you know dark and hopeless uh people who believe that things are external specific and unstable thing the next time I can do better I can bounce back from this and do better and it's not only in in terms of the S Spirit of people and trying to get people to you know bounce back from setbacks uh it's even things like job applications so Seligman did the following bet uh I don't even know what corresponds to it before but there used to be an

**[28:16]:** era when people would go knock on doors and try to sell you insurance we don't do that anymore right because if somebody comes our door knocks at our door or like calling the police but when things were more calm you know you'd open the door and talk to or they would give you cold calls which you don't get much more anymore these days you guys are have youever even heard that phrase you know you'd work at some insurance company and they go here's some random telephone numbers you call them up and you try to sell them Insurance you know and like usually you get abused on the phone or somebody slams the door but these salesp people they make commissions and they keep going and he said let me give my test he gave a questionnaire whether people interpret the world one way or the other way the resilient way or the Learned helplessness way

**[28:54]:** and you give your usual interview way for picking salespeople and let me see who picks a better salesperson and his questionnaire picked a better salesperson because if you have a sales job where people often you know kick you out and hang up on you you better be pretty uh external specific and unstable in your perception on things right because when you have a hundred people who hang up on you or close the door on you for one sale you need to have a thick resilient skin okay so lots of ways uh which I think this speaks to how do we respond to the setbacks in our lives um so talk about uh some limits to conditioning and and some of these I think touch also on what are the values by which uh we lead Our Lives it's not going to be telling you that but you know it'll make you think about some of

**[29:38]:** the mechanics of what matters to you so let me slide back for a moment so the behavior said everybody can learn everything everybody can learn everything in that sense it was seen as a very Democratic and eali terot PR perfect everybody can learn everything is just how you set up the contingencies in the environment and then Garcia uh did an famous EXP experiment where he had rats be get shocks or receive uh uh lithium chloride which made them uh nauseous and they either had a a bright noisy stimulus or they tasted some sweet water and even though these things were arbitrarily combined the rats learned uh who who received shock learned to fear the bright noise the the bright light in the noise and the Rats who got the lithium cloride made OB nauseous learned to fear the Sweet Water do you see why okay when when we get nauseous we think do we think

**[30:33]:** like boy it was noisy last night or what do you think when you wake up in the morning you feel nauseous what's the first thing you think yeah something I ate right we're prepared for internal uh nausea to think about food as a risk and if it's a shock we're prepared to think some something out there okay so these animals instinctively not on the basis of the training instinctively uh you know said if if I if I if I feel a shock it's probably related to this bright noisy stimulus that warns me if I feel nauseous it's related to the uh sugar water and they had these opposite patterns of learning for the stimulas as if these animals were prepared to fear some things and not others so it wasn't that everything could be matched with everything and in fact with humans they've done things like show you pictures either of snakes and spiders or

**[31:22]:** flowers and mushrooms and then those predicted a shock okay a small shock to humans they measured you'll G vanic skin response an autonomic response when you think you're getting in a shock you start to get a little sweaty okay uh and what they found was and this will not Shock you they had better conditioning you know you more quickly and completely learned that a snake or a spider predicts a shock than a flower or a mushroom right even though objectively they were just as predictive there's something about snakes and spiders that gets us ready for scary stuff okay and flowers and mushrooms they really have to punish us before we think they're evil War signal okay so again a human example of preparedness we don't learn everything equally we prepared to learn some things and not others e okay same thing with little Albert the child that you saw you know that kind of a

**[32:09]:** film is a funny status so we'll talk about a few more things uh this semester where that experiment why should that experiment not be done yeah nobody know yeah well you just there you would never terrorize a child right to scare the child you know okay so weirdly enough though these studies remain in the field because since we can't do them again because we know they're not right to do we you know that we we try to learn lessons from them and you know so that we learn something the rat worked okay uh he got scared of a rat and then they got scared of a bunny got scared of a a cat anything animal anything furry but if they gave him a wooden block that didn't work okay it's not everything uh it has to be prepared from that here's another one uh that show shows you uh that also uh challenges simple ideas of do we always learn simply to get a reward okay what kind of learning might exist that's rewardless so there's three groups of rats

**[33:10]:** in a gold maze um there's food rewards every day for one group they're a happy group there's no rewards for a grumbly group there's a third group that gets no reward for 10 days and then it gets reward okay so one reward all the time reward never no reward for 10 days and then you start to get rewards all the time and here's what happens to the learning if you look at uh uh this green line that's the food that gets rewarded all the time so they learn to go uh uh uh they make fewer errors so they can get to the food faster if you look at the group uh that's never rewarded this yellow line is pretty steady it's like we're I'm hanging out no reason to go anywhere there's not much going on right look at the orange group for 10 days up to here they look like the yellow line right

**[33:52]:** not getting rewarded I'm just going to saunter around there's nothing interesting to learn here but boom that first day that it's rewarded by the 12th day by just one day of learning they're all caught up they're all caught up so people call this late in learning that they were really learning a lot about their environment okay anyway even though it was of no use at the moment and then the moment it became valuable information boom they're all caught up they didn't it's as if they learned all this information that was ready to go late in learning the information was stored up just because they were in the environment even though it was unrewarded and as we go around we often pick up stuff even though there's no direct reward Bo and it sometimes it turns out to be useful people also love humans including infants contingency this this the idea that we feel like we

**[34:36]:** have control over our world so here's a two-month-old infant very young infant in a little experiment where they put the infant in a crib and above is a very colorful color mobile and in one version they they uh when the infant moves his or her head there's a switch in the pillow the mobile moves and the infants are smiling and cooing they love it this is awesome move their head the thing moves pretty impressive for two months all they can do is wle their head right they're not walking or anything um they put another group of infants in who have uh no switch and just periodically they turn on the uh mobile there's no smiling no coing and they equate the number of mobile turnings okay why is that well if it's just because you like the mobile turning then you should have both groups be equally cooing and smiling or disinterested but there there's

**[35:25]:** something fundamentally rewarding for humans even the age of two to feel like they have control over their environment and their pleasures and that's that's not explained by conditioning nobody's getting anything more they're both seeing the equal number of interesting turns but one group feels like they're controlling their environment here's another another one that I'm going to show you in an animal experiment where you can control things but you can imagine in your own life and you know in my life I see this all the time it's nice when something pleasant in happens in your life and your situation gets better in some way right you get you know you get a better iPad right uh uh uh it's you know but how hard is it emotionally when something gets worse they take all your iPads away okay all right you you know when things or more serious things when things you know get worse often

**[36:14]:** we feel pretty miserable and here's a neat example of that so these are rats in a experiment so everything's controlled so here's one group of rats uh this is how quickly they're running so it's good to be High um one groups of rats is constantly getting one pellet day okay so they learn what to do and they're sort of hanging out here doing well a second group of rats is the luckiest group of all they get eight pellets and they learn more they flatten out okay so see you get more reward you drive the behavior more that's not unexpected the interesting thing are the two contrast groups the contrast groups start off with one or eight and then after a couple days they're flipped around so some started with one then they got eight pellets others started with eight then they got one does that make sense okay so here's the contrast group of eight

**[37:00]:** uh you know they're learning one they go eight woo okay all right they're they're happy okay they're getting eight they life's got good look at the group for whom life got worse I'm getting eight I'm getting eight I'm getting eight I'm getting one I'm getting one I'm getting one it really stinks they're actually worse than the group that got one all along right this is the lowest line of all they're protesting this stinks okay I was getting eight and I'm getting one what a ripoff you know what kind of world do I live in right so the learning is protesting that now by purely conditioning one why would this group be worse they're still getting a pellet okay but because they're interpreting it as a world that's going downhill as a worsening situation they're less prone to learn it they're sort of protesting and they're learning uh things having gotten worse in the world um

**[37:55]:** a really interesting theme along the line of this reward and an unbelievable one that's gotten a lot of attention is delayed gratification you know if you think about it and now this is not purely scientific but is a a theme delayed gratification is a spectacular part of your life right especially if you go through higher education or maybe everything is like this right you know do a good job in primary thing and you'll get to a good middle school do a good job in middle school you'll get to a good high school do a good job in high school you get to MIT do a good job in MIT and you know you go to med school or law school or become a teacher whatever you want to do good job a good job with that and you'll get to this thing do a good job of that you know finally thank you very much

**[38:29]:** you're 95 and you're done right okay you know I mean I mean it's it's kind of like uh you know you guys are waiting a long time between the efforts you're doing and what you might consider a palpable reward um so here's a fantastically interesting study from Walter Michelle done at Stanford some years ago he had four you children were four to five years old there were about 600 that were studied they were mostly the children of faculty and graduate students at Stanford um and they would come into a room and the experiment slightly vared was basically like this they would sit in front of either a cookie or a marshmallow they use different rewards but imagine it's a marshmallow in the movie I'll show you it's Oreo cookies and they were told this and and they try to make this Walter Michelle experiment describes how he played with kids to try to get to

**[39:18]:** make the cookie look like real or marshmal look really awesome okay and the kid would sit there the four or 5-year-old uh and they were told I'm going to leave the room now there's a bell here if you really want to eat that marshmallow uh ring the bell and just eat it but if you wait a while they didn't tell them it was 15 minutes it was 15 minutes I'll come back and I'll give you two delayed gratification I can have one marshmallow now or if I hang in there for 15 minutes and don't touch it I can get two okay let me tell you the result uh uh uh and then let me show a film of this and when I was looking on YouTube the internet stff there's a lot of posed movies out there this is the only the only one I found from from malter Michelle himself that's a real movie

**[39:59]:** of real kids really doing this I don't know why all the ones are posed on the out there but uh here's something astounding okay so these are all pretty similar children children of faculty and uh you know children varied how long they could wait and if they took for example the children who didn't wait very long they could only wait 30 seconds until they ate the cookie and they compared it to children who could wait for the full 15 minutes to eat the cookie these are four or fiveyear olds when they looked at them at 18 and they SATs uh they got the children who would wait 15 minutes scored over 200 points higher which you know is a lot that was in the days we just had two Sat tests it was 1,600 okay uh uh and by many many measures uh in terms of uh educational attainment in terms of health and many

**[40:44]:** measures the longer you waited at age four or five the healthier you were the better SAT scores you were the further you went in school astounding correlations with real life stuff uh and people are stunned by this uh you know partly it's again like who knows what goes into that at four or five how much is the influence of parents how much is the influence of cultures and genes who knows but that fateful moment when the child goes for the marshmallow or not correlates with incredibly important outcomes uh in the future so um let's see here we

**[41:23]:** go and you know right you know in college I remember personally like it always was a battle like am I going to have fun or study am I going to have fun or study there's so many chances to have fun and there's so many pressures to study right um well studying can be fun learning is fun okay how about when rewards harm again these are things that are not strictly predicted by conditioning uh I have three or four more minutes so uh rats love to run if you had a hamster or anything like that they love running Wheels what if you give them every time they run a food reward okay um then it turns out rats in a long run for fun they only run when the food is G is is is given okay if you start to extinguish they stop running so what had been a natural thing that seemed like a

**[42:07]:** whole lot of fun if it became the price you pay for a reward all of a sudden becomes a chore okay uh how about a widely studied cited experiment for um Mark leper he took preschoolers so four-year-old children who love to draw lots of preschoolers love to draw then they started to give them every time they drew a gold star you know every time they get a reward what happened when they stopped giving the gold stars the kids stopped drawing this is incredibly often cited in incredibly interesting debates about whether children in impoverished uh uh schools should be given cash rewards for studying hard okay there's lots of debates about whether it's a good thing or not a good thing an effective things are not an effective thing the study is often cited for arguing if you make something that you should love like learning purely rewarded the minute the reward stops it becomes a

**[43:01]:** complete chore okay and you won't but we don't know how that applies okay uh because we might read about things like baseball player salaries and it seems like there's a pretty good relationship between performance and salary right so uh it's it's not simply that way in the world we know that often rewards are associated with high performance and promote high performance it's pretty complicated the last slide um is about a fantastic debate that occurred here in many ways between BF Skinner at Harvard and gome Chomsky at MIT which is in what sense is language learned by conditioning so uh here's here's an example against it at one month there's a switch inside a rubber nipple hooked to a tape recorder and when the baby suck the tape plays this allows the baby to perform to behave uh and they would play uh syllables from different languages and they may or may not be in their

**[43:56]:** own languages um uh and uh by four days babies prefer the languages that they are exposed to um and one of the questions is uh people said well how do we get kids to learn language from a Behavior's perspective so one of the things that they studied uh uh children and they often young children will say wrong things like Mama isn't a boy he a girl that's not good grammar right and so there was an intuition that parents would then go that's not good gramar and sometimes parents do that but when they did studies where they recorded you know long sessions with parents doing this parents almost never corrected it was very rare okay you may feel different from your own R but statistically it's very rare to collect to you know they often say that that's right or something like that so you know how is the child picking up language if nobody's giving

**[44:48]:** them rewards and instruct in terms of whether they're right or wrong uh the idea is that we can say an infinite number of sentences well how can we possibly be rewarded or conditioned for an infinite variety of sentences how could that work um uh and and finally phenomenon of overgeneralizing you may know that children will overgeneralize things like in because we usually and Ed to make a verb past tense right um uh so but some verbs don't work that way so you would say my teacher held the rabbit children often make these overgeneralized rules they say my teacher holded the rabbit so obvious they learning a rule and then applying it to an instance that they never hear because no almost no adult says that so they must be producing that not on the basis of conditioning but Chomsky argued on the basis of sort of genetically prepotent um um ways in which we learn

**[45:44]:** language rules that we develop and figure out language in the world I will say that for those of you who happen to be interested in this topic there's been a striking push back on that uh in what people call statistical learning uh so so as much as this seemed like a done deal 10 years ago I'm looking at there's a lot it turns out the truth might be somewhere in the middle with some genetic pre potentiation but also a lot of learning from your environment uh uh both so um anyway so you can think as you leave here you know what are the Applause that you listen to where's your free will and your choice of uh what goals you pursue and what matters to you and how do you you know how does the world attempt to teach you uh things that you learn about what you want to do and why why you

**[46:27]:** want to do them
