Transcript

the following content is provided under a Creative Commons license your support will help MIT open courseware continue to offer highquality educational resources for free to make a donation or view additional materials from hundreds of MIT courses visit mitop courseware at ocw.mit.edu today we are introducing an exciting new pledge in 6034 anyone who has already looked at any of the neural net problems will have easily been able to see that even though Patrick only has them back up to 2006 now there still are well out of four tests perhaps two or three different ways that the noral Nets were drawn our exciting new pledge is we are going to draw them in a particular way this year and I will show you which way assuming that this works yes we are going to draw them like the way on the right the one on the left is the same as the one on the right at first not having had me explain the um difference between the two of them you might think you want the one on the left but you really want the one on the right and I'll explain why the 2007 quiz was drawn roughly similarly to this also if you somehow wind up in tutorial or somewhere else doing one of the older quizzes a lot of them were drawn exactly like this in this representation one thing I really don't like is that the inputs are called X's and the outputs are called y's but like there's two x's so the inputs are not X and Y and then they often correspond to axes of a graph and then people get confused additional um issues that many people have are the fact that the summation and multi the multiplication with the weight is implied the weights are written on the edges where outputs and inputs go and the summation of the two inputs into the node are also implied but take a look here this is the same net these um these W's here would be the W's that are um these W's here would be the W's that are written into um onto these lines are here actually the better way to draw it would be like see better way to draw it would be like so since each of these can have their own W which is different so each of the W's that are down here are being explicitly sent through a multiplier whereas here you just had to remember to multiply The Weight by the input that was coming by here you see an input comes to a multiplier you multiply by the weight then once you've multiplied all the inputs by the weight you send them through a sum that's the sigma is just a sum you sum them add them all up together send the result of that into the sigmoid function our old buddy 1 over 1 plus e negative whatever our input was with a weight for an offset and then we send the result of that into more multipliers with more weights more sums more sigin so this is how it's going to look like on the quiz and this is a conversion guide from version you know.

n beta into version 1.0 so if you see something that looks like this on one of the old quizzes that you're doing see if you can convert it and then solve the problem chances are if you can convert it you're probably going to do fine so we'll start off not only with this conversion guide but also yeah I'll leave that up here also we're going I'm going to work out the formula for you guys one more time the these are all the formula that you're going to need on the quiz and then we're going to decide what will change in the formula if and this is a very likely if there seems to be a good amount of times that this happens if the um the sigmoid function in those neurons up there was ever changed into some other kind of function H it's changed into a plus already in the problem we're about to do people change it all the time into some Bizarro function I've seen arct tangent I think so here we go let's look at the formula first of all sigmoid well our old buddy sigmoid I just said it a moment ago sigmoid is 1/ 1 + e- x also fun fact about sigmoid um the derivative of sigmoid of sigmoid is itself uh the derivative of sigmoid is um let's say that the sigmoid we we'll just turn sigmoid into like the letter say y y is the result right so if you say y = 1 / 1 + e tox then the derivative of sigmoid is y * 1 - y you you can also write out the whole nasty thing it's 1 over 1+ e x * 1 - 1 over 1+ e tox it's a nice property of sigmoid it's going to be important for us in the very near future and that future begins now now um so now the performance function this is the function we use to tell our um neural Nets when they inevitably act up and give us really crappy results uh at first uh we tell them how just how wrong they are with our performance function the performance function can be any sane function that gives you a um that gives you a better score where better can be decided is lower or higher if you feel like it um but gives you a better score if your answers are closer to the answer you're looking for uh however in this case we have with a very sneaky for very sneaky reason uh chosen the performance function to be 12 D which is the desired output minus o the actual output squared so we want a small um well it's negative so we want a small negative or zero that would mean we performed well so why this well the main reason is uh DDX of performance is the two comes down the O is the variable that we're actually ding so maybe I should say DDO um that negative comes out we get a simple D minus L and yeah we're using derivatives here so so that's those are fine these are two assumptions they could be changed during the test we're going to figure out what happens if we change them if we change the performance if we change the sigmoid that is if we change the sigmoid to some other function what's going to happen to the next three functions which are basically the only things that you need to know to do back propagation so let's look at them first W Prime this is the formula for a new weight after one step of back propagation a new weight in any of these positions that you can see up here on this on this um beautiful neural net that W all each of the W's will have to change step by step that's in fact how you do the hill climbing neural Nets you change the weights incrementally you step a little bit in the direction towards giving you your desired results until eventually you hope you have an intelligent neural net and maybe you have many different training samples that you run it on in a cycle um hoping that you don't overfit to your one sample on a computer but on the test we won't probably won not do that so let's take a look at how you calculate the weights for the next level given that you have the weights for the current level so first things first new weight weight Prime equals starts with the old weight that has to go there because otherwise we're just going to jump off somewhere at random we want to make a little step in some Direction so we want to start where we are with the weight and then we're going to add three things so if we're talking about the weight between some I and some J here are some examples of the names of Weights so this is W1 I that's the weight between one and a oh so this is W1 a it's the weight between one and a this is w2b which is the way between two and B makes sense well it makes sense so far but what what what if it's just called WB then it's the way between the these W's that only have one letter we'll get to later they're the bias they're the offset they are always attached to a negative one so you can pretty much treat um you can pretty much treat there as being a like a negative one here that is then fed to a multiplier with this WB if you like this is implied to be that all of the offsets are implied to be that so w+ sum Alpha why is it a Greek letter where does it come from how do we calculate it well Alpha is just some um value told to you on the quiz you'll find it somewhere there's no way you're going to have to calculate Alpha you might be asked to try to give a say in Alpha but probably not Alpha is supposed to give it the size of our little steps that we take when we're doing hill climbing very large Alpha take a huge step very small Alpha take tentative steps so Alpha is just there to change basically to change this answer in um make to make the new value either very close to W or very far from W depending on our taste so plus Alpha times um I so I is the value coming in um yeah I is the value coming into the into the node we're changing the weight here so I is the I is the value for instance I sub one here I would be the I would be the value at wac I would be the value coming output of w of um of node a at WBC I would be the output of node b i is sometimes specifically label as I I is the input coming in to meet that weight at the multiplier and then it's multiplied by Delta J where Delta is the Delta that belongs bels to these neural net nodes what is the Delta you say finding you may ask is a strange Greek letter it sort of comes from the fact that we're doing some partial derivatives and stuff but the main way you're going to figure out what the Deltas are are these two formula that I have not written in yet so hold off on um um trying to figure out what the Delta is until well right now I'm about to tell you what the Delta is so the Delta is basically think of the Delta as using partial derivatives to figure out which way you're going to step when you're doing hill climbing because you know when you're doing hill climbing you like look around you figure out okay this is the direction of the highest increase and then you step off in that direction so the Deltas are sort of telling you which way to step with the weights and the way they do that is by taking the partial derivative of basically you try to figure out how the weight that you're currently looking at is contributing to the performance of the net contributing to either the good performance of the net or the bad performance of the net so when you're when you're dealing with like when you're dealing with the weights like WBC and waac that pretty much directly feed into the end of the net they feed into the last node it then comes out it's the output okay then that's that's pretty easy you can tell exactly how much those weights and the values coming from them are contributing to the end and we do that by first we do that by essentially remember we want the partial derivative so partial derivative here is um in fact the is in fact the way that the final weights are contributing to the performance is just the performance function partial derivative well I've already figured out the derivative here it's just D minus o this is for this is for sort of final weights the weights in the in the last level D minus o it's that we're not done yet because we're when we do derivatives remember the chain rule to get from the end to these weights we pass through well should be a sigmoid here it's not we're going to pretend it is pretend it is for the moment we pass through a sigmoid and since we pass through the sigmoid we better take the derivative of the sigmoid function that is uh y * 1 - y well what is y what is the output of the sigmoid it's O so that's also multiplied by o * 1 - o however there is a uh let me see let me see yes sorry I'm I'm carefully studying the sheet to make sure my Gomen clature is exactly right for our new nomenclature which so new and brave that we're going doing it that we only knew for sure we were going to do it on Wednesday so we have D minus o * o * 1 -0 so you say that's fine that can get us you know these weights here even this WC how are we going to get the um how are we going to get the Deltas for the the Deltas for the new the Delta for the new weights here oh I realize um yeah yeah I got it so the Delta this is the by the way this is the Delta at C how is Neuron C contributing to the output well it's directly contributing to the output and it's got a sigmoid in it it doesn't really but we're pretending it does for now D- o * o * 1 - o what about an inner node node B node a what are we going to have to do well the way they contribute to the output is that they contribute to node C so we can do this problem recursively so let's do it recursively first of all as you probably figured out all of them are going to have an O * 1 - o factor in from the chain rule because they're all sigmoids well pretending that they're all sigmoids so we actually have a dir of good problems that are actually sigmoid on U on the web right now there's only 2007 but um here's o * 1 -0 so what are we going to do for the rest of it how does it contribute to our final result well it contributes to our final result recursively so we're talking about Delta I I is an inner is sort of an inner node it's not it's not a final node it's somewhere along the way so sum over J of w going from I to J uh times Delta J now some overall J J such that I leads to J I need to have a direct path into J so if I if I in this instance was a everyone the only possible J in this would be C that's right we would not sum over um b as one of the J because I does not lead to to b or a does not lead to B A only leads to C also note that c does not lead to B here that's going backwards so you just to figure out which J you're looking at look directly forwards at the next one so if there was another D here or something like that a does not go to D A goes to C you only look at the next then you look at the next level's children and you sum over all of those all of those children the weight between them multiply by the child's Delta that makes sense right because the way we affect if if the child's Delta is the way the child affects the output calling these children for a moment if the then if this this one directly affects the output then the way this one affects it is you use the uh it affects it because it affects this but it's also multipli by this weight so in fact for instance if the weight between a and C was Zero then a doesn't affect the output at all right because its weight is zero and when we do this problem we go this times zero and then we try to add it in there it doesn't affect anything if it's weight is very high it's going to really dominate um it's going to really dominate it see and that is taken into account here and then multiply by the Delta for the right node so I pose the following question and since I spent a lot of time with formula and not that many time that much time um starting on the problem I will not call on someone at random but rather take a volunteer and if no one volunteers I'll eventually tell you which is we've got some nice formula for on the bottom three if we change the sigmoid function what has to change that's right the only thing that changes in this crazy ass problem right here which by the way changes the sigal functions it's adders is that we take all of the O * 1 - o in Delta Alpha and Delta I equation and we change it to the new derivative we then do the exact same thing that we would have done correct and on a similar note if we change the performance function how many of these equations at all have to change all on the bottom three yeah that's right just one just Delta F take the D minus o make it the new derivative of the new performance function and in fact Delta I doesn't change at all does everyone see that because it is very common for something to be replaced I think three of the four quizzes that we have replaced in some changed something in some way all right let's go we're going to do 2008 quiz because it has a part at the end that screwed up everyone and so let's make sure we get to that part that's going to be the part that you probably care about the most at this point so these are all adders instead of sigmoids that means that they simply add up everything as normal for a normal noral net and then there's no sigmoid threshold they just give some kind of value question yeah um so we talking about the A and are the multiplier things we don't count those they are not neural net nodes that is one of the reasons why one of the reasons why that other form that you can see over there is elegant it only has the actual nodes on it it's very compact it's one of the the forms we've used used in previous test the question is do those multipliers count as NOS um however by not putting in the multipliers we feel it sometimes confuses people of explicitness the ones that are nodes will always have a label like a or here you see there's a sigmoid and an L1 um the multipliers are there for your convenience to remind you to multiply and also those if you look those sigmas that are over there you're there for your convenience to remind you to but in fact the only thing that counts as a node in the noral net and that's a very good question is usually the sigmoids here it's the adders we've essentially taken out the sigmoids these adders are the um the oh here's the way to tell if it's got a threshold weight associated with it then it's one of the actual nodes a threshold weight I guess the multipliers look like they have a weight but this is just the weight that's being multiplied in to um this is a weight that's being multiplied in with the input but if it has a threshold weight just like wa WB oh I promised I would tell you guys the difference between the two weights so let's do that very quickly the kind of Weights that that say w2b or W1 a our weight that comes between input one and a or between a and C they're meant to be multiplying the input by this weight and then eventually that's that's added together the threshold weights they just have a like WB wa a WC they are essentially to decide the threshold for a success or a failure for a one or a zero um or anything between at any of the given nodes so the idea is maybe you at some node want to have a really high cut off you have to have a very high value coming in or else this is zero so you put a high threshold the weight is multiplied by1 and in fact in fact um the threshold weight one you one could consider if you wanted to that the threshold weight time-1 was also added in at that sum instead of putting it um at the same location as the node if that works better for you when you're converting it you can also think of it that way because the threshold weight is essentially multipli by ne1 and added in it that same sum over there so that's that is another way to do it um there's a lot of ways to visualize these noral nets just make sure you have a way that make sense to you and that you can tell pretty much whatever we write as long as it looks vaguely like that how to get it in your mind into the representation that works for you because once you have the representation right for you you're more than halfway to solving these guys they aren't that bad they just look nasty they don't bite okay so these are just adders so if it's just an Adder then that means that if we take all the we take um all the X inputs coming in let's do X and Y for the moment so we can figure out the derivative then what comes out after after we just add up the X what comes out Y equals X right we're just adding it up all the adding up all the input we're not doing anything to it Y equals X is what this node does did people see that so the derivative is just one so that's pretty easy because the first problem says um what is the new formula Delta F so I'll just tell you you guys probably figured it out it's O * 1 - o cuz we replace D minus o with one okay Mak sense so far please ask questions along the way because I'm not going to be asking you guys I'll do it myself question why repus one oh um that's a good question the reason is because I did the wrong thing so see it's good that you guys were asking questions um it actually should be replacing o * 1 - o with one the answer is Delta f equals D minus o so yes perhaps I did it to trick you no I actually messed up um but yes please ask questions along the way again I don't I don't have time to call on you guys at random to figure out if you guys are following along so I'll do it myself we're replacing the O * 1 - with 1 because of the fact that the sigmoid is gone and we get just Delta FAL D minus l so great we uh we now want to know what the equation is for Delta I at the node a so Delta a well let's take a look the O * 1 - is gone now we just have the sum over J which is you guys already told me is only C of wac time Delta C and we know that Delta C is D minus o so the answer is Delta a is just w a c * D minus o that time I got it right I see the answer here though it's written in a very different format from the old quiz any questions on that well that's part A that we finished out of C that's go to Part B Part B is doing one step of back propagation there's almost always going to be one of these in here so the first thing it asks is to figure out what the output O is for um this normal net if all weights are initially one except that this guy right here is5 all the other ones start off as one so let's do a step oh what Also let's see what are the inputs the inputs are also all one this is desired output is also one and in fact the rate constant Alpha is also one this is the only thing that isn't one folks so let's see what happens 1 * 1 is 1 then this is - 1 * 1 isga 1 that's zero the exact same thing happens here because it's symmetrical so these are both zero 0 * 1 is 0 0 * 1 is 0 then this is-1 * .5 is pos5 so 0 + 0 + pos5 the output is pos5 does everyone see that if not um you can convince yourself that it's positive .5 that would be a good exercise for you running through one forward run the output is definitely positive .5 first time around okay now we have to do one step of back propagation to do that let's calculate all the Deltas so that we can calculate all the new weights the new weight primes so Delta C that's easy you guys can tell me what Delta C is we figured out what the new Delta C is going to be it's a simple addition or subtraction problem everyone Delta C is five2 yes all right and we know that Delta a and Delta B are just W A Time Delta C WBC * Delta C so they are also 12 because all the weights were one Easy Street okay we've got all the Deltas are 1/2 and all but a few of the weights are one so let's figure out what the new weights are so okay new wac okay so yeah so let's see what's going to be the new wac so the new wac is going to be old W uh uh so old wac which is one because all of them are one except for WC plus the rate constant which is one times um the input coming in here but remember that was zero so at actually it's just going to be the same as the old wac and since it's this is a symmetrical problem between b and a at the moment uh this is going to be the same all right some things are going to change though what about WC that was the one that was actually not one okay so new WC WC uh remember the I for WC the the um the I that we use in this equation is always ne1 because it's a threshold so we have the old WC which is .5 + 1 * uh + 1 * - 1 * Delta C which is 12 so we have .5 +5 equals 1 W1 a well we've got W1 a starts as one then we also know that um we're W1 a is going to be equal to 1 + 1 time the input which is 1 time um Delta of a which is 1/2 so 1.5 and since it's symmetrical between a and b um then w2b is also 1.5 and then finally W and WB the offsets here well they start at 1 + 1 * -1 * .5 so they're they're both everyone one half that's right that's right because negative 1 is their I negative 1 * 12 plus positive one is just2 that's one full step maybe it might easier than you might be used to seeing but that's a full step and it asks what's going to be the output after one step of back propagation while we can take a look so we have 1 * the new W1 a which is 1.5 we've got 1.5 then the new wa is just .5 -.5 that's a one coming into an Adder we've got another one coming in here because it's symmetrical so one and a one 1 * wac is 1 1 * WBC is 1 so we have two ones coming in here they're added that's two then this has become -1 in fact at this point so 1 * 1 that's three and the output is three all right cool we've now finished Part B which is worth over half of everything oh no we've not one more thing these are adders they're not sigmoids what if we train this entire neural net to try to learn this data so that it can sort of draw a line on the graph or draw some lines or do some kind of learning to separate off the minuses from all the pluses you've seen maybe and if not you're about to in a second cuz they it ask you to do this in detail that Neal Nets can usually draw one line on the graph for each of the sort of nodes in the net because each of the nodes has some kind of threshold and you can do some logic between them like ands and ores so what do you guys think this net is going to draw if the anyone could volunteer I'm not going to ask anyone to give this answer no okay this is a little bit tricky because usually if you had this many nodes you could easily draw like a box and box off the minuses from the pluses however it draws this and it asks what is the error the error is oh yeah it even tells you the error is 1/8 because why these are all adders you can't actually do anything logical because this entire net boils down to just one node because it just adds up every time and it takes a threshold at any point so you you can't turn it into logical ones and zeros because it's basically it's not digital at all it's analog it's giving us some very high number so it all blows on to one cut off and that's the best one the one that I drew right here okay did that not make sense to you that's okay this problem is much harder and putting them both on the same quiz was a bit was a bit brutal but by the time you're done with this this you will understand what a noral net can do or not I put these in simplified form because of the fact that we don't care about their values or anything like that but inside of these little circles as a sigmoid the multipliers and the summers are implied just so that I think in the in the simplified form when we're not actually doing back propagation it's easier to view it and see how many nodes there are for the same reason you asked your question about how many there are so all of those big circles are a node and in those node is a sigmoid now not those crazy adders we have the following problem we have to try to match each of ABCDE EF to 1 2 3 four five stations using each of them only once that's important because of the fact that some of the more powerful networks in here can do a lot of these so it's like yes the powerful networks could do some of the easier problems here but we want to match each net to a problem it can do and there is exactly one mapping that will map with that is one to one and Maps um exactly uses all six of the Nets to solve all six of these problems here so some of you may be looking at like what how am I going to solve these problems I gave away a hint before which is that each node in the neural net each sigmoid node can usually draw one line on the it can draw one line into the picture the line can be diagonal if that node receives both of the inputs which here i1 and I2 you see there's an i1 and an I2 axis like X and A Y axis the node has to be horizontal or vertical if the uh sorry the line has to be horizontal or vertical if the node only receives one of the inputs and then if you have a deeper level these these secondary level nodes can sort of do a logical can do some kind of Boolean thing like and or or of the first two which can help you out all right and so let's try to figure it out so right off the bat and I'll hope that people will help and call um call this out because I know we we don't have enough time that I can force you guys to all get it right off the B which one of these looks like is the easiest one six that's great six is definitely the easiest one it's a single line so this is just how I would have solved this problem is find the easiest one now which of these the crappiest net a a is the crappiest net in fact there's no way in hell that a is going to be able to get any of these except for six so let's WR off the bat say that six is a all right six is a that's a we don't have to worry about a okay cool now let's look at some other ones that are very interesting all the rest of these draw two lines ex well these three draw two lines these three draw three lines they draw triangle so despite the fact that this C is a very powerful neural net indeed with three whole levels here of sigmoids it looks like there's only two Nets in our little stable of Nets that are equipped to handle number one and two and those are and F because and F have three nodes at the first level they can draw three lines and then they can do something logical about those lines like for instance maybe if it's inside all of those lines there's a way to do that you just basically can give negative and positive weights as you so choose to make sure that it's under certain ones above other ones and then make the threshold such that it has to follow all three of your rules so between E and F which one should be two and which one should be one anyone see well let's look at two and one which one of these is easier to do between two and one two it's got a horizontal and a vertical one has all three diagonal and which one of these is a weaker net between and f f f f has one node that can only do a horizontal and one node that can only do a vertical line so which one is f going to have to do two and E does one good job guys good job guys you got this so now let's look at the last three number three is definitely the hardest it's an exor those of you who have played around with2 kind of stuff or even just logic probably know that there is no way to um to make a sort of simple linear combination of um in one level of logic to create an exor exor is um very difficult to create there are um some interesting problems involving trying to uh trying to teach an exor to a noral net if the noral net is simple enough it's just not going to be able to get the xor because of the fact that you can tell it okay I want this one to be high and this one to be low that's fine you can say these both have to be high that's fine but it's hard to say like it's pretty much impossible to say this one or this one but not the other because of um need to be high in a single node because of the fact that if if you just just play with it you'll see you need to set a threshold somewhere and it's and it's not going to be able to distinguish between if the threshold is set such that the ore is going to work the whole or is going to work it's going to accept when both of them are positive as well so how are we going to do an X or we need more logic we need to use some combinations of ends and ores in a two-l way and to do that we need the deepest noral net that we have there's only one that's capable of that and that is it's C so you there are many different ways to do let's think of a possibility i1 and I2 draw these two lines or sorry not i1 these two let's call these one two three four five of node one and node two draw these two lines I'll just sort of draw it here for you guys then maybe node three gives value to um yeah let me see node three can give value to perhaps no let's see node three can give value to everything that is oh I mean there there there are a lot of possibilities here node three could give value to everything that is up here actually why don't no three could give value to everything except for this bottom part and then um node four or could give value to say oh no doesn't do it yet but there's a few there's there's a few different ways to do it if you play it around the key um the key idea is that node three and node four can give value to some combination of and or or not and um then node five can give value based on um based on being above or below a certain threshold combination of three and four you can build an ex or out of the logic gates I will I will ponder on that in the back burner for a moment um as we continue onward but clearly C has to do number three okay now we're left with four and five I think intrinsically five looks like it may be more complicated than four because of the fact that it need it needs to do both different directions instead of two of the same direction however there's um and so however just the idea of the one with the fewer lines being the simpler one may not get us through here and there's a reason why look what we have left to use we have to use D or B what is the property of the two lines that D can draw D being the simpler one one horizontal and one vertical that's right so even though it may look simpler to just have two horizontal lines it actually requires b b is the only one that can draw two horizontal lines because D has to draw one horizontal and one vertical so that leaves us with B on this D on this and excellent we have a question I would have thought it would been possible we had no questions or maybe I just explained this the best I ever have question I get why has to be to horizontal uh all right so the question is I don't understand why B has to be two horizontal lines the answer is it doesn't um B can be anything but D can't be two horizontal lines and so by process of elimination it's B well take a look at d right so D has three nodes one 2 3 node one and node two can just draw a line anywhere they want involving the inputs they receive what input does node one receive which inputs go to node one I only i1 so it can only make a cut off based on i1 so therefore for it can only draw by making a cut off above or below I a certain point i1 it can only draw vertical line node two can only draw a horizontal line because it can only make a cut off based on where it is in I2 therefore they can't both draw a horizontal that's why this is the trickiest part this last part because B is more powerful B does not only have to do two horizontal lines it can do two lines two diagonal lines it can do anything it wants it just happens that it's stuck doing this somewhat easier problem because of the fact it's the only one left that has the power to do it so let's see we've done we're done and we have aced this part of the quiz that like no one got well not no one but very few people got when we put it on in um 2008 the only thing we have left to ask is let me see the only yeah the only thing we have left to ask is what are we going to do here for this all right let's see um for for the exor let's see if I can do this xor so okay um how about this one ready I'm an idiot this is the easiest way number one draws this line number two draws this line number three and the line the two lines number three says only if both of them are true will I accept number four knots the two lines and number five ores between three and four thank you no it's not that hard I just completely blank there's because there's another way that a lot of people like to do it that involves drawing in a lot of lines and then making the cut off B two but I can't remember it at the moment are there any other question because I be I think that if you have a question now like four other people have it and just aren't raising their hand so ask any questions about this drawing thing question why do we do this why do we do this drawing thing that that a very good question the answer is so that you can see what kinds of Nets you might need to use in these simple problems to um to answer these simple problems so that if um you know Athena forbid you have to draw use a noral net in a in a job somewhere to do some actual learning and and you see some sort of quality about the problem you know not to make a net that's too simple for instance and you wouldn't want a net that's more complex than it has to be so that you can sort of see what the Nets do at each level and and more visibly understand I think a lot of the people who drew problems like this just want make sure people know oh yeah it's not just these numbers that were mindlessly back propagating from the other part of the problem to make them higher or lower this is what we're doing at Easter level this is what we're this is the space that we're looking at each node is performing Logic on the steps before so that if you actually have to use a neurl net later on down the road well then you'll be able to you'll be able to figure out what your next going to need to look like you'll be able to figure out what it's doing at least as as well as you can figure out what it's doing for a neural net since it often will start getting out these really crazy numbers you'll have all sorts of nodes in a in a like a real noral net that's being used nowadays there will be tons of nodes and you'll just see the numbers fluctuating wildly and then suddenly it's going to start working or not uh that's a good question any other questions we still have a few minutes not many but a few any other questions on any of the stuff question sorry this question on what she just asked I'm just completely confused as to why a machine needs to learn by drawing pictures like that like ask you're confused why the machine is learned by what by the pictures on the right oh okay machine does not have to learn by drawing pictures and calling them in let me give you some real applications my friend at University of Maryland recently actually used noral Nets um because yeah he he actually did um because of the fact that he was doing a um a game plan competition where the game was not known when you were designing your AI it had to be able to there were some very elegant General game solver thing that you had to be able to hook up into and then they made up the rules and you had a little bit of time and then it started so some of the AIS what they did was they trained once they found out what the rules were on their own with the rules in his case he had a noral net because it was so generic you just have a web of random stuff random goop he thought it could learn anything and then um he never did tell me how it went probably didn't go well but um maybe it did um it it basically tried to it it tried to learn some things about the rules some of the other people who were more principled game players actually tried to find out fundamental properties of the space of the rules by testing a few different things so that they could you know more knowledge is less search so that they could do less search when the actual game plan came on and then when the actual game plan came on pretty much everyone did some kind of um game tree based stuff he telling me that a lot of Monte Carlo based um game stre stuff that is just very non-deterministic is what they're doing nowadays rather than our more deterministic Alpha Beta although he said it converges to Alpha Beta if you give it enough time that's what he told me but that's someone I know who is using Neal Nets I've also in a cognitive science class I took saw Neal Nets that tried to attach like qualities to objects by having just this huge huge number of nodes in levels in between and then eventually it was like a duck flies and you're like how is it doing this again I'm not sure but it is so the basic idea is that one of the main reasons that were used so much back in the day is the people on many different sides of this problem cognitive science AI whatever were all seeing wait a minute there's networks of neurons and they can do stuff and we're seeing it in different places and when you see it in so many different places at once must be a genius idea that's going to revolutionize everything and so then everyone started using them to try to connect all these things together which I think is a noble Endeavor but unfortunately people just stopped using it it didn't work as as they had wanted it turned out that figuring out how our Norms worked in our head was not the um the way to solve all AI hard problems at once and they fall into disfavor although are still used for some reasons like U some reasons like that so we wouldn't use it just to draw these pictures the reason why we have these pictures is because we give you simple Nets that you can work out by hand on the quiz um any net that was really used nowadays would make would make you your head explode if we tried to make you do something with it on the quiz it would just be horrible looking so I think that's a good question if there's no other questions or even if there are cuz we have to head out um if there's any other questions you can see me as I'm walking out