# Mega-R5. Support Vector Machines

**Channel:** MIT OpenCourseWare
**Source:** https://www.youtube.com/watch?v=6nDqY8MPLDM
**Transcript page:** https://www.withtranscript.ai/video/6nDqY8MPLDM

## Chapters

- 0:00 — Chapter 1 (0:00)
- 5:23 — Chapter 2 (5:23)
- 8:29 — Chapter 3 (8:29)
- 9:57 — Chapter 4 (9:57)
- 14:58 — Chapter 5 (14:58)
- 20:44 — Chapter 6 (20:44)
- 25:50 — Chapter 7 (25:50)
- 28:10 — Chapter 8 (28:10)
- 31:31 — Chapter 9 (31:31)
- 40:12 — Chapter 10 (40:12)
- 42:56 — Chapter 11 (42:56)
- 44:15 — Chapter 12 (44:15)

## Transcript

**[0:00]:** the following content is provided under a Creative Commons license your support will help MIT OpenCourseWare continue to offer high quality educational resources for free to make a donation or view additional materials from hundreds of MIT courses visit MIT opencourseware at ocw.mit.edu hope everyone had a great Veterans Day break yesterday spending it of course as I checked my tests on Ian's spending it mostly doing your opera research or psets but at least one person got to watch TV so at least one person got to have a real break and that's something truly amazing and special so now we're going to talk about svms they're pretty much the hardest thing in 6:54 however in recent in recent years a few shortcuts have popped up that will sometimes allow you to solve the question depending on what they're asking for without solving some vast ugly set of equations with the vast Uglies number of unknowns so I'm going to show that

**[1:11]:** to you guys I'm also going to try to explain all the alphabet soup that's in neural nets and what all the letters stand for because I took me a few times going to North go to the S campus it took me a few times going through SVM's to actually find out for sure what all those letters stood for and if you guys figure it out first try that's going to be great and you guys will be just fine so let's take a look at the problem is perhaps most optimized for using some of the shortcuts to solving it and not putting up all the equations then I will because I am not because I'm sadistic but because I'm being nice I will force you with me to solve some of the things they didn't ask for us to solve so that you can see that we can't get away with everything without doing some of

**[2:09]:** the harder stuff and of course definitely ask questions as always but this time even more so you guys well if you were looking around you saw nobody in this entire lecture hall raise their hand that they're already set and ready and no svm so if you have a question maybe everybody else does so let's go you'll start right here as always pretend that I can draw and therefore all the pluses and minuses are only on integer coordinates so we are asked in this problem to circle the support vectors draw the edges of the street and then the dotted line in the middle that separates them the separator as a dashed line and then to give both W and B so what are W and B well there's a few important equations in in svms that we really hope and I'm going to tell you we're lucky in this because we don't have to but we

**[3:25]:** really hope that we don't have to use because they provide a huge number of variables so one of those crucial equations is that for a spur a plus support vector W vector dot X plus the plus support vector plus B equals 1 WX minus plus B with minus 1 and W dot that dotted line I don't know we'll call it dot dot plus B equals 0 so what does this mean there are a lot of vectors well I mean we're usually in two-dimensional space so we can basically just say that there's two components of this W vector W 1 and W 2 and they're just two coefficients and a linear equation so to get for instance what we're interested in finding this that dot line we'll just call that X with nothing on it actually maybe that'll be easier this is equivalent to saying W 1 X 1 plus W 2 X 2 plus B equals 0 where X

**[4:45]:** 1 is this and X 2 is this we would possibly call them x and y so one way to think about it is you know W 1 we call a a x+ called W to be B Y oh don't call it B well a X plus CY plus B equals put this on parenthesis this is basically an equation like this or you know y equals

**[5:23]:** negative a over C X minus B over C it's basically y equals MX plus B does everyone see that this thing that we're looking for this w dot X plus B equals 0 is the equation of a line on in Cartesian coordinates it just looks uglier normally when we're doing all this solving for W and B we'd have to put in equations with the you tons of equations plug in all the support vectors in there and we have to use these little devils called alphas which alphas essentially if it wasn't clear in the lecture which it usually isn't completely clear to everyone wasn't clear to me completely alphas way I'd like to think about them the Alpha is in this problem is they are the weight of how significant any particular point on the graph is towards creating the boundary this the higher the alpha is the more that that point narrows in the boundary

**[6:31]:** the lower the alpha is the less at that point narrows in the boundary the wider the road can be and if that point do anything if that point is irrelevant and could be removed it wouldn't affect the boundary the Alpha is everyone zero well that was one person but you can suffice for everyone the Alpha is zero and that means if it's not a support vector if it's not one of the vectors on the boundary lines it will always have an alpha of zero because it doesn't affect so keeping that in mind there's a few fun and important equations about alphas that will mean if we're solving many equations for many unknowns which hopefully we won't have to do the sum over the positive alphas equals the sum over and the alphas of the negative points this is true over all the points but since all the alphas are 0 except for the support vectors

**[7:29]:** it also means the alphas of the positive support vectors are equal to the alphas of the negative support vectors additionally our buddy the W vector is equal to the sum over the all I that R plus vectors of WI alpha I minus M over J minus vectors of W J alpha J now all these equations can be used in a bloody mess to figure out the answer to what we're trying to find which is circle as well actually they can't be used to circle of support vectors and draw the draw the dotted line but once we do that all these equations can be used in a bloody mess to give us the but the next thing that we want which is W and B

**[8:29]:** so fortunately there's another way to get W and B if you guys really want at the at the end of the hour we can also try to get try to drive W and B using the many equations and many unknowns but it's a bit painful we'll try to do it the cool way so let's start this is the one we're looking at we need to find what the support vectors are so first thing we need to do is simply I it fortunately on the test there will always be ones that you can I if you're supposed to circle the support vectors there's obviously some number of pluses and some number of minuses I say obviously but maybe not but hopefully obviously and we'll find out because I'm going to call them random people so give me a positive support vector

**[9:32]:** yeah perfect that's good all right that's what ah not give me a native support Oh

**[9:48]:** No ah

**[9:57]:** that is a good question question is what is a support vector how many other people are will meant to having this question see you're not alone okay

**[10:14]:** before I go on I'm going to assume guys make sure I'm correct money was uh just being served so I can tailor based on this Monday was the support vector machine lecture but it was also very difficult to follow that's what I that's what I usually expect so um what is a support vector well all these pluses and minuses if we were if we were me and we if I guess yeah if we were me and I was describing this problem the one that we work out in class I would call them points because they're on the graph their points their data points I would cut but however in more difficult versions of this problem that have n dimensions where n is some ridiculous number of dimension that you're never going to graph like say you know some of the research I'm doing now I could use support vector machines on some of these articles

**[11:11]:** that I'm reading about cyber events to try to figure out if there's a real event or if it's just someone complaining about how we're really vulnerable or something like that and no event actually happened so the reason why they call these guys vectors is when you're not able to graph them on a Cartesian plane there's still this long you know vector of many different dimensions right now though these points represent the vectors it's very simple it's easier to view them this way but for instance that plus at negative 1/2 represents the fact that there's a vector going in the direction of negative 1/2 with the magnitudes such that it reaches negative 1/2 so all these points are just a point representation in vector you probably in any class that worked with vectors saw this saw vectors being represented as points question hi the question is all respect to the origin answers economically when directions are

**[12:12]:** represented at points yes it's always good respect to the origin so that's the basic idea is that all these points are vectors so what are support vectors well we could call them support points for this case but the reason we call the support vectors is again in the generalized case that you might be doing in the real world with real AI you're going to have the giant vector it's not just going to be points on a graph well usually so the support vectors the support points we found one of them correctly it's this guy they're going to be the ones that again they don't have an alpha of 0 they're the ones that bind in the as petracles of the road the boundary lines they're going to be on the edge of plus whichever direction we draw it this plus is the edge of the plus region if we made this the edge of the

**[13:06]:** plus region and everything on this side is plus everything on this size - we'd be screwed because there's two pluses on the on the other side of that generally when trying to find support vectors you do something a little bit similar to my crazy method of doing nearest neighbors and try to find a plus my excuse me a plus minus pair that's close to each other sometimes though it's not just two points because sometimes if you try to draw the simple-minded thing which is sort of the perpendicular bisector of the two points you get screwed because there's another point in your way so now that I've given away a clue let's go and hopefully that made sense to you guys the support vectors are the ones on the edges they're just barely a plus for sure but barely - for sure let's go back can you give me a negative support vector the one on

**[14:00]:** Paulo yeah and does anyone think that there's a third support vector well let's simple-minded Lee try the thing that remember support vectors always attempt to have the widest possible space between the pluses and the minuses that they can so let's simple-minded Lee try to do the perpendicular bisector and see if it see if it screws us over so when we simple-minded Lee do the perpendicular bisector it goes through like here

**[14:35]:** like this and it's just fine so these are the only two support vectors

**[14:58]:** and there's our divider line so we're on the homestretch but we have to find W and B in olden days we would find W and B by plugging in W dot the plus support vector plus B equals one that's very crucial these W dot X plus X minus our only true equaling one or negative one are only true for support vectors it's always true that W dot any positive point plus B will be some positive number but it won't always be supportive it don't won't always be one in fact it will always be the greater than one up over here it will always be less than negative one down over there so in olden days we would plug in negative 1/2 into this equation we plug in 3 negative 2 into this equation we plug in alpha plus equals alpha minus it's the sums and since there's only one plus or minus we know that

**[16:02]:** they were equal and then we fidget around with this W equation however there is a better way to do it and so let's use this cheap strategy to solve to solve this version of the SVM here's how first and I know I didn't draw these completely straight sorry but can anyone by looking at the fact that this is 3 negative 2 and this is negative 1/2 can anyone tell me what the equation uh if you can do y equals MX plus B can anyone tell me what the equation of that dotted line is supposed to be if I was a good at drawing you said Y minus 1 and I say yes why I'm just - why so therefore the pluses would be y is greater than or equal to X minus X minus 1 indeed so we've already seen that WX plus B like somehow can be converted into this form right so therefore if we have y equals X minus 1 then we know that we have in w1 that we know that we have W X plus B equals zero let's

**[17:19]:** do that here so we know that W 1 W 1 X 1 being even call it X and why I think it'll be fine no one will come after us w 2 y plus B equals 0 but we also know that y equals x minus 1 which means that if y equals X minus 1 then that means that according to this thing that we have over here then negative w1 over w2 equals

**[18:00]:** so we know that negative w1 over w2 and we have negative negative B over w2 so y equals x minus 1 and if we solve this equation to make it look like this we would have y equals negative w1 over w2 minus B over w2 so we know that in some way shape or form that we know that Nick did then therefore w1 over w2 is some scalar multiple of minus 1 and we know that B over w2 is in fact some scalar multiple of positive one scalar multiple scalar multiple well why is it a scalar multiple why is it it just going to be negative 1 or positive 1 because just because in this equation we can multiply the entire equation by any number and it will still have the same boundary line you guys see that if you know if X here if we multiplied everything since it's all

**[19:25]:** divided by - if we double w - but also w BB and w1 it would be the exact same question do you guys agree so there's in fact the infinitely many possible equations you say well great mark you figured out what form it is so you figured out that w1 over w2 equals on some scalar multiple of negative one so it's negative one times what's everyone's favorite letter okay negative 1 times K and we figured out that B over w2 is I guess we can just do negative K is positive K but what's K how are we going to figure it out well it's a good question I will tell you how I'll assert the following fact as true without proof then I will not prove it 1 over the magnitude of W which is this vector here duck with w1 w2 equals this where this is that line that I just drew the line from here to this point

**[20:44]:** 1 over the magnitude of W equals this therefore since 1 over the magnitude of W equals this and this equals I believe 2 root 2 because we're going over to down to Pythagorean theorem 2 or 2

**[21:09]:** so therefore flip everything over magnitude of W equals 1 over 2 root 2 so therefore magnitude of W equals root 2 over 4

**[21:22]:** but why are we okay well how do we calculate the magnitude of W do people know in general magnitudes of vectors generally for these vectors we do it by thing the square root of the sum of the components squared so the square root of W 1 squared plus W 2 squared equals root 2 over 4 but that's not all that's not all we say because we know from this over here that the ratio of W 1 and W 2 is

**[22:06]:** yeah the ratio of the ratio of W 1 and W 2 is going to be actually sorry I shouldn't put a K here I realize I probably should have been confusing you guys a lot w1 over w2 is just negative 1 B over w2 is just 1 that's just the fact there's no K the K is to determine what W 1 and W 2 is w1 and w2 are so W 1 equals like negative K and W 2 equals positive K and then B equals also positive pay by the way here's a question for you could I have put the K the negative sign on a different I'm going to put the negative sign on W 2 and B instead of on W 1 so maybe people said yes that's a very strange actually not because of the fact that the pluses are on the negative x-axis it's just a little trick I picked up

**[23:11]:** when one of them is negative in the other one is it follow the pluses so we know that W 1 is negative K W 2 is positive K and B is positive K w1 over w2 is negative 1 B over W 2 is positive 1 so what do we know about the ratio of W 1 and W 2 it's equal to negative 1 and that means that when we square it W 1 squared equals W 2 squared so therefore this is the square root of 2 W 1 squared which equals root 2 W 1 equivalently well actually no it doesn't equal root 2 w 1 because W 1 is actually negative so it we you know it's negative 2 W when it doesn't matter the point is that if that equals root 2 over 4 then W 1 is everyone negative 1/4 bingo and if W 1 is negative 1/4 everything else falls into place

**[24:18]:** butter W 2 and B everyone positive 1/4 we got it we've done with this part of the problem however bonus let's do conquer the alphas which they didn't ask you to calculate actually you know what we'll do the office if we have enough time since they didn't actually ask you to calculate them however my recommendation is since there's only one alpha plus and 1 alpha minus they must be equal from this equation since this is some of the alpha plus equals the sum of the alpha minus and so therefore some of the W or say W equals the sum of W sorry should be an X of course there's not a million w's in this equation the sum of the points the positive data points times their alpha is minus the negative data points times their office so we're looking at here on negative 1/4 1/4 equals um what do we got here positive point

**[25:15]:** negative 1/2 so we've got alpha plus or alpha of that point negative 1/2 minus alpha of that minus point and what is that it's 3 negative 2 3 negative 2 so if both of the alphas which are equal were 1 we'd have negative 4 and we'd have negative 4 4 but we want negative 1/4 1/4 so actually both of the alphas are

**[25:50]:** 1/16 and that's the answer we'll do that boring gap if we have time but we won't so let's do number two so let's go into faster mode number two very similar to number one in many ways but as you can see one of the main things that they added an extra minus sign at two negative one so I think we can all agree that this will still be our plus actually add another plus sign there too so maybe this plus sign is a split vector but it's not this plus sign is the support vector what do you guys think about the new negative sign will it become the support vector since it is like strictly closer to the pluses yup you're right okay so this is a very beautiful division because if I do this correctly which I didn't but if we pretend that I did then the dotted line is y equals x

**[26:56]:** okay so with the dotted line at y equals x then we know that just like we did up here we know that if y equals x plus 0 we know that first of all B equals 0 second of all we know that we've got just with y equals x then we know that negative w1 over w2 equals 1

**[27:32]:** the plus are still on the left and up so we know that w1 is some negative number negative K w2 is some positive number K great how are we going to figure it out well let's call that this D for distance or whatever you want to call it so on over W equals D D in this case is not on 2 root 2 can everyone tell what D is here

**[28:10]:** it's actually so it goes over two and one so it should be one and a half root two since this with distance which is twice as much goes over 3 and 3 which is 3 root 2 so it's one and a half root 2 I don't like putting in decimals and stuff there so we'll say that 2 over negative W equals to D equals 3 root 2 so therefore right high-diving 1 2 3 1 2 3 3 root 2 so therefore magnitude of W equals let's see switch them over we should get root 2 over 3 and it's magnitude of W is root 2 over 3 we can do our same trick from before square root of 2 W 2 squared equals root 2 over 3 and this is just root 2 times W 2 equals root 2 over 3 so therefore W 2 is 1/3 and W 1 is negative 1/3 bingo we've got W 1 we've got W - we know that B was 0

**[29:37]:** because obviously at 0 its y equals x and we're done ah that was fast the alphas might have taken longer actually the alphas on this one or more of a pain in the ass than anywhere else because let's take a look at this one if you can see it we've added yet some new points we've got this point up here this point down there so I think pretty clearly these plus and minus are the closest to each other but what happens if we take the perpendicular bisector between these two and dude like this this plus is in the middle so we're going to have 2 feet so therefore this plus is going to have to also be support vector so we can't just draw this line we have to include this what's our best division vertical lines that's right vertical lines just saw and that means that the equation of our equation of our boundary the dotted line here y-axis

**[30:45]:** so the equation of the boundary of the y-axis then B equals zero and how W 2 equals zero so the only thing that is not W 2 equals 0

**[31:03]:** so W 2 equals 0 B equals 0 but W 1 is not equal to 0 because it's just the equation of the y-axis so we therefore know that the equation is just W 1 times x equals 0

**[31:31]:** so if w1 times x equals zero and we know that that just means essentially that X is going to be some K it's also going to be negative because of the fact that that plus is our store on the left then we're going to figure out what that K is we'll use our old trick by this point old hopefully one over magnitude of W equals D this time D is just 1 so therefore magnitude of W equals one there's only one component in W so therefore W 1 is minus 1 because the plus is on the left people see that not too bad this one's easy to calculate a W but it's not as easy to get all the alphas but let's move on to a new and even more fun maybe not um question which is this guy as you can see this is a lease this well maybe not this is a one-dimensional

**[32:34]:** vector these vectors only have a single dimension so it just looks like a number line here that dimension varies from it looks like negative 9 to positive 9 either just has one component you don't have to worry about any of these crazy magnitudes with two components just everything as a single component however it's obvious that a linear basis line is going to completely screw us up here since lines at this point are just like glump all these are pluses all these are Mises well not great that doesn't get them off so how are we going to do it well we're going to use what is usually perhaps the hardest thing in SVM's but in this case is not going to be too bad for us we're going to use a kernel now based on how little I understood kernels the first time I've took this class I'm guessing that you guys would like to have

**[33:28]:** some explanation on these kernels you probably saw them you remember the kernels from Patrick's lecture vaguely they're like this Phi and then there's this K and then they get really complicated okay so here's how the kernel works the basic idea is this and I'll write it over here oh wow there's more stuff okay I'll write it right here the basic idea is this we're taking the normal space which is you know this number line or it could be any kind of normal space and we're going to take a vector we're going to put it into it of a function called Phi and Phi of vector X brings X into some new dimension Phi or fee if you like it better is usually a nasty piece of work and something that you'd never ever want to look at sometimes it's not too bad Phi is the function that brings it into the new dimension okay and

**[34:28]:** when you brought the data into a new dimension sometimes you can just cut a straight line in that dimension and you'll just be happy however something that was noted by the very very smart inventor of support vector machines is that you don't actually need to to work with function Phi even if Phi is an absolutely horrible monstrosity because of the fact that you never need to know what the vector X what all these vectors X actually are in the new space at least not directly in none of these equations up here do we ever use X by itself however we do use X being dot product with something else so he figured out a very sneaky and excellent shortcut okay so I shouldn't use x1 x2 L use X and Z so if you have two vectors X and Z which are in a regular space you put them into this function called the kernel then it will tell

**[35:44]:** you Phi of X

**[35:50]:** dotted with 5z and if you have that you don't need five does everyone see that does everyone see why we don't need five look at all these equations up here we never look at X by itself in these in these vector equations at least now calculating alphas yeah that gets a little bit fuzzy but in the vector but in the vector equations also you get you may ask like why would you do this you can't complete the alphas it turns out we're um that actually other than for these very simple linear problems human minds cannot calculate the alphas in fact you run a very complicated quadratic optimization in fact finding out the best alphas is the thing that you will climb on when you're doing SVM's in the real world you say alright I'll run my algorithm and I know there's only one peak which is very very good because it's quadratic optimization let me

**[36:51]:** figure out the office so in fact it doesn't matter that you can't do that use these alpha places to figure out the alphas um if you've gone into if you only know the kernel function out the Phi function because normally the computer figures out the alphas for you with quadratic optimization just in these simple problems we know you can calculate the alphas so we have the kernel which replaces that basically gives us the dot product of the things in the new space so being that as it may I'll give you the kernel here I'd like you to give me the Phi so a kernel for that someone got an idea whose name was Susan Q random student apparently she got an idea that if we had a kernel for X and Z actually they're not vectors I guess they're just single components and the kernel equals cosine PI over 4 x times cosine pi over

**[37:53]:** 4z plus sine PI over 4 X plus pi over 4z so that is the new dot product cosine PI over 4x oh wait sorry I put one of the disease not inside the parenthesis that was still with me so cosine of the quantity PI over 4 x times cosine is upon T pi over 4z plus sine is upon T PI over 4 X plus sine of the quantity PI over 4 z is the new dot product so that begs the question in this one this is an easy one so we can calculate the 5 what is Phi of X we're actually taking it from one dimension and we may be playing around with it a lot to get this this thing has become a new dot product it replaces dot product now remember the dot part for scalars would have just been multiplying the two numbers together so it actually makes it a little bit

**[39:03]:** more complicated does anyone think they know the Phi ah we got one what do you think

**[39:13]:** you need to come back two dimensional absolutely that's exactly correct how would you solve this on the next whole quiz if you're not our brave volunteer well that K if you squint at it not very much actually it's pretty much a dot product between cosine PI over 4 s and sine PI over 4 is I mean look at it dot product is just remember if the dot product of X and Z vectors is X 1 is X 1 Z 1 plus X 2 Z 2 so that basically is X 1 Z 1 plus X 2 Z 2 o this should have been many times yeah there's trillion times sorry there's a plus there anyone who missed it because of that my bad that should have been a x there should've been times up there the plot only plus is its cosine PI over 4 X cosine 2 pi over 4z plus sine PI over 4

**[40:12]:** x times sine PI over 4 Z so yeah it's basically the dot product between cosine PI over 4 X and sine PI for bingo all right last thing and then we well we're not done yet because we're going to maybe ask some questions and then we're going to see if we can conquer those alphas but last thing let's graph in this new dimension all the points so obviously cosines and sines so we're going to get results between 1 and minus 1 let's see maybe I can graph it um

**[40:50]:** can I write on all these wait maybe this one oh no people do weird stick figures there ok oh yeah this one's kind of messy but we'll do it on this ok so we've got

**[41:16]:** this is 1 negative 1 1 half and 1 or so 1/2 negative 1/2 1 negative 1 negative 1/2 1/2 ok so given that let's try to graph all these points on this number line into this new brave new dimension by using their cosine sine PI over 4 so all right great so let's do the pluses first what about the plus X 0 is cosine 0 sine 0 so what is that that's 1 0 that's right in fact the 8 and the minus 8 are also that

**[42:06]:** times 3 the eight of the minus 8 are also that because then it's just 2 pi n minus 2 pi which well cosine and sine are periodic ok great what about the one well that's cosine PI over 4 sine PI over 4 what's that

**[42:26]:** yeah it's root 2 over 2 root 2 over 2 so it's something like here we'll say and in fact the 9 and the 9 and the negative 7 are also that so there's three of these two okay what about the negative one that's cosine negative PI over 4 sine PI over 4

**[42:56]:** that's right the x value is positive root 2 over 2 and the y value is negative so and again there's three of them all right great now let's do the minuses there's the minus at 3 which is also the same as I minus at negative 7 minus at 3 is cosine 3 PI over 4 sine 3 PI over 4 which one is that

**[43:27]:** so that's in a second ah yeah that's going to be the second quadrant we're going to have the cosine is going to be negative but the sine is going to be positive and so we get three points here and as you may predicted the other one the 5 PI over 4 is in the third quadrant we get three points here why are the support vectors

**[44:02]:** but I understand where you're getting three of these of plus the person's

**[44:15]:** already lonely to keep ah there's two negatives here and there's two natives here good cop doesn't change the problem in fact if we just grapple points there might have been more but there's a very subtle and important distinction there are two negatives but otherwise yeah these are grasped correctly does anyone see where the support vectors are

**[44:43]:** some popular minus and plus will try to do the perpendicular bisector let's see it

**[44:54]:** that works but guess what these guys are on the same line so we'd better circle them so it's actually the question is what isn't a support vector only this only those big question this is one affection I mean you just proved that actually just show that those land on the same line so you really need like the cosine term and a side just the cosine alright classic is so then we've done this in one dimension all we do is do only the cosine so if we did only the cosine then they would have still been divided easily divisible the answer is absolutely we could have however the question did not because Susan Q random student decided to do cosine inside but yes if we had said you 654 student find a fine that will work for this you could have found a Phi that was just cosine that would be easier however it's important to

**[45:55]:** be able to work with what somebody else gives you in this case they gave you they give you that transformation which yeah was a wasteful with an extra dimension you didn't need the sine because you didn't need the y-axis really here you just needed the X does everyone see this how this works can maybe transform dimensions the main hardest part is they'll usually give you a K and ask for a fire or give you a fire ask for a K it's not too bad just remember if they give you a Phi do a dot product with it and if they give you a fire that's just one component dot product of one component just multiplying it together easy enough so they give you a K treat it as a dot product and try to reverse-engineer it's usually something like this that's easy to reverse-engineer I really haven't seen it where it's not so it may

**[46:45]:** look it often looks like the scariest problem but it's usually not too bad to go between five and KS does anyone have any questions on anything that we did on spectrum change but what is some what's the intuition behind the building like we solve it and figure out numbers integrations what is it but what is intuition what is w w is the dividing line it's the drop-dead dividing line when I say the drop-dead dividing line you like those big bold solid lines over there those are you're pretty certain lines everything passed that was a minus and you're doing your training is that everything has to plus the big bold line there was a plus in your training separate the dotted line is the one you're really going to use in the test data in the test data when push comes to shove you might get something that's inside

**[47:38]:** of that gutter and if it's on the say of that one up there it's on the up left side that dotted line you're going to call it a plus so that dotted line is your decision boundary and that is that is basically the idea and in fact the way that the algorithm would do it on the computer is it would contract the optimize the Alfa's which messes around with the dotted line and by quadratically i'll math-amazing the out those alphas you see how the alphas add up to a w it just checks it around and eventually it finds Oh making the alpha of this one zero makes that that makes it a better optimization you're trying to get the widest possible road it would eventually come out to this this is trivial for a human to eyeball but some real problems with two hundred data points that have to get one or two of them

**[48:22]:** wrong classified and so you may be using a quadratic kernel or something like that you can't do that you just can't well maybe you can in which case you should be getting a MacArthur Fellowship or something like that but the computer can't and the basic idea is when it comes down to it it figures out the Alpha is to give it the bet the best W for widest road in the W is your decision about it good question any other questions about our old friend SVM I have a question for you um after seeing this and let's pretend that they only ask you to solve for WS B's you know these kind of criminals and Pfizer typical things how many people now think that you can go through and work an SVM problem alright we've got we've got a few we a happy few band of brothers maybe eight people raised their hand there that's

**[49:17]:** good how many people know what a support vector is now that's really good because that if that's all you learn from today's minor sensation is still good it really is I'm telling you I had to take two classes on this and then ta it before I really really understood it so you guys are ahead of me all right take care have a great weekend and we'll see you for boosting and vampires next week
