# 7. Solving Ax = 0: Pivot Variables, Special Solutions

**Channel:** MIT OpenCourseWare
**Source:** https://www.youtube.com/watch?v=VqP2tREMvt0
**Transcript page:** https://www.withtranscript.ai/video/VqP2tREMvt0

## Chapters

- 0:10 — Chapter 1 (0:10)
- 12:15 — Chapter 2 (12:15)
- 17:44 — Chapter 3 (17:44)
- 22:13 — Chapter 4 (22:13)
- 29:02 — Chapter 5 (29:02)
- 34:00 — Chapter 6 (34:00)
- 36:44 — Chapter 7 (36:44)

## Transcript

**[0:10]:** okay here's linear algebra lecture s I've been talking about Vector spaces and especially the null space of a matrix and the column space of a matrix what's in those spaces now I want to actually describe them H how do you describe all the vectors that are in those spaces how do you compute these things so this is the uh turning the idea the definition into an algorithm what's the algorithm for solving ax equals z so that's the null space that I'm interested in so can I take a particular Matrix a and describe the natural algorithm and I'll execute it for that Matrix uh here we go uh so let me take the Matrix as an example so we're definitely talking rectangular matrices in this chapter so I'll make I'll have four columns and three rows 2 468 and 36810

**[1:35]:** okay if I just look at those columns uh and rows well I noticed right away that column two is a multiple of column one it's in the same direction as column one it's not independent I I'll expect to discover that in the process actually with rows I noticed that that row plus this row gives the third row so the third row is not independent so uh all that should come out of elimination so now what am I my my algorithm is elimination but extended now to the rectangular case where we have to continue even if there's zeros in the pivot position we go on okay so let me execute elimination for that Matrix my goal is oh was while I'm doing elimination I'm not changing the null space that's very important right I'm solving ax equals z by elimination and when I do these operations that you already know when I subtract a multiple of

**[2:55]:** one equation from another equation I'm not changing in the solutions so I'm not changing the null space actually I am changing the column space as you'll see so you have to pay attention what does elimination leave unchanged and it the answer is the solutions to the system are not changed because I'm doing the same thing to I'm doing a legitimate operations on the equations of course on the right hand side it's always zero and I don't plan to write zero all the time okay so I'm really just working on the left side but the right side is is keeping up always zeros okay so what's elimination well you know where the first pivot is and you know what to do with it so can I just take the first step below here so that pivot row is fine I take two times that row away from this one and I get zero Z that's signaling

**[4:02]:** a difficulty two two tws away from the six leaves me with a two two tws away from the eight leaves me with a four and now three of those away from here is zero again another zero 3 tws away from that eight is the two 3 twos away from that 10 is a four okay that's the first stage of elimination I've got the First Column straight so of course I move on to the second column I look in that position I see a zero I look below it hoping for a nonzero that I can do a row exchange but it's zero below so that's telling me that that column is well what it's really going to be telling me is that that column is a combination of the earlier column it's depend that second column is dependent on the earlier column but I don't stop to think here that column there's nothing to do I

**[5:11]:** go on to the next so here's the next pivot so there's the first pivot and there's the second pivot can I just keep this elimination going downwards so so the next step keeps the first row keeps the second row with its pivot so I've got my two pivots and does elimination to clear out the column below that pivot so actually you see the multiplier is one it subtracts row two from Row three and produces a row of zeros okay that I would I would call that Matrix U right that's our upper well I can't quite say upper triangular uh maybe upper I don't know upper something it's got uh it's it's in this so-called Echelon form e the word Echelon means like staircase form it's the the nonzeros come in that staircase form if there was another pivot here the then the staircase would include that but here's a case where we

**[6:34]:** have two pivots only okay so actually we've already discovered the most important number about this Matrix the number of pivots is two that number we will call the rank of the Matrix so let me put it put immediately the rank of a so I'm telling you what the this word rank means in the algorithm it's equal to the number of pivots and in this case two okay for me that number two is the crucial number okay now I go to you remember I'm always solving ax equals z but now I can solve uux equals 0 right same Solutions same null space okay so I could stop here why don't I why don't I stop here and do the back substitution so now I have to ask you how do I describe the solutions there are solutions right to ax equals z I knew there would be I had three equations in four unknowns I certainly

**[7:50]:** expected some solutions now I want to see what are they okay here's the critical step I and I refer to it up here as separating out the pivot variables the pivot columns which are these two here I have two pivot columns those are those obviously they're The Columns with the pivot so I have two pivot columns and I have the other columns I'll call free two these are free columns

**[8:32]:** okay why do I use those words why do I use that word free because now I want to write I want to find the solutions to ux equals z here's the way I do it these free columns I can assign any number freely to those to those to those to the to the variables X2 and X4 the ones that multiply columns 2 and four so I can assign anything I like to X2 and X4 and then I can solve the equations for X1 and X3 let me say that again if I give let me let me assign so so one particular X is to assign say the value one to the to X2 and zero to X4 those are that was a free choice but it's a convenient Choice okay now I want to solve ux equals 0 and find numbers 1 and three F complete the solution okay can I write down let's see

**[9:49]:** shall we just remember what ux equals 0 represents what are my equations that first equation is X1 plus just I'm just saying what are these matrices of meaning that's the first equation and the second equation was 2x3 + 4x4 = 0 those are my two equations okay now the point is I can find X1 and X3 by back substitution so we're building on what we already know the new thing is that there were some some free variables that I could give any numbers to and I'm systematically going to make a choice like this one and zero now what is X3 let's let's go backwards back up I look at the last equation X3 is zero from the last equation because because X4 we've we've set X4 to zero and then we get X3 as zero okay now we set X2 to be one so what is X1 -2 right so that I have -2 +

**[11:08]:** 2 0 and 0 correctly giving zero there is a vector in the null space there's a solution to ax equals 0 in fact what solution is that that simply says that -2 * the First Column plus 1 * the second column is the zero column of course that's right minus two of that column plus one of that or minus two of that plus one of that that solution is that that's just what we saw immediately that the second column is twice as big as the First Column okay tell me some more vectors in the null space I found one tell me uh how to get a bunch more immediately out of that one just take multiples of it it any multiple of I could multiply this by anything I might as well call it I could say C some multiple of this so let me so X could be any multiple of this one okay

**[12:15]:** that that describes now a line an infinitely long line in four-dimensional space but which is in the null space is that the whole null space no I've got two free variables here I made this Choice One and zero for the free variables but I could have made another Choice let me make the other choice zero and one you see my system it my so let me repeat the system this is the algorithm that you you just learn to do find do do elimination decide which are the pivot columns and which are the free columns that tells you that that variables 1 and three are pivot variables two and four are free variables then those free variables you assign them you you give one of them the value one and the others the value zero in this case we had a one and a zero and complete the solution and you do you give the other

**[13:25]:** one the value one and and zero and now complete to solution so let's complete that solution I'm looking for a vector in the null space and it's absolutely going to be different from this guy because uh because any multiple of that zero is never going to give the one so I have somebody new in the null space and let me finish it out what's X3 here so we're going by back substitution looking at this equation now X4 we've changed we're doing the other possibility where X2 is zero and X4 is 1 so X3 will happen to be min-2 and now what do I get for that first equation uh X1 let's see 2 x3s is min -4 plus two do I get a two there perhaps yeah two for X1 - 4 and two okay that's in the null space what does that say about the columns that says that two of this column minus

**[14:36]:** two of this column plus this column gives zero which it does two of that minus two of that and one of that gives the zero column okay now now I found another Vector in the null space now we're ready to tell me the whole null space what are all the solutions to ax equals 0 I've got this guy and when I have him what else is goes into the null space along with that these are my two special Solutions I call them special I just invented that name special Solutions what's special about them is the special numbers I gave to the free variables the values zero and one for the free variables I could have given the free variables any values and got vectors in the null space but this was a good way to be sure I got got everybody okay so once I have him I also have any multiple right so I

**[15:45]:** could take any multiple of that and it's in the null space and now what else I left a little space for what what a plus sign I can take any combination here's a line of vectors in the null space a bunch of solutions would you rather I say in the null space or would you rather I say okay I'm solving ax equals z well really I'm solving ux equals z well okay let me put in that crucial plus sign I'm taking all the combinations of my two special Solutions that that's my conclusion now the null space contains contains exactly all the combinations of the special Solutions and how many special Solutions are there there's one for every free variable and how many free variables are there oh I mean we can see all the whole picture now if if the rank R was two this is the

**[16:59]:** this is the number of pivot variables right because it counted the pivots so how many free variables well you know it's two right what is it in for a matrix that's M rows n columns n variables that means with rank R how many free variables have we got left if R of the variables are pivot variables we have n minus r in this case 4-2 fre

**[17:44]:** variables you see that first of all we get a ni we get clean answers here we get R pivot variables so there really were our equation equations here there looked like three equations but there were really only two independent equations and there were n minus r variables that we could choose freely and we gave them those special 01 uh values and we got the special Solutions okay that for me we could stop at that point that gives you a complete algorithm for finding all the solutions to ax equals z

**[18:37]:** okay again you do elimination going onward when a column when when there's nothing to be done on one column you just continue that this number R the number of pivots is crucial and it leaves n minus r free variables which you give value Zer and one to I would like to take one more step I would like to clean up this Matrix even more so now I'm going to go to this is in its this is in Echelon form upper triangular if you like I want to go one more step to make it as good as it can be okay so now I'm going to speak about the reduced row Echelon form okay so now I'm going to speak about the Matrix R which is the reduced row Echelon form so what does that mean that means I just I I can I can work harder on you so let me start let me suppose I got as far as U which was good

**[20:08]:** notice how that row of zeros appeared I I didn't comment on that but I I should have that row of zeros appeared because Row three was a combination of rows one and two and elimination discovered that fact when we get a row of zero that's telling us that the that that the original row that was there was a combination of other rows and elimination knocked it out okay so we got this far now how can I clean that up further I can do uh elimination upwards I can get zero above the pivots so this reduced row Echelon form has zeros above and

**[21:04]:** below the pivots so let me do that so now I'll subtract one of this from the row above that will leave a zero and a minus two in there and that's

**[21:25]:** good okay and I can clean it up even one more step I can make the pivots the pivots I'm going to make equal to one because I can divide equation two by the pivot that won't change the solutions so let me do that and then I really I'm ready to stop 1 2 0- 2 0 0 1 two I divided the second equation by two because now I have a one in the pivot and Zer below okay this is my Matrix

**[22:13]:** R I guess I'm hoping that you could now execute the whole algorithm mat lab will do it immediately with the command RR R EF Ro reduced row Echelon form of a so if I input that original Matrix a and then I write then I type that command press return that Matrix will appear That's the reduced row Echelon form and it's got all the information as clear as can be what what information has it got well of course it immediately tells me the pivot rows pivot rows one and two pivot columns one and three and in fact it's got the identity Matrix in there right it's it's got zeros above and below the pivots and the pivots are one so it's so it's got a so notice the 2x two identity Matrix that's sitting in the pivot rows and pivot columns it's it's it's I in the pivot rows and

**[23:45]:** columns it's got zero rows below those are always indicating that original rows were were combinations of other rows so we really only had two rows there and now it also so there's the identity now it's also got its free columns and uh they're cleaned up as much as possible actually actually it's now so cleaned up that the special Solutions I can read off you you remember I went through the steps of computing this uh this uh doing back substitution let me let me instead of that system let me take this improved system so I'm going to use these numbers right in these equations what did I do I divided this equation by two and oh yeah and I had subtracted two of this which knocked out this guy and made that a minus sign is that what that's that's now I've now written RX equals

**[25:09]:** 0 now I guess I'm hoping everybody in this room understands the the solutions to the original ax equals z the Midway halfway ux equal Z and the final RX equals 0 all the same because going from one of those to another one I didn't mess up I I just multiplied equations and subtracted from other equations which I'm allowed to do okay but my point is that now if I do this free variables and back substitution it's just the numbers are there when I let X so in this guy I let X2 be one and X4 b0 I I guess what am I seeing here let me let me sort of separate this out here I'm seeing in the pivot in the pivot columns if I if I put the pivot columns here I'm seeing those and I'm in the free columns I'm seeing what am I seeing in the free columns the two Z and

**[26:24]:** that first free column the the X2 column and a minus 22 in the in the in the fourth column the other free column and and the row of zeros below which of course have no that equation is 0 equals z that's satisfying here's my point that when I do back substitution these numbers are exactly what shows up oh their signs get switched I was going to say those numbers 2 - 2 02 can I just circle the this is the free part of the Matrix this is the identity part this is the free part maybe I'll call it F this of course I call I because it's the identity the free part is a I mean I'm just doing back substitution and the those free numbers will show up with a minus sign because they pop onto the other side of the equation so I see min-2 0 and I see 2 -2 C can

**[27:38]:** I so that wasn't magic it had to happen let me uh show you clearly why it happened okay so that's this this is what I'm interested in here and now let me let me just like do it do it for or let's suppose we've we've got to let's suppose we've got this system already in in our RF form so my Matrix R is what does it look like can I'll let me pretend that the pivot columns come first and then whatever is in the free cup and there might be some zero rows below there's a typical a pretty typical reduced row Echelon

**[28:48]:** form you see what's typical it's it's got this is R by R this is this this is R pivot rows

**[29:02]:** this is our pivot

**[29:08]:** columns and here are n minus r free columns okay tell me what are the special Solutions what are the what's what's X if I want to solve RX equals z in fact let me I'm really going to uh do the whole since these this is now block matrices I might as well do all of the special Solutions at once so I want to solve RX equal 0 and I'll have some special Solutions let me actually can I do them all at once I'm going to create a null space Matrix

**[30:00]:** okay

**[30:04]:** matrix it's it's it's columns are the special solute The Columns are the special

**[30:18]:** Solutions this is I'm making it sound harder it's going to be totally easy n will be this null space Matrix I want r n to be the zero Matrix these Columns of n are supposed to multi get multiplied by R and give zero columns so what n will do the job let me put I'm going to put the identity in the free variable part and then minus F will show up in the pivot variables just the way it did in that example there we had the identity and F here and in the special solution so these columns are there's the Matrix of special Solutions and actually there so there's a matlb command or a teaching code command null n equal so this is the the produces the null basis the null space Matrix null of a and there it is do you so and how does that command actually work it uses matlb to compute R then it picks out the pivot variables the free variables puts puts ones and zeros in for the free variables and copies out the pivot variables

**[31:51]:** it it does back substitution but back substitution for this system is totally simple what what is this system uh RX = Z so this is R is i f and X is the pivot variables and the free variables and it's supposed to give zero so what does that mean that means that the pivot variables plus f * the free variables give zero so let me put ftimes the free variables on the other side I get minus ftimes the free variables there's there's my uh equation as simple as it can be that's that's what back substitution comes to when I've reduced and reduced and reduced this system to the to the best form okay and uh then if the free variables I make this special choice of the identity then the pivot V pivot variables are minus F okay can I do uh another example could you do another example can I let me just take

**[33:11]:** another Matrix and and let's go through this algorithm once more okay here we go here's a Blackboard for another Matrix okay so I'll call The Matrix a again but let me make it uh yeah how big should we make it this time ah why don't I do this just for the heck of it let me take the transpose of this a and see what happens to that 2 468 and 3 6 8 10

**[34:00]:** before we do the calculations tell me what's coming how many pivot variables do you expect here how many columns are going to have pivots how many we have three columns in that Matrix but are we going to are we going to have three pivots no because this third column is the sum of the first two columns I'm totally expecting totally expecting that these will be pivot columns because they're independent but that this third guy the third column which is dependent on the first two is going to be a free column elimination better discover that and elimination will also straighten out the rows we'll also Identify some zero some some dependent rows and some independent rows what's the what's the row reduced Echelon form for this let's just do it okay so so that's the first pivot two * that away from that gives me a row of zeros two * that away from that gives

**[35:22]:** me a 022 and 2 * that away from that gives me a 0

**[35:31]:** 44 okay First Column is straight first variable is a pivot variable no problem on to the second column I look at the second pivot it's a zero I look below it there's a two okay I do a row exchange so this zero is now there I now have a perfectly good pivot and I use it okay and I subtract two of that row away from this row you all right if I do it like that I I've got to the form U now this was my a now there's my U I can see I can see now I I have to stop right I would go on to the third column I should have tried I quit but without trying I shouldn't have done that on to the third column look at the pivot position it's got a zero in it look below all zeros now I'm entitled to stop okay so the rank is

**[36:36]:** two

**[36:44]:** again uh what about the null space how many special Solutions are there this time how many special solutions for this Matrix I've got an which are the free variables and which are the pivot variables and so on pivot columns I've got two pivot columns and that's no accident the number of pivot columns for a matrix a that's a great fact that the number of pivot columns for a and a trans a transpose are the same and then I have a free column there's a free column one free column because the count is 3 - 2 3 - 2 gives me one free column okay so now let me solve uh what's in the null space okay so how do I H let's see these vectors have length three they only have three components I'm making too much space for the to write X X has just got three components

**[38:00]:** and what are they I'm looking for the null space okay so how do I start I give the free variable some convenient value and what's that I set it to one I set the free variable to one if I set the free variable to zero and solve for the pivot variables I'll get all zeros no progress but by setting the free variable to one you see my two equations now are shall I my equations are X1 + 2x2 + 3x3 is z that's my first equation and my second equation is now 2x2 + 2x3 = 0 and uh okay so if X3 is 1 then X2 is minus1 and if X3 is 1 and X2 is min -1 then maybe X1 is

**[39:11]:** minus1 and actually I go back to check now I don't like I I did I did a quick calculation mentally can I mentally do a quick check that Matrix that solution X says that minus this column minus this column plus this one is the zero column and it is minus that minus that plus that is zero so that's in the null space and now you can tell me what else is in the null space what's what's the whole null space now it's I multiply by C right the whole null space is a line so that's the description if I ask you on a homework or a quiz or the final what give me describe tell me the null space find the null space of this Matrix you could take those steps and that's the answer I'm looking for and I'm looking for that c too because that's telling me that you're remembering that it's a

**[40:17]:** whole space and not just one vector later I will ask you for a basis for the null space then I just want this Vector but if I ask for the whole null space then there's the whole line through that Vector okay now one more natural thing to do with this example right is keep going to the the reduced Matrix R so can I push onwards to r that should be quick but let's just practice let me keep going to R okay so what do I do here I subtract I I want I clear out above the pivot so I subtract that from that that's one1 is left right when I subtracted this Row from this it produces zero Above This pivot and now I want that pivot to be a one so for the r Matrix I'll divide this equation by two and of course these zero zeros are great they don't change there's R

**[41:24]:** that's r you see what R is you see the identity Matrix sitting up here you see the Free Part F the F part here and you see the zeros below this is if F0 0 and what's the X the X has the identity well it's only a single number one but it's the identity Matrix in the free in the free part and what does it have in the pivot variables what did back substitution give it gave minus these guys you see that what this is is any multiple of this is the identity there and this is minus F here that's our null space this is our null space Matrix n for this our our null space Matrix is the guy whose columns are the special Solutions so their free variables have these special values one and the pivot variables have minus F so do you see the how the minus F just automatically shows up

**[42:43]:** in the special

**[42:48]:** Solutions that's it really I I don't think there's anything more I can say about ax equals z there's more I can say about ax equal B but that'll be uh Friday okay so that's that's the null space thanks
