# 6. Column Space and Nullspace

**Channel:** MIT OpenCourseWare
**Source:** https://www.youtube.com/watch?v=8o5Cmfpeo6g
**Transcript page:** https://www.withtranscript.ai/video/8o5Cmfpeo6g

## Chapters

- 0:09 — Chapter 1 (0:09)
- 11:34 — Chapter 2 (11:34)
- 16:46 — Chapter 3 (16:46)
- 27:41 — Chapter 4 (27:41)
- 30:43 — Chapter 5 (30:43)
- 36:35 — Chapter 6 (36:35)
- 43:34 — Chapter 7 (43:34)
- 45:55 — Chapter 8 (45:55)

## Transcript

**[0:09]:** okay this is uh lecture six in linear algebra and uh we're at the start of this new chapter chapter three in the text which is really getting to the center of linear algebra and uh I had time to make a first start on it at the end of lecture five but now is lecture six uh is officially the lecture on Vector spaces and subspaces and then especially there are two ve two subspaces that we're specially interested in one is the column space of a matrix the other is the null space of the Matrix so I got to tell you what those are okay so first to remember from lecture five what is a vector space it's a bunch of vectors that where I'm allowed where I can add I can add any two vectors in the space and the answer it stays in the space or I can multiply any Vector in the space By

**[1:26]:** Any constant and the result stays in the space so that's in fact if I combine those two into one you can see that if I can add and I can multiply by by numbers that really means that I can take linear combinations so the quick way to say it is that all linear combinations C any multiple of V plus any multiple of w stay in the space so can I give you examples that our Vector spaces and also some examples that are not to to make that point clear so suppose I'm in uh three dimensions then uh one way to get us one uh one space is the whole three-dimensional space so the whole Space R3 threedimensional space would be a vector space because if I have a couple of vectors I can add them and I'm certainly okay and they follow all the rules so R3 is easy now I'm interested also in subspaces

**[2:39]:** so there's this key word subspaces that's a space of that's some vectors inside the given space inside R3 that still make up a vector space of their of their own it's a vector space inside a vector space and the simplest example was a plane so like can I just sketch it there there is a plane it's got to go through the origin and of course it goes infinitely far that's a that's a Subspace now do you see that if I have two vectors on the plane and I add them the result stays in the plane if I take a vector in the plane and I multiply by min-2 I'm still in the plane so that plane is a Subspace so let me just make that point plane through zero through that 0 0 0 is a Subspace okay and also uh another Subspace would be a line a line through 0000 yeah the line has

**[3:54]:** to go through the origin all subspaces have got to contain the origin contain zero the zero Vector so this line is a Subs line is a Subspace really if I if I want to say it really correctly I should say a Subspace of R3 that's I that of R3 was like understood there now so let me call this plane P and let me call this line L and let me ask you about other uh set other sets of vectors suppose I took yeah here here so here's a first question suppose I take two subspaces like p&l and I just put them together take their Union take all the vectors so so now I you've got P and L in mind here so I have two subspaces I have two subspaces and for example p a plane and L A Line okay now I want to ask you about the union of those so P Union

**[5:19]:** L this is all vectors in P or l or MO is that a Subspace is this a Subspace this is or is not a Subspace because we I just want to be sure that I've got the central idea suppose I take the vectors in the plane and also the vectors on that line put them together so I've got a bunch of vectors is it a Subspace can you give me like so the camera can hear it or maybe the tape can you say yes or no do I have a Subspace if I put if I take all the vectors on the plane plus all and all the ones on the line and just join them together but I'm not taking this guy that's actually I'm not taking most of them because most vectors are not on the line or the plane they're off somewhere else do I have a Subspace no right thank you no because why not because I can't add because if I that that requirement isn't satisfied if I take one vector like this guy

**[6:55]:** and another Vector that happens to come from L and add I'm off somewhere else you see that I that I've gone outside the union if I just add something from p and something from L uh then normally what'll happen is I'm outside the Union uh the and I don't have a Subspace so the correct answer is is not okay now let me ask you about the other thing we do is take the intersection so what does intersection mean mean intersection means all vectors that are in both P and L is this a Subspace yeah so I guess I want to go back up to the same question this is or is not a Subspace and you can answer me answer the question first for this particular example this picture I drew what's what is p intersect L for this case it's only zero at least sort of this was the artist's idea as he drew

**[8:10]:** it that uh that that line L was not in the plane and uh went off somewhere else and then the only point that was in common was the zero Vector is the zero vector by itself a Subspace yes absolutely and what about uh uh if I if if I don't have this plane and this line but any Subspace and any other Subspace so now can can I ask that question for any two subspaces so maybe I'll write it up here if I'm strong enough okay so this is the general question I have subspaces say S&T and I want to ask you about their

**[9:06]:** intersection s intersect T and I want it is a

**[9:16]:** Subspace do you see why do you see why if I take the vectors that are in both one that are in both of the subspaces so that's like a smaller set of vectors probably because it's we've added requirements it has to be in s and in t how do I know that's a Subspace can we just think through that abstract stuff and then I get to the examples okay so why suppose I take a couple of vectors that are in the intersection why is the sum also in the intersection okay so let me give a name to these vectors say V and W they're in the intersection so that means they're both in s also means they're both in t so what can I say about v+ W is it in s yes right if if I take two vectors V and W that are both in s then the sum is in s because

**[10:22]:** s was a Subspace and if they're both in T and I add them then the result is also in in t because T was a Subspace so the result V + W is in the intersection it's in both and requirement one is satisfied requirement two is the same if I take a vector that's in both I multiply by seven 7even times that Vector is in s because the vector was in s seven times that Vector is in t because the the original one was in t so stimes that vector is in the intersection in other words when you take the intersection of two subspaces you get probably a smaller Subspace but it is a Subspace okay so that's like sort of just emphasizing what these two requirements mean again to to to to let me Circle those because those are so important the the sum and the scalar multiplication which which combines into linear combinations

**[11:34]:** that's what you have to do inside the Subspace okay on to the column space okay so my lecture last time started that and I want to continue it okay column space of a

**[11:51]:** matrix of a okay can I can I take an example say say 1 2 3 4 1 1 1 1 2 3 4

**[12:10]:** five okay that's my Matrix

**[12:18]:** a so it's got columns three columns those columns are vectors so the columns so the column space of this a of this a let me let's stay with this example for a while the column space of this Matrix is a Subspace of r r what so what space are we in if I'm looking at The Columns of this Matrix R4 right these These are vectors in R4 they're four-dimensional vectors so it say column space of a is uh is a Subspace of R4 here because a was 4 by a is a 4x3 matrix this tells me how many rows there are how many components in a column and so we're in R4 Okay now what's in that Subspace what's so the column space of a it's a Subspace of R4 I call it the column space of a like that so that's that's my little symbol for some Subspace of R4 what's in R what's

**[13:35]:** in that Subspace well that column certainly is 1 2 3 4 this column is in this column is in and what else so it's got The Columns of a in it but that's not enough certainly right I don't have a Subspace if I just put in three vectors so how do I fill that out to be a Subspace I take their linear combinations so the column space of a is all linear combinations combinations of the colors and that does give me a Subspace it does give me a vector space because if I have one linear combination and I multiply by 11 I've got another lar combination if I have a linear combination I add to another linear combination I get a third combination so that this is like the smallest space like it it's got to have those three columns in it and it has to have their combinations and that's where we stop okay

**[14:48]:** now I I'm going to be interested in that space so like get some idea what's in that space how big is that space is that space the whole four dimensional space or is it a Subspace inside can you let me just see if you can I mean we can get a yes or no answer sometimes without without being ready for the complete proof what do you think is the Subspace that I'm talking about here the combinations of those three guys does that fill the full four-dimensional space maybe yes or no on that one no no somehow our feeling is and it happens to be right that if we start with three vectors and take their combinations we can't get the whole four-dimensional Space uh now so somehow we get a smaller space but how much smaller uh that's okay that's going to come up here that's not so immediate uh let me let me uh

**[16:02]:** first make this critical connection with with um linear equations because behind our abstract definition we have a purpose and that is to understand ax equal B so suppose I make the connection does axal B always have have a solution for every B have a have a solution for every right hand

**[16:46]:** side I guess that's going to be a yes or no question and then I'm going to ask which right hand sides are okay that that's really the question I'm after is which right hand sides B do make a you can see from the way I'm speaking what's the answer to this question as it is the answer is no ax equal B does not have a solution for every B why why do you say why do I say no because ax equal B is it's really this is like this is four

**[17:33]:** equations and only three unknowns right X is let me let let me let me write out that whole what the whole thing looks like yeah let me write out ax equal B ax is these columns are 1 2 3 4 1 one one 1 and 2 3 4 5 then X of course has three components X1 X2 X3 and I'm trying to get the hit the right right hand side B1 B2 B3 and

**[18:24]:** B4 so my first point is I can't always do it in a way that just says again what you told me five minutes ago that the combinations of these columns don't fill the whole four-dimensional Space there's going to be some vectors b a lot of vectors B that are not combinations of those of these three columns because the combinations of those columns are like going to be just a little plane or something inside inside R4 now so and you see that I do have four equations and only three unknowns so like anybody is going to say no you dope you can't usually solve four equations with only three unknowns but now I want to say sometimes you can for some right hand sides I can solve this so that's the that's the bunch of right hand sides that I'm interested in right now which right hand sides allow me to solve this this is the

**[19:35]:** question for today it's going to have like a nice clear answer so my my question is that so my question is which B's which vectors B allow me allow this system to be solved and I I want to ask you so that's like gets two question marks to indicate that's this is the important question okay first before we give a give a total answer give me just a partial answer tell me one right hand side that I know I can solve this thing for so all zeros okay that's the like guaranteed if B's were all zero then I know I can solve it let the X's all be zero no problem okay so that's that's always okay ax equals z i can always solve now tell me another right hand side just a specific set of numbers for which I can solve this this these three these four equations with only three unknowns but if

**[20:56]:** you give me the a good right hand side I can do it so tell me one 1 2 3 4 if I can I solve is this a is that a good right hand side can you solve can you find a solution that X1 + X2 + 2 X3 is 1 2 X1 + X2 + 3x3 is 2 and two more

**[21:26]:** equations so I'm asking you to solve in your head in within 5 Seconds uh four equations and three unknowns but you can do it because the right hand side is like showing up here it's it's it's one of the columns so tell me what's the x that does solve it one 0 one 0 0 solves it because well you can multiply this out by rows but oh God it's much nicer to say okay this is one of this column zero of this zero of this so it's one of that column which is exactly what we wanted okay so there is a b that's okay now tell me another B that's okay another right hand side that would be all right well all ones actually and then what's the solution in that case 0 one0 thanks and in fact it's much like one way to do it is think of a solution first right and then just see what B turns out to

**[22:49]:** be what B turns out to be right okay uh so I think of a solution so I think of an X I think of any X1 X2 X3 I do this multiplication and what have I got where now now I'm ready to answer the big question I can solve axal B exactly when the right hand side B is a vector in the column space good I can solve axal B B when B is a combination of the columns when it's in the column space so let me write that answer down I can I can solve axal B exactly

**[23:49]:** when B is in the column space let me just say again why that is because it the column Space by its definition contains all the combinations it contains all the AXS the column space really consists of all vectors A * n x so that's those are the B's that I can deal with if B is a combination of the columns then that combination tells me what x should be if B is not a combination of the columns then there is no X there's no way to solve ax equal B okay so the column Space is really the that's that's really why we're interested in this column space because it's the central guy it says when we can solve and and so so we got to understand this column space better let's see do I do I want to think yeah somehow oh well let's just as long as we've got it here what do I get for this particular example if I take combinations of this and this and

**[25:23]:** this I I'll tell you the question that's in my mind it's it's not even proper to use this word yet but you'll know what it means are those three columns independent if I take the combinations of the three columns does it does each column contribute something new or not so that if I take the combinations of those three columns do I like get some three-dimensional Subspace do I have three vectors that are like you know independent whatever that means or do I do a or do or does is one of those columns like contributing nothing new so that actually only two of The Columns would have given the same column space yeah that's a good way to ask the question finally I think of it can I throw away any columns and have the same column space yes and which one do you suggest I throw away well three is the natural like guy to

**[26:36]:** Target so if I and why because what what did you what's so bad about three here column three it's it's the sum of these right so it's not if I'm taking if I have combinations of these two and I put in this one actually I don't get anything more so later on I will call these pivot columns and the third guy will not be a pivot column in this with that with those numbers now actually but honesty makes me ask you this question could I have thrown away column one yes I could I could so when I'd say pivot columns my convention is okay I'll keep the first ones as long as they're not dependent so like keep this guy he's fine he's Al line I keep the second guy it's in a second Direction but the third one which is in the same plane as the first two gives me nothing new it's dependent

**[27:41]:** in the in the language that we will use and I don't need it okay so I would describe the column space of this Matrix as a two-dimensional Subspace of R4 a two-dimensional Subspace of R4 okay so you're seeing how these Vector spaces work and you you're seeing that we some idea of dependence or Independence is is in our future okay now I want to speak about another Vector space the null space so again I'm getting a little ahead because it's in section 3.2 but that's okay all right now so I'm I'm ready for the null space let me keep the same Matrix and this is going to be a different totally different Subspace totally different okay now so let me make space for now here here comes a completely different Subspace the null

**[28:55]:** space of a what's in it it contains not right hand sides B it contains X's it contains all X's that solve this word null is going to I mean that's the key word here meaning zero so this contains this is all solutions X and of course X's our vectors X1 X2 and X3 to the equation a x = z well four equations because we got so you see what I'm doing I'm now saying okay columns were great the column space we understood now I'm interested in X's I'm not the only B I'm interested in now is the B of all zeros the right hand side is now zero and I'm interested in Solutions X's so where is this null space for for this example these X's are have three components so the null space is a Subspace we still have to show it is a Subspace of R three so this is

**[30:33]:** and and we will show so these these vectors X this is in

**[30:43]:** R3 where the column space was in R4 in our example in in for an M byn Matrix this is M and this is m because the number of columns n tells me how many unknowns how many X's multiply those columns so it tells me the big space in this case R3 that I'm in now tell me why don't we figure out what the null space is for this example uh just by looking at it I mean that's the beauty of small examples that my official way to find null space and column spaces and and and get all the facts straight would be elimination and we'll do that but with a small example we can see see what's going on without going through the mechanics of elimination so so this null space so I'm talking about the again the null space and let me copy Again The Matrix 1 2 3 4 1 1 one one

**[32:00]:** and 2 3 4 5 what's in the null space so I'm I'm taking a * X so let me I'm me write it again and I want you to solve those four equations in fact I want you to find all solutions to those four equations well actually first of all find one I why should I ask you for all of them tell me one well tell me one solution that you you don't even have to have to look at the Matrix to know one solution to this set of equations it is zero Vector whatever that Matrix is it's null space contains zero because a times the zero Vector sure gives the zero right hand side so the null space certainly contains zero so it's got a chance to be a vector space now and it and it will turn out it is okay tell me the tell me

**[33:02]:** another solution so this particular null space and of course I'm going to call it n of a for null space this contains well we've already located the zero vector and now you're going to tell me another Vector that's in the null space another solution another X another you see what I'm asking you for is a combination of those columns that's what I'm I'm always looking at combinations of columns but now I'm looking at the X the the the the the weights the coefficients in the combination so tell me a good set of numbers to put in there one one one one minus one thanks 1 one minus one so there's a vector that's in it okay but have I got a Subspace at this point certainly not right I got just a couple of vectors no way they make a Subspace tell me actually why don't I jump the whole way now tell me well

**[34:15]:** tell me one more Solution One More X that would work 22 -2 oh well tell me all of them that would have been easier tell me the whole lot now what what is the null space for this matrix it's all vectors of the form so what what could what could this be it could be 1 one minus one it could be any it could be any number c any number the same number again and minus C in other words actually and any multiple of this guy oh that's that's the perfect description because now the zero vectors automatically included because C could be zero the vector I had is included cuz C could be one but now any vector and that's actually

**[35:19]:** it and do I have a do I have a Subspace and what does it look like it's in so what what how would you describe this the null space this all these vectors of this form c c minus C like 7 7 - 7 - 11 - 11 + 11 what have I got here if it describe that whole null space what if I drew it what do I draw a line right the null space is a line it's the line through it's it's in R3 and the vector 1 one1 maybe goes down here I don't know where it goes they down here there's the vector 1 one1 that you gave me and where is the vector c c c it's on this line of course there's 0 0 that we had and what we've got is that whole whoops that whole line going both ways through the origin the null space is a line in

**[36:35]:** R3 okay for that example we we could find all the combinations of the columns that gave zero at site now can I just take one more time to go back to the definition of Subspace Vector space and ask you to why how do I know that the null space is a vector space how am I entitled to use this word space I'll never use that word space without meaning that the two requirements are satisfied can we just check that they are so I'm going to check that look can I just continue here check that that the solution

**[37:30]:** to ax = 0

**[37:37]:** always give a Subspace and the course the keyword is that space so what do I have to check what I have to show that if I have one solution call it X and another solution call it xar that their sum is also a solution right that's the that's a requirement to use that word space I have to say I have to convince myself that if ax is zero and also an ax star is zero or maybe I should have said if AV is zero and a w is zero then and what about V + W so I let me use those letters if a v is zero and a w is zero then what then what is it what's the if that and that then what's my point here that a * V + W must be zero that says that if V is in the n space and W is in the null space

**[39:00]:** then there some V plus W is in the null space and of course now that I've written it down it's totally absurd ridiculously simple because matrix multiplication allows me to separate that out into AV plus a w i shouldn't say absurdly simple that was a dumb thing to say we've used here a base basic law of matrix multiplication actually we've used it without proving it but that's okay we only live so long and just skip that proof uh I think it's called the distributive law that I can split these split this into two pieces but but now you see the point that that AV is zero and a w is z so I have 0 plus 0 and I do get zero it checks and similarly I have to to show that if AV is zero then a * any multiple say 12 V is also zero and how do I know that because I'm allowed to bring that 12 outside a number a scaler can move outside so I have 12 AVS 12 zeros I have zero that's

**[40:22]:** check to okay just to it it's really critical to understand the oh yeah here under what am I what I was going to say understand what's the point of a vector space let me make that point by changing the right hand side oops okay let me change the right hand side to 1 two 3 4 oh okay why don't we do all of linear algebra in one lecture then we we okay I would like to know solutions to this equation to those four equations so I have four equations I have only three unknowns so if I don't have a pretty special right hand side there won't be any solution at all but that is a very special right hand side and we know that there is a solution one were there any more solutions and did they form a vector space okay so I'm asking two questions there one is do so my right hand side now is not zero anymore I'm

**[41:35]:** not looking at the null space because I I changed from zeros so my first question is do the solutions if there are any and there are do they form a Subspace let's answer that question first yes or no do I get a Subspace if I look at at the solutions to let me let me go back to X1 X2 X3 I'm looking at all the X's at all those vectors in R3 that solve axal B the only thing I've changed is B isn't zero

**[42:16]:** anymore do the X's the solutions form a vector space no they don't they absolutely don't why what's an easy way to see that I don't have a the solutions to this do not form a Subspace the solutions don't because what how shall I see that the zero Vector is not a solution so I never even got started the zero Vector doesn't solve this system I can't the solutions can't be a vector SP now what are they like well we'll see this but let's do it for this example so one 0 was a solution you saw that right away are there any other Solutions can you tell me a a second solution to this system of equations 0 ne1 1 boy that's 01 1 yes because that says I take minus this column plus this one and sure enough that's right so there are there's a bunch of solutions

**[43:34]:** here but they're not a Subspace I'll tell you what it's like it's like a plane that doesn't go through the origin or a line that doesn't go through the origin maybe in this case it's a line that doesn't go through the origin if I graph the solutions to ax equal B so you I I think you've got the idea subspaces have to go through the origin if I'm looking at X's then they better solve ax equals z in a way I've got my two subspaces that I talking about today are kind of the two ways I can tell you what a sub about a about a Subspace if I want to tell you about the column space I tell you a few columns and I say take their combinations like I build up this Subspace I put in a few vectors their combinations make a Subspace now when I went to let me come back

**[44:34]:** to the one that is a Subspace

**[44:40]:** here here when I talked about the null space I didn't tell you what's in it we had to figure out what was in it what I told you was the equations that I'm that has to be satisfied you see those like those are the two natural ways to tell you what's what's in this what's in a Subspace I can either give you a few vectors and say fill it out take combinations or I can give you a system of equations the requirements that the X's have to satisfy and both of those ways produce subspaces and they're the important ways to construct Subspace okay so today's lecture actually got uh the essentials of 3.2 the idea of the null space now we have to tackle Wednesday the job of how do we actually get hold of that Subspace in an example that's bigger and we can't see it just by ey okay see you Wednesday

**[45:55]:** thanks
