Transcript
Okay, this is linear algebra lecture nine and this is a key lecture. This is where we get these ideas of linear independence. When a bunch of vectors are independent or dependent, that's the opposite. Uh the space they span, a basis for a subspace or a basis for a vector space, that's a central idea. And then the dimension of that subspace. So this is the day that those words get assigned clear meanings and emphasize that we talk about a bunch of vectors being independent. Wouldn't talk about a matrix being independent, a bunch of vectors being independent, a bunch of vectors spanning a space, a bunch of vectors being a basis, and the dimension is some number.
Okay. So what are the definitions? Can I begin with a fact a highly important fact that uh I didn't call directly attention to earlier. Suppose I have a matrix and I look at ax equals z. Suppose the matrix has a lot of columns so that n is bigger than m. So, I'm looking at n equations. I'm sorry, m equations. A small number of equations, m and more unknowns. I have more unknowns than equations. Let me write that down. More unknowns than equations. More unknown x's than equations. Then then the conclusion is that there's something in the null space of a other than just the zero vector. The conclusion is there are some nonzero x's such that ax is zero.
There are some special solutions and why we know why. I mean it sort of like seems like a reasonable thing. more unknowns than equations then it seems reasonable that we can solve them. But we have a a clear algorithm which starts with a system and does elimination gets the thing into an echelon form with some pivots and pivot columns and possibly some free columns that don't have pivots. And the point is here there will be some free columns. The reason so the reason is there must there will be free variables at least one. That's the reason that we we now have this a complete uh algorithm complete systematic way to say okay we take the system ax equals zero we row reduce we identify the free variables and since there are n variables and at most m pivots there will be some free variables at least one at least n minus m in fact left over and those variables I can assign nonzero values to.
I don't have to set those to zero. I can take them to be one or whatever I like and then I can solve for the pivot variables. So then it gives me a solution to ax equals z and it's a solution that isn't all zeros. So uh that's a c that's a important point that we'll use now in this lecture. So now I want to say what does it mean for a bunch of vectors to be independent. Okay. So this is like the background that we know. Now I want to speak about independence. Okay, let's see. I can give you the the the uh abstract definition and I will but I would also like to give you the direct meaning when my so the question is when vectors x1 x2 up to suppose I have n vectors are independent If now I have to give you or linearly independent I'll I'll often just say and write independent for short if okay let I'll give you the full definition these are just vectors in some vector space I can take combinations of them the question is do any combinations give zero if some combination of those vectors gives the zero vector other than the combination of all zeros.
Then they're dependent. They're independent if no combination gives the zero vector and then I have I'll have to put in and accept the zero combination. So what do I mean by that? No combination gives the zero vector. Any combination C1 X1 + C2 X2 + plus C N XN is not zero except for the zero combination. This is when all the C's all the C's are zero. Then of course that combination I I'm I know I'll get zero. But the the question is does any other combination give zero? If not then the vectors are independent. If some other combination does give zero the vectors are dependent. Okay let's just take examples.
Suppose I'm in say in two-dimensional space. Okay I give you I I'd like to first take an example. Let me take an example where I have a vector and twice that vector. So that's two vectors V and 2V. Are those dependent or independent? Those are dependent for sure. Right? There's a combin one vector is twice the other. One vector is twice as long as the other. So if the word dependent means anything, these should be dependent. And they are. And in fact, I would take two of the first. So here's here's a vector V and the other guy is a vector 2V. That's my So there's a vector V1 and my ve my next vector V2 is 2 V1.
Of course, those are dependent because two of these first vectors minus the second vector is zero. That's a combination of these two vectors that gives the zero vector. Okay, that was clear. Suppose suppose I have a vector. Here's another example. It's easy example. Suppose I have a vector and the other guy is the zero vector. Suppose I have a vector v1 and v2 is the zero vector. Then are those vectors dependent or independent? They're dependent. Again, you could say, well, this guy is zero times that one. This one is some combination of those. But let me write it the other way. Let me say what what combination how many V1s and how many V2s shall I take to get the zero vector?
If if v_sub_1 is like the vector 21 and v2 is the zero vector 0 0 then I would like to show that some combination of those gives the zero vector. What shall I take? How many v1's shall I take? Zero of them. Yeah. No. Take no v1s. But how many v2s? Six. Okay. or five. Then in other words, the point is if the zero vectors in there, if the zero if if one of these vectors is the zero vector, independence is dead, right? If one of those vectors is the zero vector, then I could always take include that one and none of the others and I would get the zero answer and I would show dependence. Okay. Now uh let me let me finally draw an example where they will be independent.
Suppose that's v1 and that's v2. Those are surely independent right. Any combination of v_sub_1 and v2 uh will not be zero except uh the zero combination. So those would be independent. But now let me let me stick in a third vector v3. independent or dependent. Now those three vectors. So now n is three. Here I'm in two-dimensional space. Whatever I'm in the plane I have three vectors that I didn't draw so carefully. I didn't even tell you what exactly they were. But what's this answer on dependent or independent? Dependent. Why? How do I know the those are dependent? How do I know that some combination of vub1, v2 and v3 gives me the zero vector?
I know because of that that's the that's the uh key fact that tells me that three vectors in the plane have to be dependent. Why is that? What's the connection between the dependence of these three vectors and that fact? Okay, so here's the connection. I take the matrix A that has V1 in its first column, V2 in its second column, V3 in its third column. So it's got three columns. And V1, I don't know, that looks like about 21 to me. V2 looks like it might be one two. V3 looks like it might be maybe two maybe two and a half minus one. Okay, those are three. Those are my three vectors and I put them in the columns of A.
Now that matrix A is 2x3. It fits this pattern that where we know we've got extra variables. We know we have some free variables. We know that there's some combination and let me instead of X's, let me call them C1, C2, and C3 that gives the zero vector. Sorry that my little bit of art got in the way. Do you see the point? When I have a matrix, I'm interested in whether its columns are dependent or independent. The columns are dependent if there is something in the null space. The columns are dependent because this this thing in the null space says that C1 of that plus C2 of that plus C3 of this is zero.
So in other words, I can go out some V1 out some more V2 back on V3 and end up zero. Okay. So let here I've given the general abstract definition but let me repeat that definition. So this is like repeat when for when the v1 let me call them v's now v1 up to vn are the columns of a matrix a in other words this is telling me that if I'm in if I'm in n if I'm in uh md dimensional space like two-dimensional space in the example. I can answer the dependence independence question directly by putting those vectors in the columns of a matrix. Then they independent. They are independent. Independent if the null space of A of A is what?
If I have a bunch of columns in a matrix, I'm looking at their combinations, but that's just A times the vector of C's and the ve and these columns will be independent if the null space of A is only the zero vector is only is the zero vector. They are dependent if there's something else in there. If there's something else in the null space, if a * c gives the zero vector for some nonzero vector c in the null space, then they're dependent because that's telling me a combination of the columns gives the zero column. I I I think you're with me because we've seen like lecture after lecture we're looking at the combinations of the columns and asking do we get zero or don't we and now we're giving the official name dependent if we do independent if we don't so I could express this in other words now I could say the rank what's the rank in this independent case the rank R of of the matrix in the case of independent columns is so the columns are independent.
So how many pivot columns have I got? All in all the columns would be pivot columns because free columns are telling me that they're a combination of earlier columns. So this would be the case where the rank is n. This would be the case where the rank is smaller than n. So in this case the rank is n and the null space of a is only the zero vector and no free variables. No free variables. And this is the case. Yes. free variables if you'll allow me to stretch the English language that far. Uh that's the case where we have um a combination that gives the zero column. So I I'm often interested in the case when my vectors are popped into a matrix.
So the the definition over there of independence didn't talk about any matrix. The vectors didn't have to be vectors in n-dimensional space. And I I want to give it some examples of vectors that aren't what you think of immediately as vectors. But most of the time this is the we the vectors we think of are columns and we can put them in a matrix and then independence or dependence comes back to the null space. Okay. So that's the idea of independence. Can I just uh yeah let me let me go on to spanning a space. What does it mean for a bunch of vectors to span a space? Well, actually, we've seen it already.
You remember if if we had a columns in a matrix, we took all their combinations and that gave us the column space. Those vectors that we started with span that column space. So spanning a space means so let me move that important stuff right up. Okay. So vectors let me call them say V1 up to I'll use some different letter say V L span a space a subspace or just a vector space I could say span a space means means The space consists of all combinations of those vectors. That that's exactly what we did with the column space. So now I could say in shorthand the columns of a matrix span the column space.
So remember it's a bunch of vectors that have this property that they span a space. And actually if I give you a bunch of vectors and say okay let s be the space that they span. In other words let s contain all their combinations. that space s will be the smallest space with those vectors in it. Right? Because any space with those vectors in it must have all the combinations of those vectors in it. And if I stop there, then I've got the smallest space and that's the space that they span. Okay? So I'm just rather than uh needing to say take all linear combinations and put them in a space, I'm I'm compressing that into the word span.
Straightforward. Okay. So if I think of a of the column space of a matrix, I've got I've got their So I start with the columns. I take all their combinations that gives me the column space. They span the column space. Now, are they independent? Maybe yes, maybe no. It depends on the particular columns that went into that matrix. But obviously, I'm highly interested in a set of vectors that spans the space and is independent. That's that means like I've got the right number of vectors. If I didn't have all of them, I wouldn't have my whole space. If I had more than that, they probably wouldn't they wouldn't be independent.
So like basis and that's the word that's coming is just right. So here let me put what that word means. A basis for a vector for a vector space is is a is a sequence of vectors shall I call them v1 v2 up to let me say v d now I'll stop with that letters that has two properties. I've got enough vectors and not too many. It's a natural idea of a basis. So a basis is a bunch of vectors in the space and it's a so it's a sequence of vectors with two properties with two properties. One, they are independent. And two, you know what's coming? They span the space. Okay, let me take So time for examples of course.
So I'm asking you now to put definition one the definition of independence together with definition two and let's look at examples because this is this combination means the set of of vectors I have is just right and that so that this idea of a basis will be central. I'll always be asking you now for a basis. Whenever I look at a subspace, if I ask you for if you give me a basis for that subspace, you've told me what it is. You've told me everything I need to know about that subspace. Those I take their combinations and I know that I need all the combinations. Okay. Examples. Okay. So, examples of a basis.
Let me start with two-dimensional space. Suppose the the space say example the space is uh let's make it R3 real threedimensional space give me one basis one basis is so I want some vectors because if I ask you for a basis I'm asking you for vectors a little list of vectors and it should be just right. So what would be a basis for three-dimensional space? Well, the first basis that comes to mind, why don't we write that down? The first basis that comes to mind is this vector, this vector, and this vector. Okay, that's one basis. Not the only basis. That's going to be my point. But let's just see.
Yes, that's a basis. Is every are those vectors independent? So that's the like the X, Y, and Z axis. So if those are not independent, we're in trouble. Certainly they are. Take a combination C1 of this vector plus C2 of this vector plus C3 of that vector and try to make it give the zero vector. What are the C's? If C1 of that plus C2 of that plus C3 of that gives me 0 0, then the C's are all zero. Right? So that's the test for independence in the language of matrices. which was under that board. I could make those the columns of a matrix. Well, it would be the identity matrix. Then I would ask what's the null space of the identity matrix.
And you would say it's only the zero vector. And I would say fine then the columns are independent. The only thing the identity times a vector giving zero the only ve vector that does that is zero. Okay. Now, that's not the only basis. Far from it. Tell me another basis. A second basis. Another basis. So, uh, give me well, I'll just start it out. One, one, two, uh, 225. Uh suppose I stop there. Have has that has that see has that have has that little bunch of vectors got the properties that I'm asking for in a basis for R3? We're we're looking for a basis for R3. Are they independent, those two column vectors?
Yes. Do they span R3? No. Our feeling is no. Our feeling is no. Our feeling is that there's some vectors in R3 that are not combinations of those. Okay. So suppose I add in a I need another vector then because they these two don't span the space. Okay. Now it would be foolish for me to put in 337 right as the third vector. That would be a goof because that if I put in 337, those vectors would be dependent, right? If I put in 337, it would be the sum of those two. It would lie in the same plane as those. It wouldn't be independent. I wouldn't I wouldn't I my attempt to create a basis would be dead.
But if I take So what vector can I take? I can take any vector that's not in that plane. Let me try. I I hope that 338 would do it. At least it's not the sum of those two vectors. But I believe that's a basis. And what's the what's the test then for that to be a basis? What? Because I just picked those numbers. And if I had picked uh 57 minus14, how would we know do we have a basis or don't we? You would put them in the columns of a matrix and you would do elimination row reduction and you would see do you get any free variables or are all the columns pivot columns? Well, now actually we have a square.
The matrix would be 3x3. So, so what's the test on the matrix then? The matrix. So, so in this case when I when my space is R3 and I have three vectors, my matrix is square. And what am I asking about that matrix in order for those columns to be a basis? So the so in this in this for Rn if I have n vectors give a basis. If the n byn matrix with those columns with those columns is what what's what's the requirement on that matrix? Invertible. Right. Right. The matrix should be invertible for a square matrix. That's the that's the perfect answer is invertible. So that's when that's when the space is is is the whole space Rn.
Let me let me be sure I I'm with you're with me here. Let me remove that. Are those two vectors a basis for any space at all? Is there a vector space that those really are a basis for? Those that pair of vectors, this guy and this one 112 and 225. Is there a space for which that's a basis? Sure, they're independent. So they satisfy the first requirement. So what space shall I take for them to be a basis of? What spaces will they be a basis for? the one they span they're combinations if it's a plane right it will be a plane inside R3 so if I take this vector 112 say it goes there and this vector 225 say it goes there those are a basis for because they span a plane and they're a basis for the plane because they're independent if I stick in some third guy like 337 which is in the plane Suppose I put in try to put in 337 then the three vectors would still span the plane but they wouldn't be a basis anymore because they're not independent anymore.
Okay. So uh we're looking at the question of so again a the case with in independent columns is the case where the column vectors span the column space. They're independent so they're a basis for the column space. Okay. So now there's one bit of intuition. Let me go back to all of Rn. So where I put 338. Okay. The first message is that the basis is not unique. Right? There's zillions of bases. I take any invertible 3x3 matrix. its columns are a basis for R3. The column space is R3. And if those if that matrix is invertible, those columns are independent. I've got a basis for R3. So there are many many bases, but there is something in common for all those bases.
There's something that this basis shares with that basis and every other basis for R3. And what's that? Well, you saw it coming because when I stopped here and asked if that was a basis for R3, you said no. And I know that you said no because you knew there weren't enough vectors there. And the great fact is that there are many, many bases, but let me put in somebody else just for variety. There are many, many bases, but they all have the same number of vectors. If we're talking about the space R3, then that number of vectors is three. If we're talking about the space Rn, then that number of vectors is N.
If we're talking about some other space, the column space of some matrix or the null space of some matrix or some other space that we haven't even thought of. Then that still is true that every basis that there are lots of bases but every basis has the same number of vectors. Let me write that great fact down. every basis we're we're given a space given a space R3 or Rn or some other column space of a matrix or the null space of a matrix or some other vector space then the great fact is that every basis for the for the space has the same number of vectors. If one basis has six vectors, then every other basis has six vectors.
So that number six is telling me like it's telling me how big is the space. It's telling me how many vectors do I have to have to have a basis. And of course, we're seeing it this way. That number six, if we had seven vectors and we've got too many. If we have five vectors, we haven't got enough. Six is sort of like just right for whatever space that is. And what do we call that number? That number is now I'm ready for the last definition today. It's the dimension of that space. So every basis for a space has the same number of vectors in it. Not the same vectors, all sorts of bases, but the same number of vectors is always the same.
And that number is the dimension. This is definition. This number is the dimension. of the space. Okay. Okay, let's do some examples because now we got definitions. Let me repeat the four things, the four words that have now got defined. Independence that looks at combinations not being zero. Spanning that looks at all the combinations. Basis that's the one that combines independence and spanning. And now we've got the idea of the dimension of a space. It's the number of vectors in any basis because all bases have the same number. Okay. Let's let's take examples. Suppose I take uh my space is examples.
Now space is the say the column space of this matrix. Let me let me write down a matrix. 1 one 2 one two and I'll just to make it clear I'll take the sum there 3 2 3 and let me take the sum of all oh let me put in one yeah I'll put in 11 one again okay so that's four vectors okay do they span the column space of that matrix let me repeat do They span the column space of that matrix. Yes, by definition that's what the column space where it comes from. Are they a basis for the column space? Are they independent? No, they're not independent. There's something in that null space. Maybe we can. So, let's look at the null space of the matrix.
Tell me a vector that's in the null space of that matrix. So I'm looking for some vector that combines those columns and produces the zero column. Or in other words, I'm looking for solutions to ax= z. So so tell me a vector in the null space. Maybe well this was this column was that one plus that one. So maybe if I have one of those and minus one of those that would be a vector in the null space. So you've already told me now are those vectors independent? The answer is those column vectors. The answer is no. Right? They're not independent because ah you knew they weren't independent anyway. Minus one of this minus one of this plus one of this zero of that is the zero vector.
Okay. Okay. So they're not independent. They span but they're not independent. Tell me a basis for that column space. What's a basis for the column space? These are all the questions that the homework asks, the quizzes asked, the final exam will ask. Find a basis for the column space of this matrix. Okay. Now, there's many answers, but give me the most natural answer. columns one and two. Columns one and two. That's the natural answer. Those are the pivot columns because I mean we we begin systematically. We look at the first column. It's okay. We can put that in the basis. We look at the second column.
It's okay. We can put that in the basis. The third column we can't put in the basis. The fourth column we can't again. So the rank of the matrix is what's the rank of our matrix? Two. Two. And and now that rank is also we now have another word. We we have a great theorem here. The rank of A that rank R is the number of pivot columns and it's also well so now please use my new word this it's the number two of course two is the rank of my matrix. It's the number of pivot columns. Those pivot columns form a basis of course. So what's two? It's the dimension. The rank of A, the number of pivot columns is the dimension of the column space.
Of course, you say it had to be right. But just watch look for one moment at the the language, the way the English words get involved here. I take the rank of a matrix. the rank of a matrix. It's a number of columns and it's the dimension of not the dimension of the matrix. That's what I want to say. It's the dimension of a space, a subspace, the column space. Do do you see I I don't take the dimension of A. That's not what I want. I'm looking for the dimension of the column space of A. If you use those words right, it shows you got the idea right. Similarly, here I don't talk about the rank of a subspace.
It's a matrix that has a rank. I talk about the rank of a matrix. And it the beauty is that these definitions just merge so that the rank of a matrix is the dimension of its column space. And in this example, it's two. And then the further question is what's a basis? And the first two columns are a basis. Tell me another basis. Another basis for the column space. You see, I just keep hammering away. I apologize, but it's I have to be sure you have the idea of basis. Tell me another basis for the column space. Well, you could take columns one and three. That would be a basis for the column space.
or columns two and three would be a basis or columns two and four or me another basis that's not made out of those columns at all. So well I guess I'm giving you infinitely many possibilities so I can't expect a unanimous answer here. I tell you let's look at another basis though. I'll just because it's only one out of zillions. I'm going to put it down and I'm going to erase it. Another basis for the column space would be uh let's see I'll put in some things that are not there. say, "Oh, well, just to make it my life easy, 222. That's in the column space." And uh that was sort of obvious. Let me take the sum of those say 6 46 or the sum of all the columns 757.
Why not? That's in the column space. Those are independent. And I've got the number right. I've got two. Actually, this is a key point. If you know the dimension of the space you're working with, and we know that this column, we know that the dimension dime the dimension of the column space is two. If you know the the dimension then then uh and we have a couple of vectors that are independent they'll automatically be a basis. If we've got the number of vectors right, two vectors in this case, then if they're independent, they can't help but span the space because if they didn't span the space, there'd be a third guy to help span the space.
But it couldn't be independent. So it it just has to be independent if we've got the numbers right and they span. Okay, very good. So you got the dimension of a space. So this was another basis that I just invented. Okay. Now, now I get to ask you about the null space. What's the dimension of the null space? So we we got a great fact there. The dimension of the column space is the rank. Now I want to ask you about the null space. That's the other part of the lecture and it'll go on to the next lecture. Okay. So we know the dimension of the column space is two. The rank. What about the null space?
This is a vector in the null space. Are there other vectors in the null space? Yes or no? Yes. So this isn't a basis because it's doesn't span. Right? There's more in the null space than we've got so far. I need another vector at least. So tell me another vector in the null space. Well, the natural choice, the choice you naturally think of is I'm going on to the fourth column. I'm letting that free variable be a one and that free variable be a zero. And I'm asking, is that fourth column a combination of my pivot columns? Yes, it is. And it's that will do. So what I've written there are actually the two special solutions, right?
I took the two free variables free and free. I gave them the values one zero or 01. I figured out the rest. So do do you see? Let me just say it in words. this vector, these vectors in the null space are telling me, they're telling me the combinations of the columns that give zero. They're telling me in what way the the columns are dependent. That's what the null space is doing. Have I got enough now? And what's the null space? Now, we have to think about the null space. These are two vectors in the null space. They're independent. Are they a basis for the null space? What's the dimension of the null space?
You see that those questions just keep coming up all the time. Are they a basis for the null space? You can tell me the answer even though we haven't written out a proof of that. Can you yes or no? Do these two special solutions form a basis for the null space? In other words, does the null space consist of all combinations of those two guys? Yes or no? Yes. Yes. The null space is two-dimensional. The null space, the dimension of the null space is the number of free variables. So the dimension of the null space is the number of free variables and at the last second give me the formula. This is then the key formula that we know how many free variables are there in terms of r the rank m the number of rows n the number of columns what do we get we have n columns r of them are pivot columns so n minus r is the number of free columns free variables and now it's the dimension of the null space.
Okay, that's great. That's the key spaces, their bases, and their dimensions. Thanks,