1.11. Basis, Dimension, Rank and Nullity

Basis §

Basis for vector space vs subspace

Basis can be defined both for vector spaces and subspaces. But usually if nothing is mentioned it is typical to infer them as basis for vector space.

Definition: Basis of a vector space (subspace) $V$ is a set of vectors $S$ that are:

• Linearly independent.
• Spans the entire vector space i.e. $\mathrm{span}(S)=V$

Every basis of a given vector space has the same size.

Basis vectors do not need to be orthogonal.

It is common to develop this misconception, because most of the time the basis we work with are orthogonal. They are referred to as orthogonal basis. But in general basis do not need to be orthogonal.

In a sense, basis is the most efficient spanning set for vector space (or subspace). By most efficient, I mean there is “no redundant vector” which can be formed from the other vectors in the basis set.

Let’s go through a concrete example to get a better understanding. Let’s restrict ourselves to ${\mathbb{R}}^3$.

Consider two vectors in ${\mathbb{R}}^3$,

$$\left\{\left[\begin{array}{c}1\\ 0\\ 0\end{array}\right],\left[\begin{array}{c}1\\ 1\\ 0\end{array}\right]\right\}$$

• Are they linearly independent? : Yes, because it is not possible to generate one vector from the other by taking linear combination. In other word, one vector in this set do lie in the span of the other vector.
• Do they span the entire vector space ${\mathbb{R}}^3$ : No, because they span of these two vectors only fills in a plane in ${\mathbb{R}}^3$ which is a subspace of ${\mathbb{R}}^3$, not the entire ${\mathbb{R}}^3$. Therefore, it is not possible to generate any vector in ${\mathbb{R}}^3$ by taking the linear combination of these two vectors.

Linearly Independent Set	✔️
Basis	❌
Spanning Set	❌

Now consider the following three vectors:

$$\left\{\left[\begin{array}{c}1\\ 0\\ 0\end{array}\right],\left[\begin{array}{c}1\\ 1\\ 0\end{array}\right],\left[\begin{array}{c}1\\ 1\\ 1\end{array}\right]\right\}$$

• Are they linearly independent?: Yes.
• Do they span the entire vector space ${\mathbb{R}}^3$ : Yes

Linearly Independent Set	✔️
Basis	✔️
Spanning Set	✔️

Now consider the following four vectors:

$$\left\{\left[\begin{array}{c}1\\ 0\\ 0\end{array}\right],\left[\begin{array}{c}1\\ 1\\ 0\end{array}\right],\left[\begin{array}{c}1\\ 1\\ 1\end{array}\right],\left[\begin{array}{c}2\\ 2\\ 2\end{array}\right]\right\}$$

• Are they linearly independent?: No. Because we have a redundant vector ${\left[\begin{array}{ccc}1& 1& 1\end{array}\right]}^{\top }$, which can be formed by linear combination of the other vectors.
• Do they span the entire vector space ${\mathbb{R}}^3$ : Yes. Even though there is a redundant vector, these vectors span the entire ${\mathbb{R}}^3$.

Linearly Independent Set	❌
Basis	❌
Spanning Set	✔️

Based on the discussion above we can claim the following:

$$\mid \text{Linearly Independent Set}\mid \le \mid \text{Basis}\mid \le \mid \text{Spanning Set}\mid$$

This means:

• Every spanning set contains a basis.
• Every linearly independent set can be extended to a basis.

Another interpretation of the equation above is:

• Basis is a maximal linearly independent set. This means if any other vector is added to the set, it will no longer be a linearly independent set.
• Basis is a minimal spanning set . This means if any vector is removed from the set it will no longer be a spanning set.

Dimension §

Dimension is defined as the number of basis vectors for a vector space (or subspace). For example, in ${\mathbb{R}}^3$, the basis consist of three basis vectors hence, the dimension of ${\mathbb{R}}^3$ is $3$. Mathematically, we would write that as $\dim ({\mathbb{R}}^3)=3$. In general for a $n$ dimensional vectors space, $\dim ({\mathbb{R}}^n)=n$.

In a sense, dimension measure the “degrees of freedom” of a vector space or subspace.

Rank §

One of the fundamental theorem in linear algebra is column rank equals row rank. Because of this, normally no distinction is made when the term “rank” is used. But let’s go through it without this notion.

Column Rank §

Definition: The column rank of a matrix is the dimension of the column space (or range or image) of a matrix.

$$Colrank(\mathbf{A}):=\dim (Range(\mathbf{A}))=\dim (Col(\mathbf{A}))=\dim (\mathrm{span}({\mathbf{a}}_1,{\mathbf{a}}_2,\cdots ,{\mathbf{a}}_n))$$

This definition can be interpreted as follows:

The column rank refers to the number of linearly independent columns.

Row Rank §

Definition: The row rank of a matrix is the dimension of the row space of a matrix.

$$Rowrank(\mathbf{A}):=\dim (Range({\mathbf{A}}^{\top }))=\dim (Col({\mathbf{A}}^{\top }))=\dim (\mathrm{span}({\mathbf{a}}_1^{\top },{\mathbf{a}}_2^{\top },\cdots ,{\mathbf{a}}_n^{\top }))$$

Here, we used a trick which is taking the transpose of the matrix, which coverts the rows into columns. This means we can just use the definitions for the column rank with the transposed version of the matrix.

The definition can be interpreted as follows:

The row rank refers to the number of linearly independent rows.

Some facts about rank:

• If a matrix has a rank of $r$ that means there are $r$ linearly independent columns and $r$ linearly independent rows.
• If a matrix is full rank that means all the columns and rows are linearly independent columns.
• Column rank is always equal to row rank.
• For a matrix ${\mathbf{A}}^{m\times n}$ , $Rank(\mathbf{A})\le \min (m,n)$. This can be deduced from the fact that row rank is always equal to column rank.
• For a full rank matrix $\mathrm{Rank}(\mathbf{A})=\min (m,n)$ .

Nullity §

Nullity is defined as the dimension of the null space.

$$\mathrm{nullity}(\mathbf{A}):=\dim (\mathrm{Ker}(\mathbf{A}))$$