On-Line Geometric Modeling Notes
VECTOR SPACES

These notes give the definition of a vector space and several of the concepts related to these spaces. Examples are drawn from the vector space of vectors in $\Re^2$.

For a pdf version of these notes look here.

Definition of a Vector Space

A nonempty set ${\cal V}$ of elements ${\vec{v} } , {\vec{w} } , ...$ is called a vector space if in ${\cal V}$ there are two algebraic operations (called addition and scalar multiplication), so that the following properties hold.

Addition associates with every pair of vectors ${\vec{v} } _1$ and ${\vec{v} } _2$ a unique vector ${\vec{v} } \in {\cal V}$ which is called the sum of ${\vec{v} } _1$ and ${\vec{v} } _2$ and is written ${\vec{v} } _1+ {\vec{v} } _2$. In the case of the space of 2-dimensional vectors, the summation is componentwise (i.e. if ${\vec{v} } _1 = <x_1, y_1>$ and ${\vec{v} } _2 = <x_2, y_2>$, then ${\vec{v} } _1+ {\vec{v} } _2 = <x_1+x_2,y_1+y_2>$), which can be best illustrated by the ``parallelogram illustration'' below:

Addition satisfies the following :

• Commutativity - for any two vectors ${\vec{v} } _1$ and ${\vec{v} } _2$ in ${\cal V}$,
  $${\vec{v} } _1 + {\vec{v} } _2 \: = \: {\vec{v} } _2 + {\vec{v} } _1$$

• Associativity - for any three vectors ${\vec{v} } _1$, ${\vec{v} } _2$ and ${\vec{v} } _3$ in ${\cal V}$,
  $$( {\vec{v} } _1 + {\vec{v} } _2 ) + {\vec{v} } _3 \: = \: {\vec{v} } _1 + ( {\vec{v} } _2 + {\vec{v} } _3 )$$
  This rule is illustrated in the figure below. One can see that even though the sum ${\vec{v} } _1+ {\vec{v} } _2+ {\vec{v} } _3$ is calculated differently, the result is the same.
  

• Zero Vector - there is a unique vector in ${\cal V}$ called the zero vector and denoted ${\vec{0} }$ such that for every vector ${\vec{v} } \in {\cal V}$
  $${\vec{v} } + {\vec{0} } \: = \: {\vec{0} } + {\vec{v} } \: = \: {\vec{v} }$$

• Additive Inverse - for each element ${\vec{v} } \in {\cal V}$, there is a unique element in ${\cal V}$, usually denoted $- {\vec{v} }$, so that
  $${\vec{v} } + (- {\vec{v} } ) \: = \: {\vec{0} }$$
  In the case of 2-dimensional vectors, $- {\vec{v} }$ is simply represented as the vector of equal magnitude to ${\vec{v} }$, but in the opposite direction.
  

  The use of an additive inverse allows us to define a subtraction operation on vectors. Simply ${\vec{v} } _2 - {\vec{v} } _1 = {\vec{v} } _2 + (- {\vec{v} } _1)$ The result of vector subtraction in the space of 2-dimensional vectors is shown below.

  

  Frequently this 2-d vectors is protrayed as joining the ends of the two original vectors. As we can see, since the vectors are determined by direction and length, and not position, the two vectors are equivalent.

  

Scalar Multiplication associates with every vector ${\vec{v} } \in {\cal V}$ and every scalar $c$, another unique vector (usually written $c {\vec{v} }$),

For scalar multiplication the following properties hold:

• Distributivity - for every scalar $c$ and vectors ${\vec{v} } _1$ and ${\vec{v} } _2$ in ${\cal V}$,
  $$c ( {\vec{v} } _1 + {\vec{v} } _2 ) \: = \: c {\vec{v} } _1 + c {\vec{v} } _2$$
  In the case of 2-dimensional vectors, this can be easily seen by extending the parallelogram illustration above. We can see below that the sum of the vectors $2 {\vec{v} } _1$ and $2 {\vec{v} } _2$ is just twice the vector ${\vec{v} } _1+ {\vec{v} } _2$
  

• Distributivity of Scalars - for every two scalars $c_1$ and $c_2$ and vector ${\vec{v} } \in {\cal V}$,
  $$( c_1 + c_2 ) {\vec{v} } \: = \: c_1 {\vec{v} } + c_2 {\vec{v} }$$

• Associativity - for every two scalars $c_1$ and $c_2$ and vector ${\vec{v} } \in {\cal V}$,
  $$c_1 ( c_2 {\vec{v} } ) \: = \: ( c_1 c_2 ) {\vec{v} }$$

• Identity - for every vector ${\vec{v} } \in {\cal V}$,
  $$1 {\vec{v} } \: = \: {\vec{v} }$$

Examples of Vector Spaces

Examples of vector space abound in mathematics. The most obvious examples are the usual vectors in $\Re^2$, from which we have drawn our illustrations in the sections above. But we frequently utilize several other vectors spaces: The 3-d space of vectors, the vector space of all polynomials of a fixed degree, and vector spaces of $n \times n$ matrices. We briefly discuss these below.

The Vector Space of 3-Dimensional Vectors

The vectors in $\Re^3$ also form a vector space, where in this case the vector operations of addition and scalar multiplication are done componentwise. That is ${\vec{v} } _1 = <x_1,y_1,z_1>$ and ${\vec{v} } _2 = <x_2,y_2,z_2>$ are vectors, then addition is

$${\vec{v} } _1 + {\vec{v} } _2 = < x_1+x_2, y_1+y_2, z_1+z_2 >$$

and, if $c$ is a scalar, scalar multiplication is given by

$$c {\vec{v} } _1 = < c x_1, c y_1, c z_1 >$$

The axioms are easily verified (for example the additive identity of ${\vec{v} } _1 = <x_1,y_1,z_1>$ is just $- {\vec{v} } _1 = <-x_1,-y_1,-z_1>$, and the zero vector is just ${\vec{0} } = < 0,0,0 >$. Here the axioms just state what we always have been taught about these sets of vectors.

Vector Spaces of Polynomials

The set of quadratic polynomials of the form

$$P(x) \: = \: a x^2 + bx + c$$

also form a vector space. We add two of polynomials by adding their respective coefficients. That is, if $p_1(x) = a_1 x^2 + b_1 x + c_1$ and $p_2(x) = a_2 x^2 + b_2 x + c_2$, then

$$(p_1+p_2)(x) \: = \: (a_1+a_2) x^2 + (b_1+b_2) x + (c_1+c_2)$$

and multiplication is done by multiplying the scalar by each coefficient. That is, if $s$ is a scalar, then

$$s p(x) \: = \: (s a) x^2 + (s b) x + (s c)$$

The axioms are again easily verified by performing the operations individually on like terms.

A simple extension of the above is to consider the set of polynomials of degree less than or equal to $n$. It is easily seen that these also form a vector space.

Vector Spaces of Matrices

The set of $n \times n$ Matrices form a vector space. Two matrices can be added componentwise, and a matrix can be multiplied by a scalar. All axioms are easily verified.

Linear Independence and Bases

Given a vector space ${\cal V}$, the concept of a basis for the vector space is fundamental for much of the work that we will do in computer graphics. This section discusses several topics relating to linear combinations of vectors, linear independence and bases.

Linear Combinations

Let ${\vec{v} } _1, {\vec{v} } _2, ..., {\vec{v} } _n$ be any vectors in a vector space ${\cal V}$ and let $c_1, c_2, ..., c_n$ be any set of scalars. Then an expression of the form

$$c_1 {\vec{v} } _1 + c_2 {\vec{v} } _2 + \cdots + c_n {\vec{v} } _n$$

is called a linear combination of the vectors.

This element is clearly a member of the vector space ${\cal V}$ (just repeatedly apply the summation and scalar multiplication axioms).

The set $S$ that contains all possible linear combinations of ${\vec{v} } _1, {\vec{v} } _2, ..., {\vec{v} } _n$ is called the span of ${\vec{v} } _1, {\vec{v} } _2, ..., {\vec{v} } _n$. We frequently say that $S$ is spanned (or generated) by those $n$ vectors.

It is straightforward to show that the span of any set of vectors is again a vector space.

Linear Independence

Given a set of vectors ${\vec{v} } _1, {\vec{v} } _2, ..., {\vec{v} } _n$ from a vector space ${\cal V}$. This set is called linearly independent in ${\cal V}$ if the equation

$$c_1 {\vec{v} } _1 + c_2 {\vec{v} } _2 + \cdots + c_n {\vec{v} } _n \: = \: {\vec{0} }$$

implies that $c_i = 0$ for all $i = 1, 2, ..., n$.

If a set of vectors is not linearly independent, then it is called linearly dependent. This implies that the equation above has a nonzero solution, that is there exist $c_1, c_2, ..., c_n$ which are not all zero, such that

$$c_1 {\vec{v} } _1 + c_2 {\vec{v} } _2 + \cdots + c_n {\vec{v} } _n \: = \: {\vec{0} }$$

This implies that at least one of the vectors ${\vec{v} } _i$ can be written in terms of the other $n-1$ vectors in the set. Assuming that $c_1$ is not zero, we can see that

$${\vec{v} } _1 \: = \: \frac{c_2}{c_1} {\vec{v} } _2 + \cdots + \frac{c_n}{c_1} {\vec{v} } _n$$

Any set of vectors containing the zero vector ( ${\vec{0} }$) is linearly dependent.

Example

To give an example of a linear independent set that everyone has seen, consider the three vectors

$$\vec{i} = <1,0,0>, \;\; \vec{j} = <0,1,0>, \;\; \vec{k} = <0,0,1>$$

in the vector space of vectors in $\Re^3$

Consider the equation

$$c_1 \vec{i} + c_2 \vec{j} + c_3 \vec{k} \: = \: {\vec{0} }$$

If we simplify left-hand side by performing the operations componentwise and write the right-hand side componentwise, we have

$$<c_1,c_2,c_3> = <0,0,0>$$

which can only be solved if $c_1 = c_2 = c_3 = 0$.

A Basis for a Vector Space

Let ${\vec{v} } _1, {\vec{v} } _2, ..., {\vec{v} } _n$ be a set of vectors in a vector space ${\cal V}$ and let $S$ be the span of ${\cal V}$. If ${\vec{v} } _1, {\vec{v} } _2, ..., {\vec{v} } _n$ is linearly independent, then we say that these vectors form a basis for $S$ and $S$ has dimension $n$. Since these vectors span $S$, any vector ${\vec{v} } \in S$ can be written uniquely as

$${\vec{v} } \: = \: c_1 {\vec{v} } _1 + c_2 {\vec{v} } _2 + \cdots + c_n {\vec{v} } _n$$

The uniqueness follows from the argument that if there were two such representations

$$\begin{aligned}{\vec{v} } & = c_1 {\vec{v} } _1 + c_2 {\vec{v... ... c_2' {\vec{v} } _2 + \cdots + c_n' {\vec{v} } _n \end{aligned}$$

then by subtracting the two equations, we obtain

$${\vec{0} } \: = \: (c_1-c_1') {\vec{v} } _1 + (c_2-c_2') {\vec{v} } _2 + \cdots + (c_n-c_n') {\vec{v} } _n$$

which can only happen if all the expressions $c_i-c_i'$ are zero, since the vectors ${\vec{v} } _1, {\vec{v} } _2, ..., {\vec{v} } _n$ are assumed to be linearly independent. Thus we necessarily have that $c_i = c_i'$ for all $i=1,2,...,n$.

If $S$ is the entire vector space ${\cal V}$, we say that ${\vec{v} } _1, {\vec{v} } _2, ..., {\vec{v} } _n$ forms a basis for ${\cal V}$, and ${\cal V}$ has dimension $n$.