04 Points and Vectors

04 Points and Vectors#

A list of n numbers can be thought of as a point or a vector in n-dimensional space. Going forward, we will think of n-dimensional vectors \(\begin{bmatrix} a_1\\ a_2\\ \vdots \\ a_ n\end{bmatrix}\) flexibly as points and as vectors.

Dot Products (algebric)#

Notation: We will use regular letters as symbols for numbers, vectors, matrices, planes, hyperplanes, etc. You will need to distinguish what a letter represents from the context.

Recall the dot product of a pair of vectors a and b:

\[\begin{split} \displaystyle a \cdot b = a_1b_1+a_2b_2+\cdots +a_ nb_ n\qquad \text {where }\, a= \begin{bmatrix} a_1\\ a_2 \\ \vdots \\ a_ n \end{bmatrix} \text { and } b= \begin{bmatrix} b_1\\ b_2 \\ \vdots \\ b_ n \end{bmatrix}. \end{split}\]
def dot(x, y):
    return sum(x_i * y_i for x_i, y_i in zip(x, y))

print(dot([1, 2, 3], [4, 5, 6]) == 32)

True

Norm#

The norm (or length) of a vector \(\mathbf{a} = [a_1, a_2, ..., a_n]\) in n-dimensional space is given by the square root of the sum of the squares of its components. This is also known as the Euclidean norm or the 2-norm, and it’s represented as follows:

\[ ||\mathbf{a}|| = \sqrt{a_1^2 + a_2^2 + ... + a_n^2} \]

This formula comes from the Pythagorean theorem and can be thought of as a generalization of the theorem to n dimensions. The norm gives the “length” of the vector from the origin to the point in n-dimensional space represented by the vector.

Here’s how it works with an example:

Let’s say we have a 3-dimensional vector \(\mathbf{a} = [3, 4, 5]\). The norm of this vector would be calculated as follows:

\[ ||\mathbf{a}|| = \sqrt{3^2 + 4^2 + 5^2} = \sqrt{9 + 16 + 25} = \sqrt{50} \approx 7.071 \]

So, the length of the vector \(\mathbf{a} = [3, 4, 5]\) is approximately 7.071.

This formula is a generalization of the Pythagorean theorem to n dimensions. In 2 dimensions, it gives the length of the hypotenuse of a right triangle. In 3 dimensions, it gives the length of the line segment from the origin to a point in space. In n dimensions, it gives the “distance” from the origin to a point in n-dimensional space.

from math import sqrt

def norm(x):
    return sqrt(sum([i**2 for i in x]))

print(norm([1, 2, 3]))
print(norm([0.4, 0.3]))
print(norm([-0.15, 0.2]))

3.7416573867739413
0.5
0.25

Dot Product using Norm (geometric definition)#

The dot product of two vectors a and b in n-dimensional space can also be represented using the norm (or length) of the vectors and the angle between them.

If \(\mathbf{a}\) and \(\mathbf{b}\) are vectors, and \(\theta\) is the angle between them, then the dot product of \(\mathbf{a}\) and \(\mathbf{b}\) is given by:

\[ \mathbf{a} \cdot \mathbf{b} = ||\mathbf{a}|| \, ||\mathbf{b}|| \, \cos(\theta) \]

where \(||\mathbf{a}||\) and \(||\mathbf{b}||\) are the norms (or lengths) of the vectors \(\mathbf{a}\) and \(\mathbf{b}\), respectively, and \(\cos(\theta)\) is the cosine of the angle between the vectors.

This formula is known as the geometric definition of the dot product. It shows that the dot product of two vectors is equal to the product of their lengths and the cosine of the angle between them.

The angle in radians between two vectors can be calculated using the dot product and the norms (magnitudes) of the vectors. The formula is:

\[ \theta = \arccos\left(\frac{\mathbf{a} \cdot \mathbf{b}}{||\mathbf{a}|| , ||\mathbf{b}||}\right) \]

where:

  • \(\theta\) is the angle between the vectors,

  • \(\mathbf{a}\) and \(\mathbf{b}\) are the vectors,

  • \(\mathbf{a} \cdot \mathbf{b}\) is the dot product of the vectors,

  • \(||\mathbf{a}||\) and \(||\mathbf{b}||\) are the norms (magnitudes) of the vectors,

  • \(\arccos\) is the inverse cosine function.

from math import acos, pi

def angle(x, y):
    return acos(dot(x, y) / (norm(x) * norm(y)))

print(angle([1, 2, 3], [4, 5, 6]))
print(angle([0.4, 0.3], [-0.15, 0.2]))

0.2257261285527342
1.5707963267948966

Dot product and orthogonal (or perpendicular) vectors#

Two vectors are orthogonal (or perpendicular or \(\pi /2\)) to each other if their dot product is zero.

In the case of 3-dimensional vectors \(\mathbf{x} = [x_1, x_2, x_3]\) and \(\mathbf{y} = [y_1, y_2, y_3]\), they are orthogonal if:

\[ \mathbf{x} \cdot \mathbf{y} = x_1y_1 + x_2y_2 + x_3y_3 = 0 \]

This is because the dot product of two vectors is equal to the product of their magnitudes and the cosine of the angle between them. If the vectors are orthogonal, the angle between them is 90 degrees, and the cosine of 90 degrees is zero, making the entire dot product zero.

e.g. Two vectors are orthogonal (or perpendicular) to each other if their dot product is zero.

In the case of 3-dimensional vectors \(\mathbf{x} = [a_1, a_2, a_3]\) and \(\mathbf{y} = [a_1, -a_2, a_3]\), they are orthogonal if:

\[ \mathbf{x} \cdot \mathbf{y} = a_1*a_1 + a_2*(-a_2) + a_3*a_3 = 0 \]

Simplifying, we get:

\[ a_1^2 - a_2^2 + a_3^2 = 0 \]

So, the vectors \(\mathbf{x}\) and \(\mathbf{y}\) are orthogonal when \(a_1^2 - a_2^2 + a_3^2 = 0\).

Unit Vector#

A unit vector is a vector of length 1.

If you have a vector \(\mathbf{x} = [x_1, x_2, ..., x_n]\), you can calculate its unit vector \(\hat{\mathbf{x}}\) by dividing each component of the vector by its norm (or length).

The formula to calculate the unit vector is:

\[ \hat{\mathbf{x}} = \frac{\mathbf{x}}{||\mathbf{x}||} \]

where \(||\mathbf{x}||\) is the norm of the vector \(\mathbf{x}\), calculated as \(||\mathbf{x}|| = \sqrt{x_1^2 + x_2^2 + ... + x_n^2}\).

So, if you have a vector \(\mathbf{x} = [3, 4]\), its norm is \(||\mathbf{x}|| = \sqrt{3^2 + 4^2} = 5\). The unit vector would then be \(\hat{\mathbf{x}} = [3/5, 4/5] = [0.6, 0.8]\).

The unit vector has the same direction as the original vector, but its length is 1.

Projection#

In linear algebra, the projection of one vector onto another is a vector that is the “shadow” of one vector (the vector being projected) along another vector (the vector onto which the first vector is being projected).

Let’s say we have two vectors, \(\mathbf{a}\) and \(\mathbf{b}\). The projection of \(\mathbf{a}\) onto \(\mathbf{b}\) is a vector that lies on the line defined by \(\mathbf{b}\) and is closest to \(\mathbf{a}\).

The formula to calculate the projection of \(\mathbf{a}\) onto \(\mathbf{b}\) is:

\[ \text{proj}_{\mathbf{b}}\mathbf{a} = \frac{\mathbf{a} \cdot \mathbf{b}}{||\mathbf{b}||^2} \mathbf{b} \]

where:

  • \(\mathbf{a} \cdot \mathbf{b}\) is the dot product of \(\mathbf{a}\) and \(\mathbf{b}\),

  • \(||\mathbf{b}||\) is the norm (or length) of \(\mathbf{b}\),

  • \(\mathbf{b}\) is the vector onto which \(\mathbf{a}\) is being projected.

This formula gives us a vector that is in the direction of \(\mathbf{b}\) and has length equal to the component of \(\mathbf{a}\) in the direction of \(\mathbf{b}\).

Signed magnitude of projection#

The signed magnitude of the projection of one vector onto another, often referred to as the scalar projection, is a scalar value that represents how much of one vector goes in the direction of another vector.

Let’s say we have two vectors, \(\mathbf{a}\) and \(\mathbf{b}\). The signed magnitude \(c\) of the projection of \(\mathbf{a}\) onto \(\mathbf{b}\) is given by:

\[ c = \frac{\mathbf{a} \cdot \mathbf{b}}{||\mathbf{b}||} \]

where:

  • \(\mathbf{a} \cdot \mathbf{b}\) is the dot product of \(\mathbf{a}\) and \(\mathbf{b}\),

  • \(||\mathbf{b}||\) is the norm (or length) of \(\mathbf{b}\).

This formula gives us a scalar value that is positive if the angle between \(\mathbf{a}\) and \(\mathbf{b}\) is less than 90 degrees (i.e., \(\mathbf{a}\) is in the same general direction as \(\mathbf{b}\)), negative if the angle is greater than 90 degrees (i.e., \(\mathbf{a}\) is in the opposite general direction), and zero if the vectors are orthogonal (i.e., at right angles to each other).

In the case of 3-dimensional vectors \(\mathbf{a} = [a_1, a_2, a_3]\) and \(\mathbf{b} = [a_1, -a_2, a_3]\)

\[ (a_1^2 - a_2^2 + a_3^2)/\sqrt{a_1^2 + a_2^2 + a_3^2} \]
Contents