10 Matrices and Vectors#
Matrices and vectors are fundamental concepts in linear algebra.
A matrix is a rectangular array of numbers arranged in rows and columns. For example, a 2x3 matrix has 2 rows and 3 columns:
A = [[a, b, c],
[d, e, f]]
A vector is a special type of matrix that has only one row or one column. For example, a column vector with 3 elements looks like this:
v = [[x],
[y],
[z]]
The product of a matrix A and a vector v is another vector Av that’s obtained by multiplying each row of A by v and summing the results. In order for this product to be defined, the number of columns in A must be equal to the number of rows in v. In general If you have a row vector and a matrix, the vector should be on the left of the product i.e. v.A, and if you have a column vector and a matrix, the vector should be on the right of the product i.e. A.v.
M = [[a, b],
[c, d]]
v = [[x],
[y]]
the product of Mv would be
Mv = [[ax + by],
[cx + dy]]
The inverse of a square matrix A (a matrix with the same number of rows and columns) is a matrix A^-1 such that the product AA^-1 is the identity matrix. The identity matrix is a special square matrix that has 1s on the diagonal and 0s elsewhere. Not all matrices have an inverse; if an inverse exists, it can be computed using various algorithms.
Here’s an example in Python using the numpy library. The code first computes the product of A and v, then computes the inverse of A, and finally verifies that AA^-1 is the identity matrix.
import numpy as np
# Define a 3x3 matrix A and a row vector v
A = np.array([[1, 2, 3], [4, 5, 6], [1, 2, 1]])
v = np.array([2, 1, 3])
# Compute the product Av
Av = np.dot(A, v)
print(Av)
# Compute the inverse of A
A_inv = np.linalg.inv(A)
print(A_inv)
# Verify that AA^-1 is the identity matrix
I = np.dot(A, A_inv)
print(I)
is_identity = np.allclose(I, np.eye(I.shape[0]))
print(is_identity)
[13 31 7]
[[-1.16666667 0.66666667 -0.5 ]
[ 0.33333333 -0.33333333 1. ]
[ 0.5 -0. -0.5 ]]
[[1.00000000e+00 0.00000000e+00 0.00000000e+00]
[4.44089210e-16 1.00000000e+00 0.00000000e+00]
[1.11022302e-16 0.00000000e+00 1.00000000e+00]]
True
Matrix Multipilication#
The size of the output is the number of rows of the left matrix, and the number of columns of the right matrix.
Let \(\mathbf{A}= \begin{pmatrix} 1 & -1 & 2 \\ 0 & 3 & -4 \end{pmatrix}\) and \(\mathbf{B}= \begin{pmatrix} -1 & 0 & 0 \\ 2 & 0 & 1 \\ 0 & 1 & 3 \end{pmatrix}\). The dimensions of the product AB are 2x3.
If A = \(m \times n\) and B is \(n \times k\) then AB = \(m \times k\). if C is \(k \times j\) then ABC = \(m \times j\)
Vector Inner and Outer Product#
Inner Product (Dot Product)
The inner product of two vectors is a single number (scalar) obtained by multiplying corresponding entries and then summing those products.
If we have two vectors a = [a1, a2, ..., an] and b = [b1, b2, ..., bn], their inner product is calculated as:
a . b = a1*b1 + a2*b2 + ... + an*bn
In Python you can either define your own function or use numpy.dot
import numpy as np
# Define two vectors
a = np.array([1, 3])
b = np.array([-1, 1])
def dot(x, y):
return sum(x_i * y_i for x_i, y_i in zip(x, y))
# Compute the inner product / dot product
print(f"Inner product using np.dot: {np.dot(a, b)}")
print(f"Inner product using user defined dot: {dot(a, b)}")
Inner product using np.dot: 2
Inner product using user defined dot: 2
Outer Product
The outer product of two vectors is a matrix where each element is a product of the elements of the original vectors.
If we have two vectors a = [a1, a2, ..., an] and b = [b1, b2, ..., bn], their outer product is a matrix:
a x b = [[a1*b1, a1*b2, ..., a1*bn],
[a2*b1, a2*b2, ..., a2*bn],
...
[an*b1, an*b2, ..., an*bn]]
Here’s how you can compute the outer product of two vectors in Python:
import numpy as np
# Define two vectors
a = np.array([1, 3])
b = np.array([-1, 1])
# Compute the outer product
outer_product = np.outer(a, b)
print(outer_product)
[[-1 1]
[-3 3]]
In this example, the outer product of a and b is a 3x3 matrix where each element is a product of the corresponding elements of a and b.
Matrix Transpose#
The transpose of a matrix A is obtained by flipping the matrix over its diagonal. This swaps the row and column indices of each element.
For a matrix A:
The transpose A' is:
In Python, you can compute the transpose of a matrix using the numpy.transpose function or the T attribute of a numpy array:
import numpy as np
# Define the matrix A
A = np.array([[1, 2, 3], [4, 5, 6], [1, 2, 1]])
# Compute the transpose of A
A_transpose = A.T
print(A_transpose)
[[1 4 1]
[2 5 2]
[3 6 1]]
Matrix Determinant
The determinant of a square matrix is a special number that can be calculated from its elements. For a 2x2 matrix it has the forumule
The determinant det(A) of a 3x3 matrix A:
is calculated as
In Python, you can compute the determinant of a matrix using the numpy.linalg.det function:
import numpy as np
# Define the matrix A
A = np.array([[1, 2, 3], [4, 5, 6], [1, 2, 1]])
# Compute the determinant of A
A_det = np.linalg.det(A)
At_det = np.linalg.det(A.T)
print(f"Determinant of A: {A_det}")
print(f"Determinant of A transpose: {At_det}")
Determinant of A: 6.0
Determinant of A transpose: 6.0
Rank of Matrix#
The row rank of a matrix is the maximum number of linearly independent rows in the matrix. The column rank of a matrix is the maximum number of linearly independent columns in the matrix. For any given matrix the row rank and column rank are always equal. This is known as the rank theorem in linear algebra.
Given the matrix A:
We can see that the second row is a multiple of the first row 2*1 = 2, 2*3 = 6. Therefore, these two rows are linearly dependent, and the row rank of A is 1. Similarly the second column is a multiple of the first column 3*1 = 3, 3*2 = 6.
In Python, you can compute the rank of a matrix using the numpy.linalg.matrix_rank function:
import numpy as np
# Define the matrix A
A = np.array([[2, 1, 0], [1, 4, 4], [5, 6, 4]])
# Compute the rank of A
rank_A = np.linalg.matrix_rank(A)
print(rank_A) # Output: 1
2