Matrix Inversion
We defined the inverse of a square matrix \(M\) is a matrix of the same size, \(M^{-1}\), such that \(M \cdot M^{-1} = M^{-1} \cdot M = I\). If the dimension of the matrix is high, the analytic solution for the matrix inversion will be complicated. Therefore, we need some other efficient ways to get the inverse of the matrix.
Let us use a \(4 \times 4\) matrix for illustration. If we have \(M = \begin{bmatrix}
m_{1,1} & m_{1,2} & m_{1,3} & m_{1,4}\\
m_{2,1} & m_{2,2} & m_{2,3} & m_{2,4}\\
m_{3,1} & m_{3,2} & m_{3,3} & m_{3,4} \\
m_{4,1} & m_{4,2} & m_{4,3} & m_{4,4}
\end{bmatrix}\), and the inverse of \(M\) is \(X = \begin{bmatrix}
x_{1,1} & x_{1,2} & x_{1,3} & x_{1,4}\\
x_{2,1} & x_{2,2} & x_{2,3} & x_{2,4}\\
x_{3,1} & x_{3,2} & x_{3,3} & x_{3,4} \\
x_{4,1} & x_{4,2} & x_{4,3} & x_{4,4}
\end{bmatrix}\), therefore, we will have:
\[\begin{split}M \cdot X = \begin{bmatrix}
m_{1,1} & m_{1,2} & m_{1,3} & m_{1,4}\\
m_{2,1} & m_{2,2} & m_{2,3} & m_{2,4}\\
m_{3,1} & m_{3,2} & m_{3,3} & m_{3,4} \\
m_{4,1} & m_{4,2} & m_{4,3} & m_{4,4}
\end{bmatrix} \begin{bmatrix}
x_{1,1} & x_{1,2} & x_{1,3} & x_{1,4}\\
x_{2,1} & x_{2,2} & x_{2,3} & x_{2,4}\\
x_{3,1} & x_{3,2} & x_{3,3} & x_{3,4} \\
x_{4,1} & x_{4,2} & x_{4,3} & x_{4,4}
\end{bmatrix} = \begin{bmatrix}
1 & 0 & 0 & 0\\
0 & 1 & 0 & 0\\
0 & 0 & 1 & 0\\
0 & 0 & 0 & 1
\end{bmatrix}\end{split}\]
We can rewrite the above equation to four separate equations, such as:
\[\begin{split}
\begin{bmatrix}
m_{1,1} & m_{1,2} & m_{1,3} & m_{1,4}\\
m_{2,1} & m_{2,2} & m_{2,3} & m_{2,4}\\
m_{3,1} & m_{3,2} & m_{3,3} & m_{3,4} \\
m_{4,1} & m_{4,2} & m_{4,3} & m_{4,4}
\end{bmatrix}\left[\begin{array}{c} x_{1,1} \\x_{2,1} \\ x_{3,1} \\x_{4,1} \end{array}\right] =
\left[\begin{array}{c} 1\\0 \\0 \\0 \end{array}\right]\end{split}\]
\[\begin{split}
\begin{bmatrix}
m_{1,1} & m_{1,2} & m_{1,3} & m_{1,4}\\
m_{2,1} & m_{2,2} & m_{2,3} & m_{2,4}\\
m_{3,1} & m_{3,2} & m_{3,3} & m_{3,4} \\
m_{4,1} & m_{4,2} & m_{4,3} & m_{4,4}
\end{bmatrix}\left[\begin{array}{c} x_{1,2} \\x_{2,2} \\ x_{3,2} \\x_{4,2} \end{array}\right] =
\left[\begin{array}{c} 0\\1 \\0 \\0 \end{array}\right]\end{split}\]
\[\begin{split}
\begin{bmatrix}
m_{1,1} & m_{1,2} & m_{1,3} & m_{1,4}\\
m_{2,1} & m_{2,2} & m_{2,3} & m_{2,4}\\
m_{3,1} & m_{3,2} & m_{3,3} & m_{3,4} \\
m_{4,1} & m_{4,2} & m_{4,3} & m_{4,4}
\end{bmatrix}\left[\begin{array}{c} x_{1,3} \\x_{2,3} \\ x_{3,3} \\x_{4,3} \end{array}\right] =
\left[\begin{array}{c} 0\\0 \\1 \\0 \end{array}\right]\end{split}\]
\[\begin{split}
\begin{bmatrix}
m_{1,1} & m_{1,2} & m_{1,3} & m_{1,4}\\
m_{2,1} & m_{2,2} & m_{2,3} & m_{2,4}\\
m_{3,1} & m_{3,2} & m_{3,3} & m_{3,4} \\
m_{4,1} & m_{4,2} & m_{4,3} & m_{4,4}
\end{bmatrix}\left[\begin{array}{c} x_{1,4} \\x_{2,4} \\ x_{3,4} \\x_{4,4} \end{array}\right] =
\left[\begin{array}{c} 0\\0 \\0 \\1 \end{array}\right]\end{split}\]
Therefore, if we solve the above four system of equations, we will get the inverse of the matrix. We can use any method we introduced previously to solve these equations, such as Gauss Elimination, Gauss-Jordan, and LU decomposition. Here, we will just show an example of matrix inversion using Gauss-Jordan method.
Recall that, in Gauss-Jordan method, we convert our problem from
\[\begin{split}
\begin{bmatrix}
m_{1,1} & m_{1,2} & m_{1,3} & m_{1,4}\\
m_{2,1} & m_{2,2} & m_{2,3} & m_{2,4}\\
m_{3,1} & m_{3,2} & m_{3,3} & m_{3,4} \\
m_{4,1} & m_{4,2} & m_{4,3} & m_{4,4}
\end{bmatrix}\left[\begin{array}{c} x_1 \\x_2 \\ x_3 \\x_4 \end{array}\right] =
\left[\begin{array}{c} y_1 \\y_2 \\ y_3 \\y_4 \end{array}\right]\end{split}\]
to
\[\begin{split}
\begin{bmatrix}
1 & 0 & 0 & 0\\
0 & 1 & 0 & 0\\
0 & 0 & 1 & 0\\
0 & 0 & 0 & 1
\end{bmatrix} \left[\begin{array}{c} x_1 \\x_2 \\ x_3 \\x_4 \end{array}\right] =
\left[\begin{array}{c} y_1' \\y_2' \\ y_3' \\y_4' \end{array}\right]\end{split}\]
and get the solution. Essentially, we are converting
\[\begin{split}
\begin{bmatrix}
m_{1,1} & m_{1,2} & m_{1,3} & m_{1,4} & y_1\\
m_{2,1} & m_{2,2} & m_{2,3} & m_{2,4} & y_2\\
m_{3,1} & m_{3,2} & m_{3,3} & m_{3,4} & y_3 \\
m_{4,1} & m_{4,2} & m_{4,3} & m_{4,4} & y_4
\end{bmatrix}
\end{split}\]
to
\[\begin{split}
\begin{bmatrix}
1 & 0 & 0 & 0 & y_1'\\
0 & 1 & 0 & 0 & y_2'\\
0 & 0 & 1 & 0 & y_3'\\
0 & 0 & 0 & 1 & y_4'
\end{bmatrix}
\end{split}\]
Let us generalize it here, all we need to do is to convert
\[\begin{split}
\begin{bmatrix}
m_{1,1} & m_{1,2} & m_{1,3} & m_{1,4} & 1 & 0 & 0 & 0\\
m_{2,1} & m_{2,2} & m_{2,3} & m_{2,4} & 0 & 1 & 0 & 0\\
m_{3,1} & m_{3,2} & m_{3,3} & m_{3,4} & 0 & 0 & 1 & 0\\
m_{4,1} & m_{4,2} & m_{4,3} & m_{4,4} & 0 & 0 & 0 & 1
\end{bmatrix}
\end{split}\]
to
\[\begin{split}
\begin{bmatrix}
1 & 0 & 0 & 0 & m_{1,1}' & m_{1,2}' & m_{1,3}' & m_{1,4}'\\
0 & 1 & 0 & 0 & m_{2,1}' & m_{2,2}' & m_{2,3}' & m_{2,4}'\\
0 & 0 & 1 & 0 & m_{3,1}' & m_{3,2}' & m_{3,3}' & m_{1,4}'\\
0 & 0 & 0 & 1 & m_{4,1}' & m_{4,2}' & m_{4,3}' & m_{1,4}'
\end{bmatrix}
\end{split}\]
Then the matrix
\[\begin{split}
\begin{bmatrix}
m_{1,1}' & m_{1,2}' & m_{1,3}' & m_{1,4}'\\
m_{2,1}' & m_{2,2}' & m_{2,3}' & m_{2,4}'\\
m_{3,1}' & m_{3,2}' & m_{3,3}' & m_{1,4}'\\
m_{4,1}' & m_{4,2}' & m_{4,3}' & m_{1,4}'
\end{bmatrix}
\end{split}\]
is the inverse of \(M\) we are looking for.
Can you explain how to use LU decomposition to get the inverse of a matrix?
< 14.5 Solve Systems of Linear Equations in Python | Contents | 14.7 Summary and Problems >
