Engineering Math - Matrix

 

 

 

Determinent

 

To be honest, I am not sure on how to describe the mathemtical definition of a determinant that can apply to any size of matrix. Even thought there is an algorithm (usually used in computer programming) that can calculate the determinant of a matrix with any size, in textbook you would see different method of determinant calculation is used depending on the size of the matrix. In this page, I mostly focus on the determinant of 2x2 and 3x3 matrix but the 'Practical meaning and Application of the determinant' section can be applied to any size of matrix (any size of square matrix).

Let's fix two facts before we start. A determinant exists only for a square matrix, so a 3x2 matrix has none. The result is a single number, and it tells you how much the matrix scales area or volume. Each section adds one step to that idea, from 2x2 to 3x3 and then to larger matrices.

Determinant of a 2x2 Matrix

The 2x2 case is the one to understand first. Its formula is short enough to remember. Its picture, the area of a parallelogram, carries over to every larger size.

As you may learned in High school math or linear algegra course, the definition of Determiant for 2 x 2 matrix as follows is very simple as shown below.

 

2x2 matrix A with entries a b c d

 

Determinant of the matrix is expressed in mathmatical symbol and can be calculated as follows :

 

Determinant of a 2x2 matrix equals ad minus bc

 

But this kind of definition would not help much about understanding the real meaning of determinants and the calculation process gets exponentially complicated as the size of the matrix gets larger. If the size of matrix is equal to or greater than 4 x 4, it would be almost impossible to calculate it by hand.

So I would focus more on explaining the practical meaning of the determinants rather than calculation process.

Practical meaning of Determinant of matrix of 2 x 2 is as follows. It is the area of the shape enclosed by the two vectors which are made out of each row of the matrix.

 

Parallelogram spanned by the row vectors a b and c d

The two row vectors span a parallelogram, and the determinant measures its area.

  • The shape is a parallelogram : the label in the picture says rectangle, but the two row vectors (a,b) and (c,d) are not perpendicular in general. The shaded shape is a parallelogram, and its area is |ad - bc|.
  • The sign carries the orientation : ad - bc is positive when the turn from (a,b) to (c,d) is counterclockwise, and negative when it is clockwise. The area itself is the absolute value.
  • A worked example : for the rows (3,1) and (1,2), the determinant is 3 x 2 - 1 x 1 = 5. So the parallelogram has an area of 5 square units.
  • Rows or columns give the same answer : the determinant of AT equals the determinant of A. So the columns (a,c) and (b,d) span a parallelogram of the same area.

 

In some cases as follows, the determinants gets 0.

 

Two cases where the row vectors lie on one line and the determinant is zero

When the two rows lie on one line, the parallelogram is flat and the determinant is 0.

  • Both pictures show dependent rows : on the left, (c,d) points the same way as (a,b). On the right, it points the opposite way. In both cases one row is a multiple of the other.
  • A worked example : the rows (1,2) and (2,4) give 1 x 4 - 2 x 2 = 0.
  • What a zero determinant means : the matrix squeezes the plane onto a line, so its effect cannot be undone. The last section of this page returns to this point.

Determinant of a 3x3 Matrix

The 3x3 case takes more work by hand, but it is still manageable. Two methods are common, and both produce the same six products. The first is a diagonal method. The second breaks the matrix into 2x2 determinants, and it is the one that extends to larger sizes.

Let's look into the case of 3 x 3 matrix as follows.

 

3x3 matrix A with entries a to i

 

Mathematical symbol for this 3x3 matrix can be represented as follows :

 

Determinant symbol of the 3x3 matrix A

 

How to calculate this 3x3 matrix determinant ? There are couple of ways you can find in the textbook. One of the most common way for the calcuation would be the one shown below. As you see here, rearrange the matrix as shown below (copy the first two rows and paste it at the bottom of the matrix). And group each elements as highlighted below. And then, multiply all the elements within a highlighted strap and put a negative sign for the red strap and put the positive sign for the blue strap.

 

Diagonal method for a 3x3 determinant with the first two rows copied below

Three products along the blue straps are added, and three products along the red straps are subtracted.

  • This is the rule of Sarrus : the name is worth knowing, because textbooks use it. The two copied rows only make the wrap-around diagonals easy to see.
  • It works only for 3x3 : the same diagonal trick gives a wrong answer for a 4x4 matrix. A 4x4 determinant has 24 products, and the diagonals give only 8.

 

Then combine all the results for each strap into a single equation. You will get the result as follows.

 

Expanded 3x3 determinant and its grouping by the first row

 

Here, you can represent this result in another form as shown below. This is another commonly used method for 3x3 matrix determinant calculation. The basic idea for this method is that split a 3x3 matrix into combination of multiple 2x2 matrix determinant. As you see, 2x2 determinant calculation is very simple and repeating the several 2x2 matrix determinants calculation you can compute the 3x3 determinant.

 

3x3 determinant written as three 2x2 determinants along the first row

Each entry of the first row multiplies the 2x2 determinant that remains after its row and column are deleted.

  • This is cofactor expansion : each 2x2 determinant is called a minor. The signs follow the checkerboard pattern +, -, + along the first row.
  • Any row or column works : the expansion can run along any row or any column, as long as the checkerboard signs are used. A row with zeros saves work, because a zero entry removes its whole term.
  • A worked example : take the matrix with rows (2,1,3), (0,4,1) and (5,2,0). The expansion gives 2(4 x 0 - 2 x 1) - 1(0 x 0 - 5 x 1) + 3(0 x 2 - 4 x 5). That is -4 + 5 - 60 = -59. The diagonal method gives 0 + 0 + 5 - 60 - 4 - 0, which is also -59.

 

 

The practical meaning of Determinant of matrix is the volume of the object defined by the three vectors which is made out of each row of the matrix.

 

Parallelepiped spanned by the three row vectors of a 3x3 matrix

The three row vectors span a parallelepiped, and the determinant measures its volume.

  • The shape is a parallelepiped : the picture draws a box, but in general the three edges are not perpendicular. The volume is the absolute value of the determinant.
  • The link to vector products : the determinant equals the scalar triple product. That is the dot product of (a,b,c) with the cross product of (d,e,f) and (g,h,i).
  • Zero volume means dependent rows : if the three rows lie in one plane, the box is flat and the determinant is 0.

 

If the size of matrix gets equal to/higher than 4 x 4, it would be difficult to visualize the definition geometrically as above, but I hope you can have your own image at least.

Determinant of a NxN Matrix

The two methods above do not scale in the same way. The diagonal method stops at 3x3. Cofactor expansion works for any size, but its cost grows very fast. So let's see what a computer does instead.

We have seen in previous section the method to calculate the determinant of 2x2 and 3x3 matrix ? Then, you may ask .. how can I calculate 4x4 matrix, 5x5 matrix etc ?

There is no simple and easy method to do this, but the most common way would be to convert the matrix into the combinations of mutiple 2x2 or 3x3 matrix equation and then apply the method of 2x2, 3x3 determinant calculation method.

Cofactor expansion is the general form of that idea. An NxN determinant expands into N determinants of size N-1, and each of those expands again. In the end the sum has N! products. That is 24 products for 4x4, 120 for 5x5 and 3,628,800 for 10x10. So the expansion is fine by hand for a 4x4 matrix, but no program uses it for a large one.

Instead, a computer uses row reduction. Adding a multiple of one row to another row does not change the determinant. So Gaussian elimination can turn the matrix into an upper triangular one with the same determinant. The determinant of a triangular matrix is the product of its diagonal entries, and each row swap on the way flips the sign. This takes about N3 operations instead of N! products. The det function of Matlab and numpy.linalg.det both work this way, through an LU decomposition.

Here is a 4x4 example. Take the matrix with rows (2,1,0,1), (1,3,1,0), (0,1,4,1) and (1,0,1,2). Elimination with no row swaps gives the pivots 2, 5/2, 18/5 and 1. Their product is 18, which is the determinant.

  • Triangular matrices are easy : the determinant is the product of the diagonal entries. Row reduction exists to reach that form.
  • Each row operation has a known effect : a row swap flips the sign, and scaling a row by k scales the determinant by k. Adding a multiple of one row to another changes nothing.
  • Determinants multiply : det(AB) = det(A) det(B). So a chain of transformations scales area or volume by the product of the individual factors.

Practical Meaning and Application of the Determinant

The determinant is a single number, so it cannot describe everything a matrix does. But it answers one question well: how much the matrix scales area or volume, and whether it collapses space.

Real importance of Determinant can be described as follows. You can understand basic characteristics of a matrix from determinant. When you apply a Matrix to vectors, you can make pretty reasonable guess just by looking at Determinant without doing all the calculation. If the process requires only one time calculation (e.g, one time multiplication of Matrix and vector) it would be no problem to perform the calculation, but if you have to do many times it would be handy to make a inference of the result from Determinant rather than doing all the 'matrix x vector' multiplication.

The diagram below sorts matrices by the value of the determinant. Read its labels with care, because the two middle branches are swapped.

 

What the value of the determinant says about a matrix

The size of the determinant is the factor by which the matrix scales area or volume.

  • The two middle branches are reversed in the diagram : a determinant larger than 1 in size magnifies, and one smaller than 1 shrinks. For example, the diagonal matrix with entries 2 and 3 turns a unit square into a 2 by 3 rectangle, and its determinant is 6.
  • The size matters, not the sign : the comparison with 1 applies to the absolute value. A negative determinant means the matrix also flips orientation, like a mirror. The swap matrix with rows (0,1) and (1,0) has determinant -1, and it keeps areas unchanged.
  • A determinant of 1 preserves area : a rotation matrix has determinant cos2θ + sin2θ = 1. It moves shapes without changing their size.
  • Zero collapses space into a lower dimension : a 2x2 matrix with determinant 0 maps the plane onto a line or onto a single point. The diagram says a dot, but only the zero matrix does that.
  • Zero also means no inverse : the inverse formula divides by the determinant, so it fails. The system Ax = b then has either no solution or infinitely many, never exactly one. The diagram says there is no solution, which is only one of the two cases.
  • The link to eigenvalues : the determinant equals the product of the eigenvalues. The page linked below shows both ideas graphically.

 

You will see some examples of using the determinants of Matrix in Geometric/Graphical Meaning of Eigenvalue and Determinant