Matrix multiplication is the operation behind almost everything else on these pages. A transformation of a shape, a system of linear equations and a MIMO channel all use it. Unlike addition, it is not done entry by entry. I'll start with the product of two 2 x 2 matrices and trace where each entry comes from. Then we'll describe the same product with inner products, and finish with the question of order.

• How are two 2 x 2 matrices multiplied ?
• Where does each entry of the result come from ?
• How does the inner product describe matrix multiplication ?
• Does the order of the two matrices matter ?

How are two 2 x 2 matrices multiplied ?

A 2 x 2 example is small enough to write out in full and large enough to show the pattern. Let's look at the result first, and then take it apart in the next section.

How do we do multiply two matrix ? For simplicity, let's think of the multiplication of two 2x2 matrix as shown below.

Figure 1. The question. The four entries of the product are still unknown.

Just jumping to the conclusion. The result is as shown below.

Figure 2. The result. Each entry of the product is a sum of two products.

Let's put numbers into Figure 2. Take A = [[1, 2], [3, 4]] and B = [[5, 6], [7, 8]], where each inner bracket is one row. The top left entry is 1 x 5 + 2 x 7 = 19. The top right entry is 1 x 6 + 2 x 8 = 22. The bottom row gives 3 x 5 + 4 x 7 = 43 and 3 x 6 + 4 x 8 = 50. So AB = [[19, 22], [43, 50]].

• The dot in the figures means the matrix product : it is not the element-wise product. For the same A and B, the element-wise product is [[5, 12], [21, 32]], which is a different matrix.
• Each entry needs two multiplications and one addition : so the whole 2 x 2 product needs 8 multiplications and 4 additions.
• No entry of AB comes from a single entry of A : every entry of the product uses a full row of A and a full column of B.

Where does each entry of the result come from ?

The four formulas in Figure 2 are easier to remember as one pattern than as four separate lines. The pattern pairs one row of the left matrix with one column of the right matrix. The position of that pair decides where the result lands.

How did we get this result ? What is the meaning of each elements in the resulting matrix ? Following shows you how each of the elements in resulting matrix are calculated.

Figure 3. Row i of the first matrix and column j of the second matrix give the entry in row i and column j of the result.

• Top panel : row 1 of the first matrix, in red, and column 1 of the second matrix, in blue, give the entry in row 1, column 1, in green.
• Second panel : row 1 and column 2 give the entry in row 1, column 2.
• Third panel : row 2 and column 1 give the entry in row 2, column 1.
• Bottom panel : row 2 and column 2 give the entry in row 2, column 2.

If it is not clear to you yet on how each of the elements in the resulting matrix came, take a look at the following arrows. I hope it got clearer to you.

Figure 4. The first entry traced. The first element of the row multiplies the first element of the column, and the second multiplies the second.

The same pattern works for any size. Let A be m x n and B be n x p, and let C = AB. The entry in row i and column j of C is c_ij = a_i1b_1j + a_i2b_2j + ... + a_inb_nj. The middle index runs along row i of A and down column j of B at the same time. So row i of A and column j of B must have the same length n. That is the size rule on the Dimension page.

• The inner index is shared : in c_ij = Σ_k a_ikb_kj, the index k is the column index of A and the row index of B.
• The result has the outer size : an m x n matrix times an n x p matrix gives an m x p matrix. For two 2 x 2 matrices, the result is 2 x 2.
• The cost grows with the cube of the size : two n x n matrices need n³ multiplications and n²(n - 1) additions. For n = 2 that gives the 8 and 4 counted above.

How does the inner product describe matrix multiplication ?

The row-times-column pattern has a name. Each entry is the inner product of two vectors, and writing the product that way makes the pattern easy to apply at any size.

If you are familiar with the concept of the inner product of two vectors, each element of the resulting matrix can be described by the inner product of a row vector in the first matrix and the column vector of the second matrix as shown below.

Figure 5. The product as a table of inner products. The entry in row i and column j is the inner product of r_i and c_j.

• Row vectors : r₁ = [a₁₁ a₁₂] and r₂ = [a₂₁ a₂₂] are the rows of the first matrix.
• Column vectors : c₁ and c₂ are the columns of the second matrix.
• Inner products : the four entries of the result are the four combinations of one row vector and one column vector.

The same product can also be read in two other ways. Both are useful when the matrices come from real problems. The first is the column view. A matrix times a vector is a combination of the columns of the matrix, weighted by the entries of the vector. For A = [[1, 2], [3, 4]] and x = [2, -1]^T, A x = 2 x [1, 3]^T - 1 x [2, 4]^T = [0, 2]^T. Each column of AB is A times the matching column of B, so it is also a combination of the columns of A.

The second is the outer product view. Column k of A times row k of B is a full matrix, and AB is the sum of these matrices over k. For the example, [1, 3]^T[5 6] = [[5, 6], [15, 18]] and [2, 4]^T[7 8] = [[14, 16], [28, 32]]. Their sum is [[19, 22], [43, 50]], the same AB as before.

• Row view : each entry of AB is an inner product of a row of A and a column of B. This is the view in Figure 5.
• Column view : each column of AB is a combination of the columns of A. This view explains why A x lies in the space spanned by the columns of A.
• Outer product view : AB is a sum of n rank-one matrices, one for each value of the shared index. This view is the basis of low-rank approximations.

Does the order of the two matrices matter ?

For numbers, 3 x 5 equals 5 x 3. For matrices, the order usually changes the result. Sometimes the order even decides whether the product exists at all, so you should never swap two factors without checking.

Take the same A and B as before, and multiply them in the other order. BA = [[5 x 1 + 6 x 3, 5 x 2 + 6 x 4], [7 x 1 + 8 x 3, 7 x 2 + 8 x 4]] = [[23, 34], [31, 46]]. This is not [[19, 22], [43, 50]]. So AB and BA are different matrices, even though both exist and both are 2 x 2. For matrices of different shapes the difference is larger. A 2 x 3 matrix times a 3 x 2 matrix is 2 x 2, while the reverse product is 3 x 3.

Some pairs do commute. The identity matrix commutes with every square matrix of its size. Two diagonal matrices commute, and so do two rotations in the same plane. Two other rules always hold, and they are the ones you use most. The grouping can change, so (AB)C = A(BC). And the transpose of a product reverses the order, so (AB)^T = B^TA^T.

• Matrix multiplication is not commutative : in general AB and BA differ, as the example shows. The order of the factors is part of the meaning.
• Grouping changes the cost, not the result : for n x n matrices A and B and an n x 1 vector x, A(Bx) needs 2n² multiplications, while (AB)x needs n³ + n².
• The rightmost factor acts first : in ABx, the vector x is multiplied by B first and by A second. This matters when a chain of matrices describes a chain of transformations, as on the Affine Mapping page.
• Use the right operator in code : in Matlab, A*B is the matrix product and A.*B is the element-wise product. In numpy, A @ B is the matrix product and A * B is element-wise.