Engineering Math - Quick Reference                                 Home : www.sharetechnote.com

 

 

 

Vector - Inner Product/Dot Product

 

The geomatrc meaning of Inner Product is as follows. Inner Product is a kind of operation which gives you the idea of angle between the two vectors. Actually the most important application of inner product are

  • it gives the angle between the two vectors
  • it gives the degree of the correlation between two vectors

For more detailed descriptions of the application, see the last section (Applied meaning of Vector Inner Product) and read the whole pages if you are interested in how the vector inner product can indicate this kind of information.

 

  

There are several different ways of representing/calculating the inner product. Equation (1) gives you the geometric meaning of inner product. Equation (2) would not shows you any idea of visualization, but it gives you a way of calculating the inner product with very simple multiplication and sums(Equation (2) would be the most common ways to calculate the inner product in most of the application). Equation (3) is the form representing the inner product as a multiplication of two vectors.

 

 

What I explained above is a kind of geometric meaning (graphical meaning) of Inner product. This kind of application can be used in 2D (two element vector) and 3D (three element vector) which can be easily visualized, but it would be difficult to interpret the inner product in this way if the number of the element grow larger. In most of the applications other than computer graphics, the inner product of vectors is done in algebric procedure as shown below.

 

As a simple example, let's think of a case of taking the inner product of two vectors which is made up of 3 elements as shown below.

 

Before I ask you to take the inner product of two vectors, I would like to talk first about how to make a proper question of inner product.

First if I ask you to perform the following operation (inner product of two vectors in the following format), does it make sense to you ? Is this a valid question ?

 

 

The answer is NO. The question itself is not valid. In any matrix inner product, there is an important rule as shown below (Vector is also a kind of matrix, so vector inner product should follow this rule as well. 'm x n', 'a x b', 'm x b' represents the dimension of a vector or matrix. If you are not familiar with the concept of dimension, refer to vector/matrix dimension page)

 

 

Now let's check if the problem I gave you meets this requirement. As shown below, the problem that I gave you does not meet the requirement, so the problem itself is invalid.

 

 

What if I change the problem as below ? (In this problem, I transposed the v2). Does this make sense ?

 

 

Yes, it make sense. It is valid problem since it meets the matrix inner product rule as shown below. One think you have to notice is that the result of this form of inner product is 1 x 1 vector. 1 x 1 vector (matrix) is same as scalar.

 

 

Now let's look how this inner product is calculated. The calculation is as simple as follows. You may have a very long calculation if the size of the vector is large. But it is just long, but it is very simple calculation. Just multiply each elements of two vectors and sum them all.

 

 

Now let's think of another example of vector porduct. This may look almost same as the previous example, but it is not exactly same. The difference is that v1 is transposed as opposed to v2 in previous example.

 

 

Before you start calculation, let's check if this problem is meaning (valid) question or not. It is valid question because it meets the inner produt rule as shown below. One important thing you need to notice in this case is that the result of the inner product in this case is 3 x 3 matrix, it is not a scalar value.

 

 

The result of calculation is as follows. I recommend you to take some time to take a close look at this.

 

 

Now let's try to increase the size of a vector and see if you can figure out the result from the simple example as explained above.

 

Let's suppose you are give two vectors as shown below.

 

And you are asked to take the inner product of the two vectors as shown below.

 

First, check if this problem itself is valid or not. (I will leave this to you. Try on your own if this problem is valid or not). Just to give you the conclusion. the answer is 'Yes, the problem is valid'.

 

What is the result of the inner product in this case ? It is going to be as follows.

 

 

If the size of vector gets very large, it would be too tedious to write the result as above. If you take a little bit closer examination of the expression, you would notice that you can simplify the expressing using 'Sum (Sigma)' notation as shown below.

 

 

 

Applied meaning of Vector Inner Product

 

When you see the case of vector inner product in real application, it is very important of the practical meaning of the vector inner product.

 

I see two major application of the inner product. One is to figure out the angle between the two vectors as illustrated above. (First, you calculate the inner product using Equation (2) and with the result and equation (1), you can figure out the angle).

This angle in turn has additional implication.  If this angle is 90 degree(0.5 pi), it means

  • the result of the inner product becomes 0
  • the correlation between the two vector is 0.
  • the two vectors are independent to each other.

If this angle is 0 degree, it means

  • the result of the inner product becomes max (multiplication of length of two vectors)
  • the correlation between the two vector is 1.
  • the two vectors are totally dependent to each other.

If this angle is between 0 and 90, the value is between min(0) and max value and they are partially dependent to each other.

 

Second application is nothing to do with geometric meaning, but it is very handy form of mathematical representation of "Sum of Multiplication/Sume of Times" as shown in "Sum of Times" page.

 

One important thing you have to remember is that the result of inner product of two vectors is a scalar.