Machine Learning  

What is it ?

Before I try to define Machine Learning, I want to talk a little bit on a few confusing terminology which sounds almost same. Those are AI (Artificial Intelligence), Machine Learning, Deep Learning. Of course, it is not easy to draw clear border lines between these terms but overall the relationship among these terms can be illustrated as follows.

Just looking at the diagram, you would notice that AI is the broadest concep that includes both Machine Learning and Deep Learning. That is, Machine Learning is a special form of AI and Deep Learning is a special form of Machine Learning.

Now let's get a little bit deeper into the definition of each area. Whenever I try to learn any new concept, I would suggest you to go back to basic and try to find the basic definition from a dictionary. Of course, there are many cases where the definition from ordinary dictionary does not correctly match the practical definition in a specific technology, but in many case the definition in ordinary dictionary will certainly help.

I found the definitions from Cambridge Dictionary on each words in Artifical Intelligence, Machine Learning, Deep Learning and got the definition as follows. I think these are well defined even in technical aspects. (I also found the definition on Machine, Deep in the same dictionary but I don't think those are such well defined in the context of technology). I highlighted some keywords or key expressions in each definition which I think is most relevant to the context we are talking about and I changed a word in the definition of Learning so that it can be more realistic / practical in our context.

Intelligence : the ability to learn, understand, and make judgments or have opinions that are based on reason

Artificial : made by people, often as a copy of something natural

Learning : the activity of obtaining knowledge ==> the activity of obtaining intelligence

Now let's think of the definition of each area with more of technical aspects.

Artificial Inteligence

Just combining the basic defintions shown above and with a little bit of modification, we may define Artificial Intelligence as follows.

Artifical Intelligence is the ability (for a machine like computer) to learn and make judgements (for a specific set of data or queries given to the machine).

Then you may have questions like 'how a machine can obtain the intelligence ?' or  'how a machine can learn ?'

The answer is .. machine learns based on some set of rules or algorithms made by human.

What kind of rule or algorithm is that ? Several decades ago when there is no such a concept of machine learning or deep learning, the rules / algorithm used in AI were very specific and explicit. Simply put (with some risk of over simplication), you can think of AI in very early stage as a set of huge list of 'if' or 'case' statement. Each and every rule is explicitely defined by human (AI programmer). The most common example of Aritificial intellgence at this stage were Expert System.

Machine Learning

The fundamental definition of Machine Learning is same as the definition of AI that I described above since Machine Learning itself is a type of AI. Then what is the special for Machine Learning in comparison to conventional AI described above ? I think the main difference is about the algorithms by which a machine(e.g, Computer, Car etc) to learn. In conventional AI, the algorithm were very specific and explicit, but in machine learning those algorithm is less specific and explicit. Human (Machine Learning Programmer) provide a set of flexible algorithm with many tunable parameters and Machine tunes those parameters byifself from the huge data set given to it as a training data. That is, Machine Learning learns something from both algorithm and training data. It is like we learn something. We are given a powerful algorithm (brain) and (the algorithm = brain) learn from various training data (the experience).

In terms of Algorithm, there are largely two different approaches in Machine Learning. One is based on statistical methods and the other one is based on Neural Network.

With the statistical methods, the machine usually learns about how to categorize a given data set into a few groups and figure out (predict) which group a given data(the new data) belong to. In this method, we (human programmer) provide the machine only with training data and does not (or almost not) provide any additional information about the data. Clustering is a typical example for this type of algorithm.

With Neural Network method, the machine learns about how to categorize (or identify) a given data by a lot of example data. These example data is made up of a pair of information. One part of the pair is the data that the machine has to identify and the other part of the pair is the label (a category name) that the data belong to. The learning goes as follows.

Human (programmer) shows the picture of a banna(a data) to a machine (e.g, computer) and says 'this is a banna(label).

Human (programmer) shows the picture of an apple(a data) to a machine (e.g, computer) and says 'this is an apple(label).

Human (programmer) shows the picture of an orange(a data) to a machine (e.g, computer) and says 'this is an orange(label).

....

this process goes on with several thousands or millions of different types of data (e.g, various different pictures of same fruits, various different pictures of different fruits and various different pictures of same cars, different cars and so on).

After all of these process (training), human shows a new picture (usually not used in the training process) to the machine and ask what is this ? and then machine comes with the answer.

Deep Learning

According to the diagram shown at the beginning of this page, you would notice that Deep Learning is a type (branch) of Aritificial Intelligence in broad sense and in a little narrower sense it is a type (branch) of Machine Learning. In previous section, I mentioned Machine Learning is largely made up of two branches. One is based on statistical method and the other one is based on Neural Network. Deep Learning belongs to the branch of Machine Learning which is based on Neural Network. I would not go into the details of Neural Network here, which is another huge topic.

In terms of structure of a Neural Network, there are roughly two categories. One is the neural network which is made up of single layer and the other type is the one which is made up of multiple layers. The type of network with the multi layer is called Deep Neural Network, the machine learning that is based on Deep Neural Network is called Deep Learning.

What is the main difference in terms of learning capability between single layer neural network and multi layer neural network (Deep Neural Network) ?

It is hard to provide short/clear explanation for this, but just jumping into the conclusion Deep Network Network based algorithm (Deep Learning) can provide solution for much diverse problems comparing to the single layered neural network.