Machine Learning Home : www.sharetechnote.com 

Evolution of Algorithm / Challenges and Solution
When I started studying the neural network/Deep Learning, most of the things seemed relatively clear and the simple fully connected feedforward network seemed to work for all the problems. But as I learn more, I realized that it is not that simple as expected and started seeing so many different algorithmic tweaking (e.g, selection of transfer function, selection of optimization functions etc) and so many different structure of the neural networks(e.g, fully connected network, CNN, RNN etc). Then this question came to my mind. Why we need to go through so many tweakings and selections ? It turns out that there is no single 'fit for all' algorithm. An algorithm works for a certain situation but does not work well for other situation. To make the neural network work better for more diverse situation, many different tricks has been invented. As the result of such inventions for a long period of time, we now see a neural networks with such a many options to choose from. In this pages, I will summarize some of important tricks that was invented to handle some of the early problems of the neural network.
Reference
[1] Deep Learning for Wireless Physical Layer : Opportunities and Challenges
