Deep learning is impacting everything from healthcare to transportation to manufacturing, and more. Companies are turning to deep learning to solve hard problems, like speech recognition, object recognition, and machine translation.
One of the most impressive achievements this year was AlphaGo beating the best Go player in the world. With the victory, Go joins checkers, chess, othello, and Jeopardy as games machines have defeated human at.
While beating someone at a board game might not seem useful on the surface, this is a huge deal. Before the victory, Go was written off as a candidate for a competent AI. Due in part to the amount of human intuition necessary to play the game. The victory makes an entire class of problems once considered intractable ripe for solving.
While it might seem like this technology is still years away, we are beginning to see commercial use. Such is the case with self-driving cars. Companies like Google, Tesla, and Uber are already testing autonomous cars on the streets.
Ford predicts self-driving cars will see widespread use within the next five years. The U.S. government has gotten on board by issuing safety guidelines.
What Is Deep Learning?
To understand what deep learning is, we first need to understand the relationship deep learning has with machine learning, neural networks, and artificial intelligence.
The best way to think of this relationship is to visualize them as concentric circles:
At the outer most ring you have artificial intelligence (using computers to reason). One layer inside of that is machine learning. With artificial neural networks and deep learning at the center.
Broadly speaking, deep learning is a more approachable name for an artificial neural network. The “deep” in deep learning refers to the depth of the network. An artificial neural network can be very shallow.
Neural networks are inspired by the structure of the cerebral cortex. At the basic level is the perceptron, the mathematical representation of a biological neuron. Like in the cerebral cortex, there can be several layers of interconnected perceptrons.
The first layer is the input layer. Each node in this layer takes an input, and then passes its output as the input to each node in the next layer. There are generally no connections between nodes in the same layer and the last layer produces the outputs.
We call the middle part the hidden layer. These neurons have no connection to the outside (e.g. input or output) and are only activated by nodes in the previous layer.
Think of deep learning as the technique for learning in neural networks that utilizes multiple layers of abstraction to solve pattern recognition problems. In the 1980s, most neural networks were a single layer due to the cost of computation and availability of data.
Machine learning is considered a branch or approach of Artificial intelligence, whereas deep learning is a specialized type of machine learning.
Machine learning involves computer intelligence that doesn’t know the answers up front. Instead, the program will run against training data, verify the success of its attempts, and modify its approach accordingly. Machine learning typical requires a sophisticated education, spanning software engineering and computer science to statistical methods and linear algebra.
There are two broad classes of machine learning methods:
Supervised learning
Unsupervised learning
In supervised learning, a machine learning algorithm uses a labeled dataset to infer the desired outcome. This takes a lot of data and time, since the data needs to be labeled by hand. Supervised learning is great for classification and regression problems.
For example, let’s say that we were running a company and want to determine the effect of bonuses on employee retention. If we had historical data – i.e. employee bonus amount and tenure – we could use supervised machine learning.
With unsupervised learning, there aren’t any predefined or corresponding answers. The goal is to figure out the hidden patterns in the data. It’s usually used for clustering and associative tasks, like grouping customers by behavior. Amazon’s “customers who also bought…” recommendations are a type of associative task.
While supervised learning can be useful, we often have to resort to unsupervised learning. Deep learning has proven to be an effective unsupervised learning technique.
Why is Deep Learning Important?
Computers have long had techniques for recognizing features inside of images. The results weren’t always great. Computer vision has been a main beneficiary of deep learning. Computer vision using deep learning now rivals humans on many image recognition tasks.
Facebook has had great success with identifying faces in photographs by using deep learning. It’s not just a marginal improvement, but a game changer: “Asked whether two unfamiliar photos of faces show the same person, a human being will get it right 97.53 percent of the time. New software developed by researchers at Facebook can score 97.25 percent on the same challenge, regardless of variations in lighting or whether the person in the picture is directly facing the camera.”
Speech recognition is a another area that’s felt deep learning’s impact. Spoken languages are so vast and ambiguous. Baidu – one of the leading search engines of China – has developed a voice recognition system that is faster and more accurate than humans at producing text on a mobile phone. In both English and Mandarin.
What is particularly fascinating, is that generalizing the two languages didn’t require much additional design effort: “Historically, people viewed Chinese and English as two vastly different languages, and so there was a need to design very different features,” Andrew Ng says, chief scientist at Baidu. “The learning algorithms are now so general that you can just learn.”
Google is now using deep learning to manage the energy at the company’s data centers. They’ve cut their energy needs for cooling by 40%. That translates to about a 15% improvement in power usage efficiency for the company and hundreds of millions of dollars in savings.
Deep Learning Microservices
Here’s a quick overview of some deep learning use cases and microservices.
Illustration Tagger. An implementation of Illustration2Vec, this microservice can tag an image with the safe, questionable, or explicit rating, the copyright, and general category tag to understand what’s in the image. DeepFilter is a style transfer service for applying artistic filters to images.
The age classifier uses face detection to determine the age of a person in a photo. The Places 365 Classifier uses a pre-trained CNN and based on Places: An Image Database for Deep Scene Understanding B. Zhou, et al., 2016 to identify particular locations in images, such as a courtyard, drugstore, hotel room, glacier, mountain, etc. Lastly, there is InceptionNet, a direct implementation of Google’s InceptionNet using TensorFlow. It takes an image (such as a car), and returns the top 5 classes the model predicts are relevant to the image.
Open Source Deep Learning Frameworks
Deep learnings is made accessible by a number of open source projects. Some of the most popular technologies include, but are not limited to, Deeplearning4j (DL4j), Theano, Torch, TensorFlow, and Caffe. The deciding factors on which one to use are the tech stack they target, and if they are low-level, academic, or application focused. Here’s an overview of each:
DL4J:
JVM-based
Distrubted
Integrates with Hadoop and Spark
Theano:
Very popular in Academia
Fairly low level
Interfaced with via Python and Numpy
Torch:
Lua based
In house versions used by Facebook and Twitter
Contains pretrained models
TensorFlow:
Google written successor to Theano
Interfaced with via Python and Numpy
Highly parallel
Can be somewhat slow for certain problem sets
Caffe:
Not general purpose. Focuses on machine-vision problems
Implemented in C++ and is very fast
Not easily extensible
Has a Python interface
Deep Learning Online Courses
Google and Udacity have collaborated on a free online deep learning course, part of Udacity’s Machine Learning Engineer Nanodegree. This program is geared towards experienced software developers, who want to develop a specialty in machine learning, and some of its subspecialties like machine learning.
Another option is the very popular Andrew Ng course on machine learning, hosted by Coursera and Stanford.
Deep Learning Books
While many deep learning courses require a rigorous educational background to get started, this isn’t the case for the book Grokking Deep Learning. In their own words “If you passed high school math and can hack around in Python, I want to teach you Deep Learning.”
Another popular book is the appropriately named Deep Learning Book. This is a great resource for bottom up as it covers of all the required math for deep learning.
Deep Learning Simplified is a great video series on youtube, that breaks down deep learning into everyday words and concepts. Watch the first video here:
The post An Introduction to Deep Learning appeared first on Algorithmia.