Convolutional Neural Networks
Convolutional neural networks (CNNs) are frequently used for the tasks of image recognition and classification.
For example, suppose that you have a set of photographs and you want to determine whether a cat is present in each image. CNNs process images from the ground up. Neurons that are located earlier in the network are responsible for examining small windows of pixels and detecting simple, small features such as edges and corners. These outputs are then fed into neurons in the intermediate layers, which look for larger features such as whiskers, noses, and ears. This second set of outputs is used to make a final judgment about whether the image contains a cat.
CNNs are so revolutionary because they take the task of localized feature extraction out of the hands of human beings. Prior to using CNNs, researchers would often have to manually decide which characteristics of the image were most important for detecting a cat. However, neural networks can build up these feature representations automatically, determining for themselves which parts of the image are the most meaningful.
Recurrent Neural Networks
Whereas CNNs are well-suited for working with image data, recurrent neural networks (RNNs) are a strong choice for building up sequential representations of data over time: tasks such as handwriting recognition and voice recognition.
Just as you can’t detect a cat looking at a single pixel, you can’t recognize text or speech looking at a single letter or syllable. To comprehend natural language, you need to understand not only the current letter or syllable but also its context.
RNNs are capable of “remembering” the network’s past outputs and using these results as inputs to later computations. By including loops as part of the network model, information from previous steps can persist over time, helping the network make smarter decisions. Long-short term memory (LSTM) units or gated recurrent units (GRUs) can be added to an RNN to allow it to remember important details and forget the irrelevant ones.
A Historical Context
All of this may feel contemporary, but it goes back decades. Big data consultant Bernard Marr and Import.io, among many others, have compiled the history of neural networks and deep learning. Here’s a short overview:
- In 1943, Walter Pitts, a logician, and Warren McCulloch, a neuroscientist, created the model of a neural network. They proposed a combination of mathematics and algorithms that aimed to mirror human thought processes. The model — dubbed the McCulloch-Pitts neurons — has, of course, evolved, but it remains standard.
- In 1980, Kunihiko Fukushima proposed the Neoconitron, a hierarchical, multilayered, artificial neural network used for handwriting recognition and other pattern-recognition tasks.
- In the mid-2000s, the term “deep learning” gained traction after papers by Geoffrey Hinton, Ruslan Salakhutdinov, and others showed how neural networks could be pre-trained a layer at a time.
- In 2015, Facebook implemented DeepFace to automatically tag and identify users in photographs.
- In 2016, Google’s AlphaGo program beat a top-ranked international Go player.
Games, especially strategy games, have been a great way to test and/or demonstrate the power of neural networks. But there are numerous, more practical, applications.
“For companies looking to predict user patterns or how investments will grow, the ability to mobilize artificial intelligence can save labor and protect investments. For consumers trying to understand the world around them, AI can reveal patterns of human behavior and help to restructure our choices,” Terence Mills, CEO of AI.io and Moonshot, writes in a piece for Forbes.
Artificial Intelligence applications are wide-ranging. Some of the most exciting are in healthcare:
- Research from 2016 suggests that systems with neural network machine learning may aid decision-making in orthodontic treatment planning.
- A research paper published in 2012 discusses brain-tumor diagnosis systems based on neural networks.
- PAPNET, developed in the 1990s, is still being used to identify cervical cancer. From the 1994 research paper: “The PAPNET Cytological Screening System uses neural networks to automatically analyze conventional smears by locating and recognizing potentially abnormal cells.”
Many interesting business applications use deep neural networks for image classification: A neural network can identify and discern relationships among images, placing each in an appropriate category. The business implications are tremendous, Mills says in his Forbes column. “You could have software crawl through a given network looking for instances of your products. You could figure out if a celebrity has been wearing a piece of jewelry that you created. You could even see if your cafe has been featured in any shots of a show made in your neighborhood.”
If you’ve watched a few too many dystopian science fiction movies, you may be starting to worry. You probably don’t need to. As Marr explains,
“The promise of deep learning is not that computers will start to think like humans. That’s a bit like asking an apple to become an orange. Rather, it demonstrates that given a large enough data set, fast enough processors, and a sophisticated enough algorithm, computers can begin to accomplish tasks that used to be completely left in the realm of human perception — like recognizing cat videos on the web (and other, perhaps more useful purposes).”
If you want to learn more about the theory behind neural networks, this series of videos provides a fantastic overview. Coursera also offers a series of lectures via YouTube, and a simple Google search will yield dozens of other videos and courses.
As for us? We’re excited by the possibilities — and relieved that HAL isn’t one of them.