Recurrent Neural Networks
Whereas CNNs are well-suited for working with image data, recurrent neural networks (RNNs) are a strong choice for building up sequential representations of data over time: tasks such as document translation and voice recognition.
Just as you can’t detect a cat looking at a single pixel, you can’t recognize text or speech looking at a single letter or syllable. To correctly perform translation and speech recognition, you need to understand not only the current letter or syllable, but also the previous data that came before it in time.
RNNs are capable of “remembering” the network’s past outputs and using these results as inputs to later computations. By including loops as part of the network model, information from previous steps can persist over time, helping the network make smarter decisions.
When to Use Neural Networks
Often referred to under the trendy name of “deep learning,” neural networks are currently in vogue. This is thanks to two main reasons:
- The proliferation of “big data” makes it easier than ever for machine learning professionals to find the input data they need to train a neural network.
- GPUs (graphics processing units) are computer processors that are optimized for performing similar calculations in parallel. Advances in GPU technology have enabled machine learning researchers to vastly expand the size of their neural networks, train them faster, and get better results.
Neural networks are best for situations where the data is “high-dimensional.” For example, a medium-size image file may have 1024 x 768 pixels. Each pixel contains 3 values for the intensity of red, green, and blue at that point in the image. All told, this is 1024 x 768 x 3 = 2,359,296 values. Each one of these values is a separate dimension and a separate input to a neuron at the start of the network.
Of course, while neural networks are an important part of machine learning theory and practice, they’re not all that there is to offer. Based on the structure of the input data, it’s usually fairly clear whether using a neural network, or another machine learning technique, is the right choice.
For example, one machine learning model that’s entirely separate from neural networks is the decision tree. Let’s say that you run a real estate website and you want to predict the value of a house based on certain information.
In a decision tree, calculating a final result begins at the top of the tree and proceeds downwards:
- At the top node of the tree, you examine a single feature of the data, such as the number of bedrooms in the house. Based on the value of this feature, the computation splits off into two or more children nodes, similar to a “choose your own adventure” book. For example, there might be one node for houses with 1 or 2 bedrooms, and another node for houses with more than 2 bedrooms.
- At the next level of the tree, the computation splits again based on a different feature of the data, such as the house’s ZIP code, its square footage, or the level of crime in the area.
- The computation ends when you reach a terminal node at the bottom of the tree. This node should have an associated value that estimates the house’s price.
Decision trees often require human input via feature selection and engineering in order to reach optimal performance. On the other hand, neural networks are capable of handling extremely large numbers of dimensions and quickly condensing them into the most important features.
Deciding when to use neural networks for your machine learning problem is all about learning from experience and exercising your best judgment.