Neural Networks 201: All About Autoencoders

(This article originally appeared at KDNuggets.com here. For more, visit https://www.kdnuggets.com/)

Autoencoders can be a very powerful tool for leveraging unlabeled data to solve a variety of problems, such as learning a "feature extractor" that helps build powerful classifiers, finding anomalies, or doing a Missing Value Imputation.

For those getting started with neural networks, autoencoders can look and sound intimidating.  But in fact, they are a conceptually simple and elegant approach that will open many doors to an ML practitioner.  They can be used for anomaly detection and missing value imputation or help in building better classifiers or clusters.  In any case, what makes them unique is that they provide you with a mechanism for leveraging your unlabelled data, which often is much easier to get than labeled data.  For instance, it's a lot easier to get a collection of images than it is to get a collection of images where each one is labeled to tell you what's in it.

First and foremost, autoencoders are trained via unsupervised learning, which means you don't need labels.  An autoencoder is learned to predict its own input by using a noisy version of itself, which forces it to take advantage of structure in the data to learn compact ways of representing it.  At a high level, this means learning to throw away the noisy details and only keeping the important stuff.  Once you have a network that can condense data down to a compact form and discard details, that opens up a lot of new doors.

To get a better understanding of how autoencoders work and how you can put them to use, I've made a short video that explains the key concepts.

Bio: Zak Jost (@ZakJost) is Machine Learning Research Scientists at Amazon Web Services working on fraud applications. Before this, Zak built large-scale modeling tools as a Principal Data Scientist at Capital One to support the business's portfolio risk assessment efforts following a previous career as a Material Scientist in the semiconductor industry building thin-film nanomaterials.