Bayesian Incremental Learning for Deep Neural Networks.

In industrial machine learning pipelines, data often arrive in parts.Particularly in the case of deep neural networks, it may be too expensive totrain the model from scratch each time, so one would rather use a previouslylearned model and the new data to improve performance. However, deep neuralnetworks are prone to getting stuck in a suboptimal solution when trained ononly new data as compared to the full dataset. Our work focuses on a continuouslearning setup where the task is always the same and new parts of data arrivesequentially. We apply a Bayesian approach to update the posteriorapproximation with each new piece of data and find this method to outperformthe traditional approach in our experiments.