^2 CO: Deep Depth Colorization.

The ability to classify objects is fundamental for robots. Besides knowledgeabout their visual appearance, captured by the RGB channel, robots heavily needalso depth information to make sense of the world. While the use of deepnetworks on RGB robot images has benefited from the plethora of resultsobtained on databases like ImageNet, using convnets on depth images requiresmapping them into three dimensional channels. This transfer learning proceduremakes them processable by pre-trained deep architectures. Current mappings arebased on heuristic assumptions over preprocessing steps and on what depthproperties should be most preserved, resulting often in cumbersome datavisualizations, and in sub-optimal performance in terms of generality andrecognition results. Here we take an alternative route and we attempt insteadto learn an optimal colorization mapping for any given pre-trainedarchitecture, using as training data a reference RGB-D database. We propose adeep network architecture, exploiting the residual paradigm, that learns how tomap depth data to three channel images. A qualitative analysis of the imagesobtained with this approach clearly indicates that learning the optimal mappingpreserves the richness of depth information better than current hand-craftedapproaches. Experiments on the Washington, JHUIT-50 and BigBIRD publicbenchmark databases, using CaffeNet, VGG16, GoogleNet, and ResNet50 clearlyshowcase the power of our approach, with gains in performance of up to 16%compared to state of the art competitors on the depth channel only, leading totop performances when dealing with RGB-D data