Combining deep learning with handcrafted features for cell nuclei segmentation

Segmentation of cell nuclei in fluorescence microscopy images provides valuable information about the shape and size of the nuclei, its chromatin texture and DNA content. It has many applications such as cell tracking, counting and classification. In this work, we extended our recently proposed approach for nuclei segmentation based on deep learning, by adding to its input handcrafted features. Our handcrafted features introduce additional domain knowledge that nuclei are expected to have an approximately round shape. For round shapes the gradient vector of points at the border point to the center. To convey this information, we compute a map of gradient convergence to be used by the CNN as a new channel, in addition to the fluorescence microscopy image. We applied our method to a dataset of microscopy images of cells stained with DAPI. Our results show that with this approach we are able to decrease the number of missdetections and, therefore, increase the F1-Score when compared to our previously proposed approach. Moreover, the results show that faster convergence is obtained when handcrafted features are combined with deep learning.