Automated detection of highly aggregated neurons in microscopic images of macaque brain

Neuron detection is a key step in individualizing and counting neurons which are important for assessing physiological and pathophysiological information. A large number of methods including deep learning networks have been proposed but mainly targeting regions with few aggregated neurons. The objective of this paper is to address an automated neuron detection problem in heterogeneous hippocampus region with different degrees of neuron aggregation. Since deep learning networks require a lot of ground truths but neuron instance annotation is impossible in regions where numerous neurons are clustered, ground truth of centroids marked at the center of neurons is created for training. We propose a multiscale convolutional neural network (CNN) to regress neuron centroid mapping across image. Using multiscale information makes the proposed network applicable not only for single individual neurons, but also for a large number of aggregated neurons. Experimental results show that our method is superior to state-of-the-art deep learning-based algorithms. To our knowledge, this is the first deep learning study to detect neurons in regions of highly clustered neurons.