Abstract: Analyzing and elucidating the attributes of cells and tissues with an observed microscopy image is a fundamental task in both biological research and clinical practice, and automation of this task to develop computer aided system based on image processing and machine learning technique has been rapidly evolved for providing quantitative evaluation and mitigating burden and time of the biological experts. Automated cell/nuclei detection and segmentation is in general a critical step in automatic system, and is quite challenging due to the existed heterogeneous characteristics of cancer cell such as large variability in size, shape, appearance, and texture of the different cells. This study proposes a novel method for simultaneous detection and segmentation of cells based on the Mask R-CNN, which conducts multiple end-to-end learning tasks by minimizing multi task losses for generic object detection and segmentation. The conventional Mask R-CNN employs cross entropy loss for evaluating the object detection fidelity, and equally treats all training samples in learning procedure regardless to the properties of the objects such as easily or hard degree for detection, which may lead to miss-detection of hard samples. To boost the detection performance of hard samples, this work integrates the focal loss for formulating detection criteria into Mask R-CNN, and investigate a feasible method for balancing the contribution of multiple task losses in network training procedure. Experiments on the benchmark dataset: DSB2018 manifest that our proposed method achieves the promising performance on both cell detection and segmentation.