Abstract: Lossless image compression is an important technique for im-age storage and transmission when information loss is not allowed. Withthe fast development of deep learning techniques, deep neural networkshave been used in this field to achieve a higher compression rate. Meth-ods based on pixel-wise autoregressive statistical models have showngood performance. However, the sequential processing way prevents thesemethods to be used in practice. Recently, multi-scale autoregressive mod-els have been proposed to address this limitation. Multi-scale approachescan use parallel computing systems efficiently and build practical sys-tems. Nevertheless, these approaches sacrifice compression performancein exchange for speed. In this paper, we propose a multi-scale progressivestatistical model that takes advantage of the pixel-wise approach and themulti-scale approach. We developed a flexible mechanism where the pro-cessing order of the pixels can be adjusted easily. Our proposed methodoutperforms the state-of-the-art lossless image compression methods ontwo large benchmark datasets by a significant margin without degradingthe inference speed dramatically.