Abstract: In recent years, deep learning methods have achieved impressive results with higher peak signal-to-noise ratio in single image super-resolution (SISR) tasks. However, these methods are usually computationally expensive, which constrains their application in mobile scenarios. In addition, most of the existing methods rarely take full advantage of the intermediate features which are helpful for restoration. To address these issues, we propose a moderate-size SISR network named matrix channel attention network (MCAN) by constructing a matrix ensemble of multi-connected channel attention blocks (MCAB). Several models of different sizes are released to meet various practical requirements. Extensive benchmark experiments show that the proposed models achieve better performance with much fewer multiply-adds and parameters.