Miper-MVS: Multi-Scale Itearative Probability Estimation with Refinement for Efficient Multi-View Stereo

Abstract

Multi-view stereo reconstruction aims at constructing 3D scenes from multiple 2D images. In recent years, learning-based multi-view stereo methods have achieved impressive results in depth estimation for multi-view stereo reconstruction. However, the current mainstream multi-stage processing method cannot solve the low-efficiency problem satisfactorily due to the use of 3D convolution and still brings a large amount of calculation. To further balance efficiency and generalization performance, we propose a multi-scale iterative probability estimation with refinement, which is a highly efficient method for multi-view stereo reconstruction. It consists of three main modules: 1) a high precision probability estimator, dilated-LSTM, encodes the pixel probability distribution of depth in the hidden state; 2) an efficient interactive multi-scale update module, fully integrates multi-scale information and improves parallelism by interacting information between adjacent scales; 3) a Pi-error Refinement module converts the depth error between views into a grayscale error map and refines the edges of objects in the depth map. At the same time, we introduce a large amount of high-frequency information to ensure the accuracy of the refined edges. Among the most efficient methods (e.g., runtime and memory), our method has achieved the best generalization on Tanks & Temples benchmark. In addition, the performance of the Miper-MVS also has highly competitive in DTU benchmark. Our code is available at https://github.com/zhz120/Miper-MVS.

Keywords: Multi-View Stereo, 3D Reconstruction, Depth Estimation, Stereo Vision