Shape Bias Diffusion Network for Removing Human Shadows in Dark Environments Based on a Diffusion Model

Abstract

When taking photos on city streets at night, shadows of ourselves or passersby often get captured, significantly affecting the quality of the photos. Current shadow removal techniques for images primarily focus on prominent shadows during the daytime, but the task of removing human shadows in nighttime images has been largely neglected. In this paper, we propose a shape bias network called Shape Bias Diffusion Network (ShapeBiasDiff) based on the diffusion model, to address this task due to the indistinct texture information between shadows and surrounding areas in nighttime images. To our knowledge, this is the first work aimed at removing human shadows while preserving other shadows in nighttime images. In our ShapeBiasDiff, we design an edge composite module to prepare for the subsequent addition of features to the diffusion model. Then, we provide dynamic feature guidance to optimize the feature guidance process. Additionally, we introduce a novel Capsule Mamba UNet (CMUNet) network as the embedding network for the diffusion model. Furthermore, we construct our own Nocturnal Human Shadow Removal Dataset (NHSR) for this task, with a hybrid stylization training method. Experiments validate that our ShapeBiasDiff significantly outperforms other networks in the task of removing human shadows from nighttime images.The code is available at the link below:https://github.com/mzh529/ShapeBiasDiff.

Keywords: Human shadow removal, Diffusion models, State space sequence models, Capsule networks, Stylized training, Computer Vision