Abstract

Restoring clear frames from rainy videos presents a significant challenge dueto the rapid motion of rain streaks. Traditional frame-based visual sensors,which capture scene content synchronously, struggle to capture the fast-movingdetails of rain accurately. In recent years, neuromorphic sensors haveintroduced a new paradigm for dynamic scene perception, offering microsecondtemporal resolution and high dynamic range. However, existing multimodalmethods that fuse event streams with RGB images face difficulties in handlingthe complex spatiotemporal interference of raindrops in real scenes, primarilydue to hardware synchronization errors and computational redundancy. In thispaper, we propose a Color Spike Stream Deraining Network (SpikeDerain), capableof reconstructing spike streams of dynamic scenes and accurately removing rainstreaks. To address the challenges of data scarcity in real continuous rainfallscenes, we design a physically interpretable rain streak synthesis model thatgenerates parameterized continuous rain patterns based on arbitrary backgroundimages. Experimental results demonstrate that the network, trained with thissynthetic data, remains highly robust even under extreme rainfall conditions.These findings highlight the effectiveness and robustness of our method acrossvarying rainfall levels and datasets, setting new standards for video derainingtasks. The code will be released soon.