Light-transport Simulation and Machine Learning – LUCE

All optical numerical simulations based on stochastic methods are subject to intrinsic noise due to mathematical variance. Currently, denoising techniques generate artefacts and loss of detail. For predictive and reliable simulations essential for research and industry, all physical laws must be respected (wavelength dependence and polarization of light). Using data from "Iso-Photographic" simulations representing reality and based on learning methods and artificial intelligence, we propose to implement a new sampling and denoising method integrated in the heart of our resolution algorithm (OMEN). An HPC implementation on GPU, will allow real-time simulations respecting radiometric values and photometric quality for an ultimate exploitation in Virtual/Augmented Reality. Our goals are to use machine learning (ML) to significantly reduce latencies between each rendering step while maintaining accurate simulations (bias-free). This hybridization idea aims to show that rendering simulations provides both a starting point for the ML algorithm, and training data for the neural networks that will be in charge of refining the output of the model.
The application of ML to rendering problems is not a novelty by itself. ML-based denoisers for instance have been shown to outperform traditional denoisers. Right now, the denoiser is executed as a post-processing filter on images obtained by averaging colour samples. This often leads to artefacts such as blurring and significant loss of detail. In this regard, our proposals to enhance OMEN renderer with ML aim to (i) integrate the rendering engine - with its spectral and polarized components - more closely with the denoiser so that samples can be sent it directly and, in feedback, allow the denoiser to better guide the placement of other samples thereby making it responsible for the final image reconstruction and adaptive sampling, (ii) enable new mechanisms – like importance sampling approaches – to extract useful information from various estimates and guide the path construction in the form of providing well-placed samples, and (iii) investigate new load balancing strategies to efficiently parallelize light transport simulations on HPC environment.