Spectral Polarization Imaging Applied to Surface Inspection – SPIASI

First, we proposed a study of the correlations between the different spectropolarimetric channels of images from existing databases. A statistical analysis was performed on a relatively large number of observations (50 scenes, ˜ 1.8 GB of data), for different reflection modes and different types of materials. The image preparation method is described in the figure below.
To delve deeper into the observation, it was necessary to perform the analysis on classified data. We chose to create categories by material type and reflection type (plastic, metal, glass, natural, etc.), and to measure the correlation between all bands. A processing chain was created for the analysis of correlations on known spectropolarimetric databases. It should be noted that the proposed analysis method is innovative and could also be valid for other imaging dimensions.
Next, we implemented and calibrated a polarimetric sensor using the «super-pixel« method. This sensor was then integrated into an optical bench to build a system capable of providing raw multispectral and polarimetric data. A database, as well as code to preprocess the images, were also built.

One of the results of the SPIASI project was the definition of a processing chain allowing the statistical analysis of 12-channel spectropolarimetric images (see illustration). From an image database, each pixel is labeled as resulting from either a specular reflection or a diffuse reflection. A segmentation is then carried out in relation to the type of reflection and the materials considered in the scene (wood, glass, metal, plastic, etc.). A study of inter-channel correlations was made available to the community, and highlights a clear dependence of these correlations on the materials analyzed. A system for instantaneous capture of polarization and spectral images was designed and calibrated radiometrically, spectrally and polarimetrically. It is composed of two cameras based on color filters and polarizers (12 bands in total for each camera) and bandpass spectral filters. The project also resulted in the publication of work that is not directly related to the surface inspection application, but which uses the instrumentation and processing chain developed within the project. This work focuses on fog removal in images and navigation based on spectropolarimetric data.

Perspective 1: Hyperspectral and polarimetric imaging applied to cultural heritage.
The intensity of specular reflections makes it difficult to capture images for cultural heritage applications. Pigment matching or conservation methods for paintings or stained glass can be impacted by these unwanted effects. This is partly addressed in the ongoing PHC AURORA project, which is attempting to upgrade a hyperspectral scanner at NTNU with a module for detecting linear polarization after reflection.
The objective would be to implement a low-noise Stokes linear spectral imaging framework, as well as to study the correlation between reflectance and the degree of polarization for an oil painting. Previous studies have shown the value of considering the wavelength dependence of the degree of polarization in the context of reflection imaging. The objectives are:
1. To propose a new spectropolarimetric system, capable of capturing images with sufficient spectral and spatial resolution for data analysis at the microstructural scale of the painting, i.e., brushstrokes.
2. To calibrate the system radiometrically and polarimetrically to ensure reliable measurements.
3. To capture paint containing specular reflections and jointly analyze the spectral and polarimetric signatures.

Another possible extension of this work would be to take the analysis further by using Mueller imaging, even if incomplete, rather than Stokes imaging. This work would be placed in a broader context aimed at achieving more in-depth material analysis in the context of cultural heritage. This could be achieved by polarizing the incident light using a linear and circular polarization state generator.

Perspective 2: Studying the Impact of Spatial Arrangements for Filtered Spectropolarimetric Sensors
The spatial arrangement of spectropolarimetric filters has been the subject of very little specific work. It is therefore necessary to study different spatial arrangements for CPFAs, with the aim of identifying whether some arrangements would be more appropriate to use than others. To my knowledge, there is only one alternative arrangement to that of the SONY IMX250 MYR sensor. Several methodologies for defining a new arrangement should be explored. One method would be to define an arrangement that performs best in terms of SNR, using an available reference database and a bank of synthesized mosaics. A brute-force matrixing-restoration-evaluation method could allow the definition of an optimal arrangement from an SNR perspective, as already done for SFAs.

Sattar, S.; Lapray, P. J.; Aksas, L.; Foulonneau, A.; Bigué, L. Snapshot spectropolarimetric imaging using a pair of filter array cameras. Opt. Eng. 2022, 61 (4), 043104-043104.

Lapray, P. J.; Thomas, J. B.; Farup, I. Bio-inspired multimodal imaging in reduced visibility. Front. Comput. Sci. 2022, 3, 737144.

Courtier, G.; Lapray, P. J.; Thomas, J. B.; Farup, I. Correlations in joint spectral and polarization imaging. Sensors 2020, 21 (1), 6.
Lapray, P. J. Exploiting redundancy in color-polarization filter array images for dynamic range enhancement. Opt. Lett. 2020, 45 (19), 5530-5533.

The project opened a collaboration with NTNU in Norway in 2022 with the PHC AURORA project «Spectral and Polarization Imaging of Specular Cultural Heritage Arti-facts«. An ANR scientific mediation project entitled «QuarCamp: Quartier du Campus« was also carried out, with Bénédicte LEBEAU as coordinator.

Polarimetric imaging enables the analysis of the polarization states of a reflected light beam. Many works have shown its benefits in
computer vision, particularly in applications such as specular reflection removing, decamouflage or dehazing. On the other hand,
multispectral imaging performs efficiently the spectral reconstruction of reflective objects along a given range of wavelengths. This
project aims to investigate the multimodal imaging model to help for material surface inspection. The work is planned in several
steps: acquire polarimetric/spectral database of images over different surfaces and illumination conditions, study the correlations
across the whole dataset and evaluate the methodology over computer vision algorithms that are dedicated to surface inspection in
industry. The contributions will be made through a demonstrator presented to industry actors, in order to build a consortium for a
future European project.