Deep Co-design of a Privacy Preserving Intelligent Camera – DOOPLER

Today, we are surrounded by surveillance systems that generate billions of images per day processed by computer vision techniques. This situation raises legal issues related to privacy. If privacy is sometimes protected by post-processing the images, for example by blurring them, this protection can be attacked by gaining access to the camera's raw output. The challenge is therefore to provide privacy directly at the camera level by deliberately degrading the quality of the images produced by the lens. The main difficulty is then to maintain a high performance analysis task despite this degradation. To design such a camera, a joint optimisation of the optical and neural network parameters is necessary, an approach we refer to as deep co-design. Furthermore, an adversarial learning method has to be developed in order to guarantee both objectives: scene analysis and privacy. State of the art methods have proposed the deep co-design of privacy preserving cameras using an adversarial learning, but they optimise a single optical component using Fourier optics based differentiable optical model, which is valid only for small field of view. In contrast, we propose the deep co-design of a privacy preserving camera using a differentiable ray tracing software developed at ONERA. This tool allows the optimisation of the full set of lens parameters and thus the design of a camera with a large field of view. In addition, we propose to investigate the use of unconventional surfaces within the lens in order to have more degrees of freedom in the lens response. Overall, the co-design of this system, including large field of view, unconventional lens and adversarial training, presents many scientific challenges to be addressed in this project.