Automatic generation of multimodal data of mechanical parts assemblies for machine learning in product reverse engineering – GENERAT3D
Automatic generation of multimodal data of mechanical parts assemblies for machine learning in product reverse engineering
/
Challenges and objectives
Collecting and labeling data for machine learning can be a time-consuming task, especially in a multimodal context (reverse engineering can be done from several modalities) where it is necessary to have labeling (by part) in several types of representations (B-Rep models, images, point clouds, depth maps, etc.). The objective of the GENERAT3D project is to develop methods for automatically generating large volumes of labeled data to feed learning methods for reverse engineering of mechanical parts and assemblies. The originality of the project lies in the fact that it is possible to automatically label certain types of data (e.g. B-Rep surfaces) and to propagate these labels during the generation of other modalities. Methods for generating photo-realistic images, point clouds as scanned, as well as depth maps will be developed. Finally, case studies using these data will illustrate their exploitation.
/
The main major result of this project is the creation of an automatic reassembly engine for mechanical parts, which was published in a journal article. This system allows proposing possible mechanical assemblies from a set of partially assembled or unassembled parts. This allows several use cases, the main one being the expansion of assembly databases from a small subset of mechanical assemblies, in order to build large databases containing a wide variety of assemblies for machine learning. This system also allows searching for compatible parts for a given interface, this can allow automatic searches for replacement parts in existing assemblies (for example to recycle existing parts in other assemblies). A second part aimed at making this data multimodal is in progress, using photo-realistic rendering engines and point cloud generators. Image and point cloud databases can be built based on these generated assemblies, in order to perform machine learning on this new data.
/
Vergez, L.; Polette, A.; Pernot, J.P. «Interface-Based Search and Automatic Reassembly of CAD Models for Database Expansion and Model Reuse.« Computer-Aided Design. 2024, 167, 103630.
Vergez, L.; Polette, A.; Pernot, J.P. «Automatic CAD Assemblies Generation by Linkage Graph Overlay for Machine Learning Applications.« Computer-Aided Design and Applications. 2021, 19 (4), 722-732.
Vergez, L.; Polette, A.; Pernot, J.P. «Détection automatique de contraintes cinématiques d'assemblages de modèles CAO.« S-MART 2023: 18ème Colloque national S. mart.
Collecting and labelling data for machine learning is a time-consuming task, especially in a multimodal context where it is necessary to have labelling in several types of data (B-Rep models, images, point clouds, etc.). The objective of this project is to set up methods for automatically generating large volumes of labelled data to feed learning methods for the reverse engineering of mechanical parts and assemblies (a problem requiring this multimodality). The originality of the project is based on the fact that it is possible to automatically label certain types of data (e. g. B-Rep surfaces) and to propagate these labels when the other modalities are generated. Methods for generating photorealistic images, synthetic point clouds and depth maps will be developed. Finally, two case studies using these data will demonstrate their usefulness.
Project coordination
ARNAUD POLETTE (Ecole Nationale Supérieure d'Arts et Métiers - Laboratoire d Ingénierie des Systèmes Physiques et Numériques)
The author of this summary is the project coordinator, who is responsible for the content of this summary. The ANR declines any responsibility as for its contents.
Partnership
ENSAM - LISPEN Ecole Nationale Supérieure d'Arts et Métiers - Laboratoire d Ingénierie des Systèmes Physiques et Numériques
Help of the ANR 172,704 euros
Beginning and duration of the scientific project:
December 2021
- 42 Months
