Learning with Multi-objective OptimizatioN – LEMON

he LeMOn project is an academic project with strong potential repercussions in many application domains. It takes place at the frontiers between three research communities that strongly interplay : machine learning, pattern recognition and multi-objective optimization. The rationale behind this project stems from the fact that most existing learning machines and pattern recognition systems are designed by optimizing a single criterion for a single task, whereas multiple criteria and/or multiple tasks are generally involved in real-world applications. As examples, one can cite the trade-offs between learning performance and generalization ability, between sensitivity and specificity or between the scalability and the decision speed. Hence, machine learning is inherently a multi-objective optimization problem and the aim of the LeMOn project is to initiate a break in the theory and the methodology of machine learning, by taking into account multiple criteria and tasks in the learning machine design. In this general framework, the LeMOn project propose to tackle two particular multi-objective learning problems : (1) The first problem concerns learning in undefined or weakly defined contexts, i.e. when a priori probabilities and/or misclassification costs are unknown. In such a context, it is well known that a single learning criterion does not lead to a classifier able to face to many sitations. In a previous work, we have proposed a multi-model selection framework based on a Pareto-based multi-objective approach working in the ROC space. By introducing the "ROC front concept", the idea was to train a pool of classifiers instead of a single one, each classifier in the pool optimizing a particular trade-off between the objectives. In the LeMOn project, we plan to extend this approach to large-scale problems, by considering on-line learning and to multi-class problems by investigating recent advances in multi-class ROC analysis. (2) The second problem concerns the Multi-Task Learning framework and its strong links with multi-objective optimization. Multi-Task Learning (MTL) is a statistical learning framework which seeks at learning several models in a joint manner. The idea behind this paradigm is that, when the tasks to be learned are similar enough or are related in some sense, it may be advantageous to take into account these relations between tasks. Such a framework strongly relies on an multi-objective optimization process where compromise have to be done between objectives related to each task. In
the LeMOn project, we will focus on two speficic problems occuring in a MTL problem tackled with kernel methods. The first one consists in optimizing the choice of mixed norm penalty in the regularizing term while the second aims at finding the best weighting of each task. In each case, the proposed algorithms will be mainly evaluated on two particular applications which are inherently multi-objective and multi-task : Brain Computer Interfaces and medical image classification. Other applications such as document image analysis or fraud detection may also be envisaged.