Educational and Interactive Plant Recognition for Smartphone Software – ReVeRIES

The research will be validated by the experimentation of 2 prototypes of mobile applications based on plants recognition algorithms from pictures.
The first prototype is a set of mobile learning games that can be created from an authoring tool that does not require any programming knowledge. For example, a user can play the role of a botanist whose objective will be to locate and identify plants, according to predefined game mechanisms (role playing, collaboration, competition, ...).
The second prototype is a tool for plants inventories in a collaborative way. The objective is to measure the impact of ludification and adaptation on this type of activities.
These prototypes will be built on the results of the ANR ReVeS project and the Folia application (Android and IOS application) as well as new multimodal recognition algorithms. They will recognize trees and shrubs found in France (native plants and main exotic plants).

- A meta-model is proposed in order to create a wide variety of mobile educational games
- Recognition of a tree from images of its leaves and bark
- A study of 400 students and 50 teachers in botany
- Needs analysis at the Echologia park

Thanks to the results of the ReVeRIES project and the developed prototypes that will be publically available on app distribution platforms, we believe the project will have significant impact on a large variety of users.
The application for the first usage scenario is addressed to neophytes in the context of a hike along a botanical trail. The game mechanics, such as treasures hunts and group activities, used for this scenario makes it particularly well adapted to children and families. It could also be used for school field trips, or outings organized by botanist associations and national parks.
The application for the second scenario is addressed to field botanists who want to improve their identification skills and collect data for botanical inventories. This tool could be used by natural parks or botanical associations who want to set up local atlases and need to train and motivate a large number of volunteers for collecting data.

P.-Y. Gicquel, S. George, I. Marfisi « Technologies sémantiques pour l’apprentissage de la botanique en mobilité », Presented at the SemWebPro professional Conference, 21 November 2016, Paris, France.

S. George, D. Coquin, T. Joliveau, V. Malécot, L. Tougne (2016), Conception d’applications ludo-éducatives mobiles en botanique, Colloque Ludovia, 23-26 août 2016, Ax-les-Thermes, France.

I. Marfisi, P.-Y. Gicquel, S. George (2016), Meta Serious Game: Supporting Creativity Sessions for Mobile Serious Games, 10th European Conference on Games Based Learning (ECGBL), 6 - 7 October 2016, Paisley, Scotland,

I. Marfisi, P.-Y. Gicquel, A. Karoui, S. George (2016), From Idea to Reality: Extensive and Executable Modeling Language for Mobile Learning Games, 11th European Conference on Technology Enhanced Learning (EC-TEL), 13 - 16 September 2016, Lyon, France,

R. Ben Ameur, L. Valet, D. Coquin (2016), Sub-Classification Strategies for Tree Species Recognition, 23rd International Conference on Pattern Recognition, IEEE ICPR 2016, 4-8 December 2016, Cancun, Mexico.

R. Ben Ameur, L. Valet, D. Coquin, S. Galichet (2016), Système d’aide à la reconnaissance d’espèces d’arbres à partir d’une base de connaissance incomplète, partielle et conflictuelle. 25ème Rencontres Francophones sur la Logique Floue et ses Applications (LFA 2016), La Rochelle, France.

R. Ben Ameur, L. Valet, D. Coquin (2016), Fusion System Based on Belief Functions Theory and approximated Belief Functions for Tree Species Recognition. 6th International Conference on Image Processing Theory, Tools and Applications, IEEE IPTA, 12-15 December 2016, Oulu, Finland.

R. Ben Ameur, L. Valet, D. Coquin (2016), A Fusion System for Tree Species Recognition through Leaves and Barks. IEEE Symposium Series on Computational Intelligence, 6-9 December 2016, Athens, Greece.

S. Bertrand, G. Cerutti, L. Tougne (2016), Bark Recognition to Improve Leaf-based Classification in Didactic Tree Species Identification, VISAPP 2017

The urbanization of society has gradually separated humans from the plant world. Most people have forgotten the names of plants and their potential uses. Yet there is a growing awareness that biodiversity is a treasure we must preserve and transmit to future generations. The identification of plant species is a necessary step to understand our environment. However, for most people, botany remains difficult to understand and to learn. It is not easy to decrypt botanical literature because it requires a solid theoretical background.
In the ReVeRIES project (French acronym that means “dreams” and that stands for Interactive, Fun and Educational Plant Recognition on Smartphones), we propose to use mobile technologies in order to help humans recognize plants that surround them. We believe that a promising way to recreate the relationship between modern human beings and their natural environment is to provide smartphone applications that help them recognize and learn about plants.

The ReVeRIES project relies on a mobile application called Folia and developed during the ANR ReVeS project. This application is capable of recognizing species of trees and shrubs (taller than 1m20 and originating from France) by analyzing photos of their leaves. This prototype simulates the behavior of a botanist when trying to determine the plant species, which makes it different from all the other tools available on the market. In the context of ReVeRIES, we propose to go much further by developing the following aspects: game-based mobile learning, multimodal images recognition and citizen sciences.
First of all, we intend to design mobile learning games that will help users learn about plant characteristics and especially learn the methods, used by expert botanists, to recognize plant families, genera and species. In order to motivate children and botanical neophytes to learn about plants and explore their natural environment, we also intend to use game mechanics for creating fun activities based on plant recognition. The users will be able to improve their skills by comparing their results to those found by the recognition algorithm.
Concerning the image recognition process, we intend to extend the previous prototype to the main exotic woody trees and shrubs. Moreover, we aim to take into account various organs of the plant. This multimodality is essential if we want users to learn and practice the correct recognition method, for which botanists use a variety of organs (i.e. leaf, bark, size of plant, flower, fruit, etc.). In addition, the use of organs should greatly improve the algorithm’s accuracy. In terms of image processing, the work done on the leaves cannot be extended directly to flowers, fruits and barks. This will greatly increase the complexity of the data fusion process.
Finally, we intend to explore ways in order to enhance social awareness of our natural resources and to support citizen science. The geolocated photos and information taken with the application and validated by experts, could be transferred to specialized networks, such as Tela Botanica, integrated into the OpenStreetMap geographic information system and mobilized by local institutions to support actions and projects involving citizens. This addresses problems related to the field of Volunteered Geographical Information.

The project raises many scientific challenges in TEL (Technology Enhanced Learning), Serious Game, image analysis, data fusion, HCI, and also in the field of collaborative environmental inventories. The possible impacts are numerous: teaching of botany at different levels and with various learning audiences, collective intelligence, citizen sciences, nature preservation and environmental collaborative games. In addition to citizens interested in nature, this system could be very useful for teachers and their students, botanists and also nature parks.