Robust Evaluation of Generative AI in Real-World Environments – REGARD

This project seeks to advance the evaluation of language generation systems by targeting in depth understanding their behavior and considering their impact on various dimensions like work environments and social good. The REGARD project will develop frameworks to define and measure performance, generalization, and robustness in real-world settings. While Large Language Models (LLMs) have achieved impressive results in controlled benchmarks, their deployment has led to significant challenges, including issues with atypical inputs, harmful outputs, and inefficiencies under resource constraints. Additionally, their broader impact on productivity, working environment, labor markets, and social cohesion is not well understood.

We propose to integrate social, algorithmic and theoretical perspectives in the evaluation of LLMs. Our main questions are: How do we define an AI system’s performance in alignment with human values and objectives? How do we identify stakeholders and evaluate the potential benefits or risks the system may pose to them? How can we evaluate a model’s capabilities to better understand its limitations?

Protocols will be developed to analyze benefit-harm trade-offs and value tensions in real-world scenarios. Evaluation methods will ensure valid and reliable measurements of these systems, grounded in our recent joint works in evaluating summarization systems and embedding models to provide theoretical foundations and formal guarantees related to performance metrics. Case studies will demonstrate the approach's applicability in real-world settings.

By combining principles from human-computer interaction, machine learning, NLP, and information theory, this interdisciplinary effort aims to help AI developers conduct more rigorous evaluations, ensuring that generative models are reliable, robust, and adaptable to complex real-world environments.