ECHO: The New Free Tool Transforming AI Chatbot Research

With artificial intelligence (AI) rapidly transforming our interactions and information processing, understanding its impact on trust, learning, and decision-making is crucial. Efforts are underway to design AI systems that are fair, efficient, and inclusive. A groundbreaking addition to these efforts is a new research tool from the University of Oklahoma, designed to facilitate these studies.

Named ECHO—Evaluation of Chat, Human Behavior, and Outcomes—this open-source, low-code platform is created to assist researchers in examining human-AI interactions, focusing primarily on conversational AI and online information seeking. The tool was developed by Jiqun Liu, an associate professor at the university, along with graduate student Nischal Dinesh and Ran Yu from GESIS—Leibniz Institute for the Social Sciences in Germany.

ECHO streamlines the setup of experimental studies by removing the need for front-end programming, thereby significantly lowering the time and cost typically associated with such projects. This accessibility encourages broader participation in research regarding the effects of AI chatbots on human behavior.

The inspiration behind ECHO is the ongoing shift from traditional keyword-based search engines to AI-powered conversational systems, altering how information is acquired and utilized. ECHO empowers researchers to investigate how these changes influence learning, trust, and decision-making by offering an accessible platform for configuring experiments, collecting data, and analyzing outcomes.

Getting started with ECHO is straightforward. Researchers can download and install the software from GitHub within about 20 minutes, aided by a comprehensive video tutorial. The platform features a dashboard for designing experimental workflows, and participants engage through a web-based interface to complete tasks and surveys. All data is meticulously logged and available for export for in-depth analysis.

The versatility of ECHO is highlighted in several ongoing projects, such as evaluating how chatbots influence information retention on contentious topics or improving online information accessibility related to intimate partner violence. These studies, among others, aim to provide valuable insights into the subtleties of human-AI interaction.

In developing ECHO, Liu aims to foster “healthy friction” in AI system design and evaluation by uncovering and addressing issues of bias and misinformation that may arise in AI-human interactions. By making this platform available for free to the research community, Liu hopes to encourage further interdisciplinary studies in this evolving field, promoting a better understanding of AI’s effects on human behaviors and decision-making.

Key Takeaways:

• ECHO is a new, free tool designed to support research into how AI chatbots impact trust, learning, and decision-making.
• It simplifies experimental study setup by eliminating the need for extensive programming knowledge, widening accessibility to AI research.
• The platform supports diverse studies, from information retention to improving access to social services, targeting critical facets of human-AI dynamics.
• By supporting an open-source environment, ECHO fosters global research participation and collaboration to tackle challenges posed by AI systems.