Artificial intelligence (AI) is revolutionizing various sectors, and assistive technology for blind and low-vision (BLV) individuals is no exception. Although we’ve achieved considerable progress, further enhancements are needed to maximize the potential of these AI tools.
A recent pioneering study by researchers at Cornell Tech evaluated an application powered by a multimodal large language model (MLLM) aimed at assisting BLV users in understanding their surroundings. This research involved 20 vision-impaired participants and demonstrated that the application performed adequately for basic inquiries like “What is this?” Nevertheless, it encountered difficulties with more intricate tasks needing in-depth analysis, such as describing artworks.
Based on their study, the researchers identified nine essential “skills” or areas that need enhancement to improve AI models’ performance. Key improvements focused on enhancing factual accuracy and implementing adaptive communication strategies to support more reliable and goal-oriented assistance. These improvements are centered on maintaining a user-centric approach, ensuring that the model aligns with the users’ needs.
Shiri Azenkot, an associate professor at Cornell Tech, discussed these advancements and highlighted necessary steps for creating more comprehensive AI tools. Azenkot, who is legally blind, emphasized the personal and societal significance of refining these technologies. She revealed that while AI tools are beneficial, they present risks as current applications yield only 56.6% accuracy in responses, with 22.2% of answers containing inaccuracies.
This research spurred the development of a visual interpretation application, VisionPal, using GPT-4o technology. Conducting a diary study from October to December 2024, participants recorded their interactions with this application. This study’s results underscored the necessity for ongoing auditing and fine-tuning of AI systems to mitigate risks and enhance user trust.
In summary, AI applications have significantly empowered individuals with vision impairments but are still evolving. Ongoing research and development are pivotal in bridging existing gaps and refining these tools to tackle complex scenarios. Ultimately, ensuring that AI developments remain centered on enhancing users’ lives is crucial to their success and adoption.