Apertus: Pioneering Transparency and Inclusivity in AI Language Models

In July 2023, an exciting collaboration among three leading Swiss institutions, EPFL, ETH Zurich, and the Swiss National Supercomputing Centre (CSCS), unveiled a groundbreaking initiative in the world of artificial intelligence. Their ambitious project, a large language model (LLM) called “Apertus,” has now been released to the public as of September, promising to serve as a versatile foundation for a wide range of applications, including chatbots, translation systems, and educational tools.

Unveiling Apertus

The model is aptly named “Apertus,” which translates to “open” in Latin, a testament to its central feature: total transparency. Unlike the many proprietary LLMs that operate behind closed doors with restricted access, Apertus distinguishes itself by documenting every aspect of its architecture, training data, and processes openly. This openness invites researchers and developers globally to utilize and build upon Apertus, marking a significant shift towards more transparent AI development.

Apertus is available in two versions, equipped with 8 billion and 70 billion parameters, ensuring it can flexibly adapt to various applications, whether for individual, educational, or commercial uses. These models can be accessed through Swisscom or downloaded from Hugging Face, a popular AI model repository, under a permissive open-source license.

Innovating Through Inclusion

Apertus not only pushes the boundaries of technological innovation but also champions inclusivity. The collaboration between EPFL, ETH Zurich, and CSCS positions the model as a tool for enhancing AI expertise across diverse sectors, from academia to industry. Trained on 15 trillion tokens across more than 1,000 languages, Apertus includes many underrepresented languages, thus addressing diverse linguistic requirements and promoting multilingual inclusivity.

In adhering to transparency, the model’s development respects existing data protection and copyright laws, alongside the EU AI Act’s transparency obligations. Its training dataset upholds data integrity and ethical considerations by excluding undesired content and honoring opt-out requests from content sources.

Forward Path

As Antoine Bosselut from EPFL states, “Apertus is the start of a journey.” This highlights the model’s potential to foster open and trustworthy AI infrastructure on a global scale. The upcoming Swiss (ai){token}{weeks} will provide developers the opportunity to experiment with Apertus, encouraging innovation and gathering feedback for future improvements. This initiative underscores their vision of establishing AI as a public good, akin to essential infrastructure like roads and utilities.

In summary, Apertus not only showcases the capabilities of open generative AI but also sets a precedent for future language models to harmonize power, transparency, and ethical compliance, ultimately serving the broader public interest.