Swiss launch open source AI model as “ethical” alternative to big US LLMs

In January, assumptions around AI were shaken up by DeepSeek, a small Chinese company that nobody had heard of. This week it was Switzerland’s turn to stir things up.

Apertus (Latin for ‘open’) is a brand new large language model (LLM) that its creators, a group of Swiss universities in collaboration with the Swiss National Supercomputing Centre (CSCS), claim is one of the most powerful open-source AI platforms ever released.

Benchmarked as roughly on par with Meta’s Llama 3 model from 2024, Apertus is not the most powerful LLM out there, but it is still formidable.

Trained on 15 trillion tokens across using the 132-Nvidia H100 Alps CSCS supercomputer, it has been released on AI open source community site Hugging Face in an 8 billion parameter version for smaller-scale use, and a larger 70 billion parameter version suitable for research and commercial applications.

However, Apertus was created to be more than big, its makers said. Their ambition was to build something completely different to the ChatGPT AI mainstream, including better serving a global user base by training it on a wide range of non-English languages.

Fully open

The first interesting feature of Apertus is simply that it is Swiss. This might sound like a detail, but could yet be an advantage in an industry dominated by nations such as the US and China.

What’s being pitched here is an idea of sovereign or national AI in which Switzerland offers European and global users something more distinctive than Silicon Valley’s commercial AI models.

A major part of this will be the model’s open, ethical credentials. What this means is that anyone using it can get under the hood to see exactly how it was trained, in a way that is intended to be transparent and reproducible.

‘Openness’ in LLMs is contentious. For Meta’s Llama, for example, it refers to open weights, which means you can see how the LLM operates on data without seeing the data on which it was trained.

The Swiss, by contrast, prefer the idea of ‘full openness’ in which users can see everything, including training data and the 15 trillion token volume used by the model.

“Fully open models enable high-trust applications and are necessary for advancing research about the risks and opportunities of AI. Transparent processes also enable regulatory compliance,” said Imanol Schlag, a research scientist at the University of Zurich’s ETH AI Center who worked on the project.

This fully open nature would also be important for the organizations its makers believe might use it. One of the biggest enterprise concerns around using AI today is the domination of large, mainly US-based tech companies that risk pushing ethical and legal boundaries beyond their breaking point.

This is especially true in the EU, where enterprises must grapple with staying on the right side of the 2024 EU AI Act and General-Purpose AI Code of Practice. These have generated anxiety about compliance, on top of the more general worries that the strict standards the EU AI Act imposes might act as a brake on AI development in the EU.

“Particular attention has been paid to data integrity and ethical standards: the training corpus builds only on data which is publicly available,” said the Apertus announcement.  

“It is filtered to respect machine-readable opt-out requests from websites, even retroactively, and to remove personal data, and other undesired content before training begins.”

This includes high compliance standards in terms of author rights, as well as around the ‘memorization’ of training data that might create privacy problems with mainstream models through reproduction of snippets of copyrighted or sensitive data.

Additionally, organizations that want to stay in control of their data have the option to download Apertus to their own servers.

However, as with all open models, if copyrighted text or personal data is later removed from the training data, there is no easy way to guarantee that downloaded versions of the model reflect this, beyond asking enterprises to monitor for upstream changes. Inevitably, this puts some pressure on enterprise AI governance, who might be held liable for compliance issues.

Need for speed

Despite the ethical appeal of Apertus, it will still need to compete with rivals in terms of AI inference. The notion that organizations needed to go to large closed-source LLM makers to get this was mistaken, according to Antoine Bosselut, assistant professor at École Polytechnique Fédérale de Lausanne (EPFL), which also collaborated on the Swiss LLM.

“Over the last few years, we heard this narrative that commercial LLM providers were light years ahead of anything that anybody else could create. What I hope we’ve shown here today is that that’s not necessarily the case and that this gap is far less wide than we had imagined,” he said in a promotional video.

“Apertus demonstrates that generative AI can be both powerful and open,” said Bosselut. “The release of Apertus is not a final step, rather it’s the beginning of a journey, a long-term commitment to open, trustworthy, and sovereign AI foundations, for the public good worldwide.”

More AI news and insights:

White House AI plan heavy on cyber, light on implementation

How the generative AI boom opens up new privacy and cybersecurity risks

10 things you should include in your AI policy