Inception, a G42 company dedicated to pushing the boundaries of AI, announced the open-source release of “Jais”, the world’s highest-quality Arabic Large Language Model. Jais is a 13-billion parameter model trained on a newly developed 395-billion-token Arabic and English dataset.
With a name inspired by UAE’s highest peak, Jais will bring the advantages of generative AI across the Arabic-speaking world. The model results from a collaboration between Inception, Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) and Cerebras Systems. It was trained on Condor Galaxy; the recently announced multi-exaFLOP AI supercomputer built by G42 and Cerebras.
Jais’ release marks a significant milestone in the realm of AI for the Arabic world. It is a model homegrown in the UAE’s capital, Abu Dhabi, offering more than 400 million Arabic speakers the opportunity to harness the potential of generative AI. It will facilitate and expedite innovation, highlighting Abu Dhabi’s leading position as a hub for AI, innovation, culture preservation, and international collaboration.
By open-sourcing Jais, Inception aims to engage the scientific, academic, and developer communities to accelerate the growth of a vibrant Arabic language AI ecosystem. This can serve as a model for other languages currently underrepresented in mainstream AI.
“We believe that innovation thrives when we collaborate,” said Andrew Jackson, CEO of Inception. “With this release, we are setting a new standard for AI advancement in the Middle East and ensuring that the Arabic language, with its depth and heritage, finds its voice within the AI landscape. Jais is a testament to our commitment to excellence and dedication to democratising AI and promoting innovation.”
Jais outperforms existing Arabic models by a sizable margin. It is also competitive with English models of similar size despite being trained on significantly less English data. This exciting result shows that the model’s English component learned from the Arabic data and vice versa, opening a new era in LLM’s development and training.
MBZUAI President and University Professor Eric Xing said,” Developing such a high-calibre Arabic LLM demanded cutting-edge AI research in addition to an in-depth and nuanced understanding of the Arabic language, its diversity and heritage, and the growing importance of LLMs across all echelons of society. Thanks to our research and partnerships with Inception and other top regional and global organisations, MBZUAI will continue pioneering LLMs that are efficient, effective, and accurate.”
Jais is a transformer-based large language model that incorporates many cutting-edge features, including ALiBi position embeddings, which enables the model to extrapolate to much longer inputs, providing better context handling and accuracy. Other state-of-the-art techniques include SwiGLU and maximal update parameterisation to improve the model’s training efficiency and accuracy.
Jais’ training, fine-tuning, and evaluation were undertaken by an Inception/MBZUAI joint team on the Condor Galaxy 1 (CG-1), the recently announced, state-of-the-art AI supercomputer co-developed by G42 and Cerebras Systems. The 13-billion parameter open-source model was trained on a unique and purpose-built dataset of 116 billion Arabic tokens designed to capture Arabic’s complexity, nuance, and richness. It also included 279 billion English word tokens to increase the model’s performance through cross-language transfer. Inception and MBZUAI will continue to expand and refine Jais as its user community grows.
“Our strategic partnership with G42 is already delivering pioneering results. A few weeks ago, we introduced the first multi-exaFLOP AI supercomputer, Condor Galaxy 1 (CG-1). Now, the partnership delivers another key breakthrough: the leading Arabic LLM for the open-source community,” said Andrew Feldman, co-founder and CEO of Cerebras Systems. “At Cerebras, our passion is building groundbreaking technology. One of the great rewards is seeing the innovative ways it is used. Jais is a significant contribution to the international open-source community. It is also a testament to how incredibly easy CG-1 is to use and enables extremely rapid AI model development.”
Jais is available for download on Hugging Face. Users can also try Jais online upon registering interest on Jais’ website and receiving an invite to access the playground environment.