Amazon Web Services (AWS) has announced that the Technology Innovation Institute (TII), the applied research pillar of Abu Dhabi’s Advanced Technology Research Council, trained its top-performing, open source Falcon 40B model on AWS.
Falcon 40B is a 40-billion-parameter large language model (LLM) available under the Apache 2.0 license that ranked #1 in Hugging Face’s Open LLM Leaderboard. It was trained on 1 trillion tokens on Amazon SageMaker, a fully managed service for developing, training, tuning and hosting machine learning models, including LLMs.
The launch represents a significant milestone in the UAE’s commitment to fostering AI innovation and scientific contributions in line with the UAE National AI Strategy 2031.
Customers can now deploy Falcon 40B from Amazon SageMaker JumpStart, a machine learning (ML) hub that offers pre-trained models, giving customers access to Falcon 40B’s state-of-the-art accuracy and industry-leading performance without having to build their own model from scratch.
Dr. Ebtesam Almazrouei, Executive Director – Acting Chief AI Researcher of AI Cross Center Unit and Project Lead for LLM Projects at TII, highlighted: "By releasing Falcon-40B as an open-source model, we provide researchers, entrepreneurs and organizations with the opportunity to harness its exceptional capabilities and drive advancements in AI-driven solutions from healthcare to space, finance, advanced manufacturing to biotech; the possibilities for AI-driven solutions are boundless.
TII turned to Amazon SageMaker to build its Falcon 40B model, which allowed them to focus on custom training mechanisms and optimizations instead of managing its ML infrastructure because SageMaker is fully managed. Moreover, AWS collaborated with TII to use SageMaker to improve resiliency and reduce developer interruptions during training.
Amazon SageMaker Jumpstart makes Falcon 40B easier for customers to access. Now, customers of any size and industry can quickly and easily deploy their own Falcon 40B model and customize it for translation, question answering, summarizing information or identifying images.
Wojciech Bajda, Managing Director – Public Sector of Middle East and Africa at AWS, said: "We are proud to have collaborated with the Technology Innovation Institute on the development of the Falcon LLM 40B model, which leverages Amazon SageMaker to train a model that is state-of-the-art in multiple dimensions: technological breakthroughs, science quality, training speed and also operational simplicity. The open source release of Falcon-40B enables organizations to leverage its exceptional capabilities and advance AI-driven solutions to create new opportunities for progress and advancements."
