Hugging Face CEO Predicts Smaller AI Models will Dominate 2024
In Brief
2024 will see the rise of Small Language Models, as companies push the boundaries of efficiency, cost-effectiveness and accessibility.
For artificial intelligence, the year 2024 is poised to mark a significant turning point — with the rise of Small Language Models (SLMs), as companies push the boundaries of efficiency, cost-effectiveness and accessibility.
The journey from the dominance of massive Large Language Models (LLMs) to the emergence of compact, powerful SLMs promises to reshape the AI landscape.
This claim has found its backing form Clam Delangue, co-founder and CEO of Hugging Face.
“Phi-2 by Microsoft AI is now the number one trending model on Hugging Face. 2024 will be the year of small AI models!” said Delangue, in a LinkedIn post.
Furthermore, in early December, French AI startup Mistral, soon after raising a substantial $415 million funding round, introduced Mixtral 8x7B, an open-source SLM that has quickly gained traction for its ability to rival the quality of GPT-3.5 on certain benchmarks, all while running on a single computer with a modest 100 gigabytes of RAM.
Mistral’s approach, termed a ‘sparse mixture of experts’ model, combines smaller models trained for specific tasks, achieving remarkable efficiency.
Not to be outdone, tech giant Microsoft entered the arena with Phi-2, the latest version of its home-grown SLM. Notably tiny with just 2.7 billion parameters, Phi-2 is designed to run on a mobile phone, showcasing the industry’s commitment to downsizing models without compromising capabilities.
Models like GPT-3, boasting a staggering 175 billion parameters, showcased the ability to generate human-like text, answer questions and summarize documents. However, the inherent downsides of LLMs, including concerns related to efficiency, cost, and customizability, have paved the way for the ascendance of SLMs.
Factors Driving Small-Scale Language Model Development
SLMs boast a streamlined approach with fewer parameters, resulting in faster inference speed and higher throughput. Their reduced memory and storage requirements make computational processes agile, challenging the conventional belief that model capacity must always parallel the growth of data appetite.
While large language models like GPT-3 incur exorbitant costs – often in the tens of millions of dollars for development – SLMs present a cost-effective alternative.
These models can be trained, deployed and operated on readily available commodity hardware, making them a financially viable choice for businesses. Moreover, their modest resource requirements position them as ideal candidates for applications in edge computing, running offline on lower-powered devices.
Similarly, a key strength of SLMs lies in their customizability. Unlike their larger counterparts, which represent compromises across domains, SLMs can be finely tuned for specific applications. Their quick iteration cycles facilitate practical experimentation, allowing developers to adapt models to particular needs.
As we approach 2024, the rise of small language models signals a transformative era in artificial intelligence. The stage is set for the Year of Small AI Models, where innovation and accessibility converge to redefine the possibilities of artificial intelligence.
Disclaimer
In line with the Trust Project guidelines, please note that the information provided on this page is not intended to be and should not be interpreted as legal, tax, investment, financial, or any other form of advice. It is important to only invest what you can afford to lose and to seek independent financial advice if you have any doubts. For further information, we suggest referring to the terms and conditions as well as the help and support pages provided by the issuer or advertiser. MetaversePost is committed to accurate, unbiased reporting, but market conditions are subject to change without notice.
About The Author
Kumar is an experienced Tech Journalist with a specialization in the dynamic intersections of AI/ML, marketing technology, and emerging fields such as crypto, blockchain, and NFTs. With over 3 years of experience in the industry, Kumar has established a proven track record in crafting compelling narratives, conducting insightful interviews, and delivering comprehensive insights. Kumar's expertise lies in producing high-impact content, including articles, reports, and research publications for prominent industry platforms. With a unique skill set that combines technical knowledge and storytelling, Kumar excels at communicating complex technological concepts to diverse audiences in a clear and engaging manner.
More articles
Kumar is an experienced Tech Journalist with a specialization in the dynamic intersections of AI/ML, marketing technology, and emerging fields such as crypto, blockchain, and NFTs. With over 3 years of experience in the industry, Kumar has established a proven track record in crafting compelling narratives, conducting insightful interviews, and delivering comprehensive insights. Kumar's expertise lies in producing high-impact content, including articles, reports, and research publications for prominent industry platforms. With a unique skill set that combines technical knowledge and storytelling, Kumar excels at communicating complex technological concepts to diverse audiences in a clear and engaging manner.
