Carnegie Mellon Researchers Present MLC LLM for Running Language Models on Any Device
Share this article
In Brief
MLC LLM and Web LLM allow users to deploy language models on any device, allowing for new applications and usages for natural language understanding.
Carnegie Mellon University researchers have presented MLC LLM, a set of tools that could revolutionize how language models are run on any device. Such models allow users to create a range of natural language applications, such as virtual assistants and smart chatbots. Through separate performance optimizations, MLC LLM can now be used on different platforms and scenarios.
The new tool, known as Web LLM, is also part of this initiative. It allows for the launch of language models directly in the browser, meaning users don’t have to manually download the system, which can be several gigabytes in size. The Vicuna 7B model, which has 7 billion parameters, has been created this way. This can be extremely useful when designing more fitting virtual assistants and chatbots as they require large amounts of parameters.
Using MLC LLM and Web LLM, it is now possible to deploy any language model of choice on any device. The researchers at Carnegie Mellon have reported that this would open up a range of applications that weren’t possible before. Any language model can now be deployed on various computing devices, such as laptops and phones, or even processors and video accelerators. This opens up a world of possibilities for natural language processing and machine learning.
The new initiative from Carnegie Mellon paves the way for a range of new applications and usages for natural language understanding. As you can use these language models on any device, it is much easier to deploy such technology in many different scenarios. Whether it’s for virtual assistants or automated customer service, this new tool could have a drastic impact on the way these tasks are done.
Carnegie Mellon’s research team developed MLC LLM to be run on any device, including iPhones and other smart home devices, with a modern processor and 6GB or more of RAM. This streamlined approach means that devices can now run language models with almost real-time generation speed, even when optimized by external developers and not Apple engineers.
In addition to allowing for natural interactions with products, MLC LLM can also be used to optimize device management tasks. With only 4GB to 6GB of RAM needed to run, this revolutionary language model will make an excellent addition to future iPhone models and could potentially be rolled into production without the lengthy development process.
By introducing the MLC LLM language model, Carnegie Mellon researchers have created an efficient and robust way to run language models on any device. The MLC LLM system is a breakthrough in natural language processing and will enable users to interact naturally with their devices more effectively. It also has the potential to significantly speed up the development process for future device management tasks. The possibilities are endless with MLC LLM, and Carnegie Mellon’s work will surely transform the way we think about language models for the better.
Read more about AI:
Disclaimer
Any data, text, or other content on this page is provided as general market information and not as investment advice. Past performance is not necessarily an indicator of future results.
The Trust Project is a worldwide group of news organizations working to establish transparency standards.
Damir is the team leader, product manager, and editor at Metaverse Post, covering topics such as AI/ML, AGI, LLMs, Metaverse, and Web3-related fields. His articles attract a massive audience of over a million users every month. He appears to be an expert with 10 years of experience in SEO and digital marketing. Damir has been mentioned in Mashable, Wired, Cointelegraph, The New Yorker, Inside.com, Entrepreneur, BeInCrypto, and other publications. He travels between the UAE, Turkey, Russia, and the CIS as a digital nomad. Damir earned a bachelor's degree in physics, which he believes has given him the critical thinking skills needed to be successful in the ever-changing landscape of the internet.
More articlesDamir is the team leader, product manager, and editor at Metaverse Post, covering topics such as AI/ML, AGI, LLMs, Metaverse, and Web3-related fields. His articles attract a massive audience of over a million users every month. He appears to be an expert with 10 years of experience in SEO and digital marketing. Damir has been mentioned in Mashable, Wired, Cointelegraph, The New Yorker, Inside.com, Entrepreneur, BeInCrypto, and other publications. He travels between the UAE, Turkey, Russia, and the CIS as a digital nomad. Damir earned a bachelor's degree in physics, which he believes has given him the critical thinking skills needed to be successful in the ever-changing landscape of the internet.
