Databricks publishes Dolly 2.0, the first commercially available open-source 12B Chat-LLM
In Brief
Databricks announced the launch of Dolly 2.0 as the first commercially available open-source 12B Chat-LLM, allowing companies to create powerful language models without investing in expensive GPU clusters.
Databricks has announced the launch of Dolly 2.0 as the first commercially available open-source 12B Chat-LLM. Dolly is a major breakthrough for the machine learning industry, allowing companies to create powerful language models—such as autoregressive LLMs with dialogue support — without having to invest in expensive GPU clusters.
| Read more: How to Use ChatGPT (GPT-4) for Free Forever |
To obtain their open-source language model, Databricks needed a large, high-quality dataset. After initially considering the Alpaca GPT-3 dataset and other open-source datasets that do not allow for commercial use, they decided to build their own high-quality dataset. To do this, they employed 5,000 people and asked them to write several examples that would be used to train an autoregressive LLM capable of simply continuing the text and not conducting a dialogue. With the help of great incentives, such as bonuses for writing the best examples, they were able to assemble an impressive 15,000 sample dataset.
Databricks took the next step and incorporated EleutherAI’s Pythia-12B language model with an MIT license, and retrained it on their own dataset, resulting in Dolly 2.0. Dolly 2.0 has been released under an MIT license, along with the accompanying code. With the launch of Dolly 2.0, the machine learning industry now has a powerful tool to create robust language models in a cost-effective way. This can be utilized by companies and individuals alike who are looking to create powerful applications that can understand and process complex language.
Overall, the launch of Dolly 2.0 as the first commercially available open-source 12B Chat-LLM is a significant milestone for the machine learning industry. By taking advantage of the MIT license and creating its own high-quality dataset, Databricks has made powerful language models accessible to companies and individuals alike. Dolly 2.0 stands to revolutionize the industry and is a major step forward for machine learning.
Users are now waiting for the developers to quantize Dolly in 4 bits and speed it up to run on the CPU, as well as a response from OpenAssistant, which is expected to provide their model in the next few days.
- The NTIA is asking for public comments on possible regulations that regulate AI systems like ChatGPT to help the Biden administration ensure that they function as promised. The NTIA released a statement regarding AI accountability rulemaking, which could prevent future mishaps and minimize the risks from hypothetical threats.
Read more about AI:
Disclaimer
In line with the Trust Project guidelines, please note that the information provided on this page is not intended to be and should not be interpreted as legal, tax, investment, financial, or any other form of advice. It is important to only invest what you can afford to lose and to seek independent financial advice if you have any doubts. For further information, we suggest referring to the terms and conditions as well as the help and support pages provided by the issuer or advertiser. MetaversePost is committed to accurate, unbiased reporting, but market conditions are subject to change without notice.
About The Author
Damir is the team leader, product manager, and editor at Metaverse Post, covering topics such as AI/ML, AGI, LLMs, Metaverse, and Web3-related fields. His articles attract a massive audience of over a million users every month. He appears to be an expert with 10 years of experience in SEO and digital marketing. Damir has been mentioned in Mashable, Wired, Cointelegraph, The New Yorker, Inside.com, Entrepreneur, BeInCrypto, and other publications. He travels between the UAE, Turkey, Russia, and the CIS as a digital nomad. Damir earned a bachelor's degree in physics, which he believes has given him the critical thinking skills needed to be successful in the ever-changing landscape of the internet.
More articles
Damir is the team leader, product manager, and editor at Metaverse Post, covering topics such as AI/ML, AGI, LLMs, Metaverse, and Web3-related fields. His articles attract a massive audience of over a million users every month. He appears to be an expert with 10 years of experience in SEO and digital marketing. Damir has been mentioned in Mashable, Wired, Cointelegraph, The New Yorker, Inside.com, Entrepreneur, BeInCrypto, and other publications. He travels between the UAE, Turkey, Russia, and the CIS as a digital nomad. Damir earned a bachelor's degree in physics, which he believes has given him the critical thinking skills needed to be successful in the ever-changing landscape of the internet.
