News Report Technology
April 04, 2023

PyTorch 2.0 Release: A Major Update to the Machine Learning Framework

PyTorch 2.0 Release: A Major Update to the Machine Learning Framework

PyTorch has announced the release of PyTorch 2.0, the open-source machine learning framework, which was highly anticipated by the data science community. The team delivered several new features and enhancements to the platform, increasing its potency and adaptability.

The framework is used for computer vision and natural language processing applications and is under the Linux Foundation umbrella. It provides tensor computing with GPU acceleration and deep neural networks built on automatic differentiation. Some deep learning software, such as Tesla Autopilot, Pyro, Transformers, PyTorch Lightning, and Catalyst, is built on top of PyTorch.

PyTorch 2.0 implements a new high-performance Transformer API, which aims to make training and deployment of state-of-the-art Transformer models more affordable. The release also includes high-performance support for training and inference using a custom kernel architecture for scaled dot product attention (SPDA).

At a similar time, PyTorch released OpenXLA and PyTorch/XLA 2.0. The combination of PyTorch and XLA provides a development stack that can support both model training and inference. This is possible because PyTorch is a popular choice in AI, and XLA has excellent compiler features. To improve this development stack, there will be investments in three main areas.

To train large models, PyTorch/XLA is investing in features such as mixed precision training, runtime performance, efficient model sharding, and faster data loading. Some of these features are already available, while others will be released later this year, leveraging the underlying OpenXLA compiler stack.

For model inference, PyTorch/XLA focuses on delivering competitive performance with Dynamo in the PyTorch 2.0 release. Additional inference-oriented features include model serving support, Dynamo for sharded large models, and quantization via Torch.Export and StableHLO.

In terms of ecosystem integration, PyTorch/XLA is expanding integration with Hugging Face and PyTorch Lightning so users can take advantage of upcoming features and the downstream OpenXLA features through familiar APIs. This includes support for FSDP in Hugging Face and Quantization in OpenXLA.

PyTorch/XLA is an open-source project, which means you can contribute to its development by reporting issues, submitting pull requests, and sending requests for comments (RFCs) on GitHub.

Read more:

Disclaimer

In line with the Trust Project guidelines, please note that the information provided on this page is not intended to be and should not be interpreted as legal, tax, investment, financial, or any other form of advice. It is important to only invest what you can afford to lose and to seek independent financial advice if you have any doubts. For further information, we suggest referring to the terms and conditions as well as the help and support pages provided by the issuer or advertiser. MetaversePost is committed to accurate, unbiased reporting, but market conditions are subject to change without notice.

About The Author

Agne is a journalist who covers the latest trends and developments in the metaverse, AI, and Web3 industries for the Metaverse Post. Her passion for storytelling has led her to conduct numerous interviews with experts in these fields, always seeking to uncover exciting and engaging stories. Agne holds a Bachelor’s degree in literature and has an extensive background in writing about a wide range of topics including travel, art, and culture. She has also volunteered as an editor for the animal rights organization, where she helped raise awareness about animal welfare issues. Contact her on [email protected].

More articles
Agne Cimerman
Agne Cimerman

Agne is a journalist who covers the latest trends and developments in the metaverse, AI, and Web3 industries for the Metaverse Post. Her passion for storytelling has led her to conduct numerous interviews with experts in these fields, always seeking to uncover exciting and engaging stories. Agne holds a Bachelor’s degree in literature and has an extensive background in writing about a wide range of topics including travel, art, and culture. She has also volunteered as an editor for the animal rights organization, where she helped raise awareness about animal welfare issues. Contact her on [email protected].

Hot Stories
Join Our Newsletter.
Latest News

Supply and Demand Zones

Cryptocurrency, like any other currency, is a financial instrument based on the fundamental economic principles of supply ...

Know More

Top 10 Crypto Wallets in 2024

With the current fast-growing crypto market, the significance of reliable and secure wallet solutions cannot be emphasized ...

Know More
Join Our Innovative Tech Community
Read More
Read more
Binance Executive Tigran Gambaryan Files Lawsuit Against Nigerian Government
Business News Report
Binance Executive Tigran Gambaryan Files Lawsuit Against Nigerian Government
March 29, 2024
FTX’s Meteoric Ascent and Crash. Inside Sam Bankman-Fried’s Ambitious Venture
Stories and Reviews Technology
FTX’s Meteoric Ascent and Crash. Inside Sam Bankman-Fried’s Ambitious Venture
March 29, 2024
Binance Unveils Launchpool Project Ethena for BNB, FDUSD and TUSD Staking
News Report Technology
Binance Unveils Launchpool Project Ethena for BNB, FDUSD and TUSD Staking
March 29, 2024
Parcl Unveils Tokenomics Ahead of PRCL Launch, Allocates 75M Tokens to Point Holders
News Report Technology
Parcl Unveils Tokenomics Ahead of PRCL Launch, Allocates 75M Tokens to Point Holders
March 28, 2024
What You
Need to Know

Subscribe To Our Newsletter.
Daily search marketing tidbits for savvy pros.