In Brief
Google announces new AI model for producing high-quality visual summaries of news articles
The Trust Project is a worldwide group of news organizations working to establish transparency standards.
Google has announced a new artificial intelligence (AI) model that it says can automatically generate “coherent and fluent” visual summaries of news articles. The technology giant says the model can help make information more accessible to users by generating “high-quality” visual summaries of long pieces of text.
The model is based on a deep learning algorithm known as a transformer, which is designed to “understand” the context of a sentence and then generate an illustration that retains the important information from the original text.
Google accomplishes the much more challenging task of comprehending the overall context of the entire text, in contrast to the currently popular text-to-image systems, which are based on a “frontal” depiction of the “what I see, I draw about” principle. Additionally, the choice of images is made to create a “picture as a whole” rather than individual statements and phrases (as practically everyone is doing right now). In other words, pictures are chosen for the entire article while taking contexts and meanings into mind.
Google says the model can generate summaries of multiple sentences from a news article and that it has been trained on a large dataset NewsStories, which contains almost 31 million articles, 22 million photographs, and 1 million videos hidden away in the bushes.
In this project, we are taking on a brand-new challenge: creating visual summaries, which are longer texts accompanied by a collection of images. By mastering two MIL (Multiple Instance Learning) subtasks, the aim is to maximize the semantic similarity between each article and the input photos.
The first step is to match the image to the complete article after it has been translated into the proper representations using both the language and image encoders.
The second step entails dividing a text article into separate sentences and encoding each sentence into a different representation. The objective is to optimize the mutual information, defined in terms of probability distributions, between image and text sequences. The greatest precision was obtained as a result.
In conclusion, this study makes a variety of contributions, from applications like automated narrative illustration to the challenging task of matching a story and a collection of illustrations. The company says the model can also be applied to other languages and that it is working on expanding the dataset to include articles from other sites.
Read more related articles:
Disclaimer
Any data, text, or other content on this page is provided as general market information and not as investment advice. Past performance is not necessarily an indicator of future results.
