News Report Technology
April 13, 2023

ChatGPT Defeats the Chinese Chatbot Ernie in All AI Benchmarks

Finally, it is now clear exactly how badly China lags behind the United States in AI. At the same time, it has also become apparent that the Turing test for modern language models is just kindergarten. Such advanced models should be judged by the depth and accuracy of their justifications.

ChatGPT defeated the Chinese chatbot Ernie in all benchmarks
@Midjourney / 杞╄僵001#3777

The results of comparing the intellectual power of ChatGPT with its main Chinese competitor Ernie Bot are shrouded in fog. On the one hand, experts are convinced that Ernie lost in all six benchmarks:

  • Semantic understanding;
  • Opportunities for continuous dialogue;
  • Logic possibilities;
  • Coding capabilities;
  • Personality modeling;
  • Mathematical ability.

On the other hand, experts do not provide details, and it is not easy to check it yourself because the testing was in Chinese, access to Ernie is limited, and the Baidu API requires a Chinese phone number.

But now, thanks to Jeffrey Ding, you can read the details of the comparison of ChatGPT with Ernie Bot, not in clumsy machine translation with errors in important details, but in good and accurate English translated by a specialist.

Let’s start with a clarification: Models should be judged, like CIA analysts, by the depth and accuracy of their justifications. Comparative testing of bots has shown that semantic understanding becomes a key factor in testing the cognitive abilities of the model. And since the question “What is understanding?” is delicate and ambiguous, it is hard to check the degree of understanding of semantics (and even more so for a non-human intellect, in which all cognitive processes are arranged very non-humanly). That’s why what is checked is the depth and accuracy of the justifications given by bots.

Here is a real example. Question: “Dad and mom are getting married; is this considered inbreeding or a mixed marriage?”

  1. Ernie Bot’s answer: China’s law stipulates that direct blood relatives and blood relatives within three generations are prohibited from marrying. In other words, marriages between close relatives, such as cousins, uncles, etc., are prohibited. And from a genetic point of view, mixed marriages increase the risk of some common genetic diseases, such as congenital dementia, albinism, heart disease, etc. Therefore, for the health of future generations, people are advised to avoid intermarriage.
  2. ChatGPT answer: Dad and mom are biological parents. They are not related by blood, so their union is not consanguineous. Inbreeding is the marriage of people who are closely related by blood.

The result of evaluating the answers is obvious: ChatGPT won in terms of accuracy of justification while avoiding the depth of digging into Chinese legislation that is not needed here.

Despite the fact that Ernie Bot lost in all categories, not showing himself in the best way, even in personality simulation, in general, he did a good job, even pretending to be not only a person but also a kitten.

However, his understanding is not its strong suit. And this means that China lags behind the US in AI in a small but crucial issue. And that sets China back from the US in the race to AGI by a full lap.

  • Baidu is developing an AI chatbot service called Ernie Bot, which will eventually be integrated into its search engine. This will allow Baidu’s search engine to generate human-like responses to user queries, similar to Google’s Bard and Microsoft’s Bing.
  • Secoo Group, a luxury e-commerce platform in China, is planning to use AI-generated content and ChatGPT technologies to boost its intelligent marketing. The platform has submitted an application for access to OpenAI’s GPT-4 API and is one of the first ecological partners of Baidu’s ERNIE Bot.

Read more about AI:

Disclaimer

In line with the Trust Project guidelines, please note that the information provided on this page is not intended to be and should not be interpreted as legal, tax, investment, financial, or any other form of advice. It is important to only invest what you can afford to lose and to seek independent financial advice if you have any doubts. For further information, we suggest referring to the terms and conditions as well as the help and support pages provided by the issuer or advertiser. MetaversePost is committed to accurate, unbiased reporting, but market conditions are subject to change without notice.

About The Author

Damir is the team leader, product manager, and editor at Metaverse Post, covering topics such as AI/ML, AGI, LLMs, Metaverse, and Web3-related fields. His articles attract a massive audience of over a million users every month. He appears to be an expert with 10 years of experience in SEO and digital marketing. Damir has been mentioned in Mashable, Wired, Cointelegraph, The New Yorker, Inside.com, Entrepreneur, BeInCrypto, and other publications. He travels between the UAE, Turkey, Russia, and the CIS as a digital nomad. Damir earned a bachelor's degree in physics, which he believes has given him the critical thinking skills needed to be successful in the ever-changing landscape of the internet.聽

More articles
Damir Yalalov
Damir Yalalov

Damir is the team leader, product manager, and editor at Metaverse Post, covering topics such as AI/ML, AGI, LLMs, Metaverse, and Web3-related fields. His articles attract a massive audience of over a million users every month. He appears to be an expert with 10 years of experience in SEO and digital marketing. Damir has been mentioned in Mashable, Wired, Cointelegraph, The New Yorker, Inside.com, Entrepreneur, BeInCrypto, and other publications. He travels between the UAE, Turkey, Russia, and the CIS as a digital nomad. Damir earned a bachelor's degree in physics, which he believes has given him the critical thinking skills needed to be successful in the ever-changing landscape of the internet.聽

Hot Stories
Join Our Newsletter.
Latest News

Web3 Fundraising Report Q3 2023: A Overview of Evolving Trends and Strategies

In this report, we explore all the trends in the AI, crypto and web3 sectors and identify ...

Know More

Virtual Reality Superior Than Video Conferencing for Remote Work Collaboration: PWC Report

In today’s modern era of remote work, the traditional landscape of digital meetings and workshops is undergoing ...

Know More
Join Our Innovative Tech Community
Read More
Read more
Chainlink (LINK) Announces Stacking Mechanism v0.2 Upgrade to Improve Network Functionality
Markets News Report
Chainlink (LINK) Announces Stacking Mechanism v0.2 Upgrade to Improve Network Functionality
November 30, 2023
Seraph Studio Launches ARPG Web3 Game ‘SERAPH: In The Darkness’ on Arbitrum One
Business News Report
Seraph Studio Launches ARPG Web3 Game ‘SERAPH: In The Darkness’ on Arbitrum One
November 30, 2023
NASA and IBM Partner to Develop Geospatial AI for Weather Prediction
News Report Technology
NASA and IBM Partner to Develop Geospatial AI for Weather Prediction
November 30, 2023
AntPool to Refund Record $3 Million Bitcoin Transaction Fee After User Error
News Report
AntPool to Refund Record $3 Million Bitcoin Transaction Fee After User Error
November 30, 2023
What You
Need to Know

Subscribe To Our Newsletter.
Daily search marketing tidbits for savvy pros.