In Brief
Ilya Pestov, a Russian AI researcher, created the logical thinking test, which was passed by 12 thousand people.
He recently obtained access to the smarter GPT-4, and conducted an experiment to see if the right query could yield some results.
The results showed that the GPT-4 outperformed the average person in logical reasoning.
The Trust Project is a worldwide group of news organizations working to establish transparency standards.
Ilya Pestov, a well-known Russian AI researcher, posted a message on his Telegram channel about how well the neural network handles logical tests. Ilya once created the @psylogicbot logical thinking test, which was taken by approximately 12 thousand people. You can check out the stats after taking the test.
| Read more: 20+ Best Telegram AI Chatbots of 2023 |
He wrote that ChatGPT also got tested, but the results left a lot to be desired. He recently got access to the smarter and more updated version of the GPT model—GPT-4—and decided to check whether it would get similar results.
The experiment was conducted as follows: The researcher created a text that described the task that the neural network had to complete. The researcher posted everything in the comments: The prompt was: “I’ll give you a logic puzzle and four possible answers; choose the one correct answer from them.” Then, for each test question, Ilya created a new dialog and sent the GPT-4 description along with the question text. The bot received the response without any corrections or hints.
There are 25 questions in total, with one point awarded for each correct answer. According to statistics, users score 13.6 points on average, with a median of no more than 14. How much did GPT-4 get? It managed to get 16 points!
Once again, the neural network outperforms the average person in logical reasoning. That is, it outperforms the majority of the people tested. And this is after taking into account:
- The test was conducted in Russian, while the model is fine-tuned for English;
- GPT-4, which is used in chat, is less intelligent than its predecessor (a side effect of ethical restrictions).
Separately, we will post an excellent answer to question 22, in which the neuron used first-order logic to derive the result mathematically. While this was covered in applied mathematics, it is not a university course everyone takes.
Still believe that neural networks are a fad? First, try to outperform GPT-4 (and share your results in the comments).
Read more about AI:
Disclaimer
Any data, text, or other content on this page is provided as general market information and not as investment advice. Past performance is not necessarily an indicator of future results.
