Google’s AI chatbot Bard has been the talk of the town since its release, but not necessarily for the right reasons. The chatbot, which was showcased in a promotional video at a company event in early February, caused Alphabet to suffer a loss of $100 billion in market value due to sharing inaccurate information. Despite this setback, Sundar Pichai, the CEO of the tech giant, remains unfazed about the extent to which their artificial intelligence models need to advance. In a letter addressed to the entire staff, Pichai acknowledged that Bard is still in its early developmental stages and stated, “As more individuals begin using Bard and testing its capabilities, they will surprise us. Things will go awry.”
Since its release, Bard has transitioned from being exclusively used by Google employees to being tested by the general public. Fortune, a prominent business magazine, recently tested the AI chatbot’s proficiency in advance of the upcoming SATs, a standardized test that assesses students’ skills in reading, writing, and mathematics and it is widely used in the United States for college admissions.
However, when Fortune’s reporters logged into Bard, a message appeared on the screen, warning users that the chatbot may provide inaccurate or inappropriate responses and advising them to double-check Bard’s responses using the “Google it” button. The message continued, saying that Bard will improve with feedback, and users should rate responses and flag any that may be offensive or unsafe.
Fortune obtained sample SAT math questions from internet study materials and discovered that Bard answered only 50 to 75 percent of them correctly, even when multiple-choice solutions were provided. Moreover, when the same issue was asked again, Bard offered solutions that weren’t even among the available choices.
For instance, Bard responded that the James Webb Space Telescope had taken “the very first photos of a planet outside of our own solar system” when asked to explain to a nine-year-old what it had discovered. According to NASA, a ground-based array called the Very Large Telescope in Chile was the first to capture an image of an exoplanet in 2004 and recognise it as such in 2005.
In a reading test, Bard fared slightly better than it did in math, getting approximately half of the answers correct. However, the chatbot’s first written language test with Fortune resulted in only 30 percent of the answers being correct, and Bard often needed the questions to be repeated before understanding them.
Despite the incorrect responses, Bard maintains a confident tone, frequently framing responses as “The correct answer is,” a common trait of large language models, as stated by Fortune.
All in all, Bard received a score of 1200, which would qualify a student for admission to universities such as Howard University, San Diego State University, and Michigan State University. When contacted for comment, a Google spokesperson informed Fortune that Bard is still in the experimental phase, and users should double-check the information provided by the chatbot. The spokesperson added that with feedback, Bard is improving every day, and thousands of testers were involved before its public launch to provide feedback to enhance its quality, safety, and accuracy.
The development of Bard is a testament to Google’s continuous pursuit of cutting-edge technology. However, it also highlights the limitations of artificial intelligence models and emphasizes the importance of human intervention in ensuring their accuracy and safety. As AI continues to advance and becomes increasingly integrated into our daily lives, it is crucial that its development is regulated to prevent harm to individuals and society as a whole.For instance, Bard stated that the James Webb Space Telescope had taken “the very first photos of a planet outside of our own solar system” when asked to explain to a nine-year-old what it had discovered. According to NASA, a ground-based array called the Very Large Telescope in Chile was the first to capture an image of an exoplanet in 2004 and recognise it as such in 2005.
