Google’s new Artificial Intelligence (AI) chatbot, Bard has faced quite a few hiccups since it’s release in late March, though the AI model is still in it’s early stages.
Google CEO Sundar Pichai said in a letter to his staff that “As more people start to use Bard and test its capabilities, they’ll surprise us. Things will go wrong.”
‘Fortune’ magazine recently tested the Chatbot’s effectiveness and knowledge by subjecting it to the SAT examination, widely used for college admissions in the United States. The exam aims to test knowledge in mathematics, reading and writing. According to ‘Fortune’, Bard AI answered anywhere between 50 to 75 percentage of questions incorrectly even with multiple choice questions in Mathematics.
As for writing, Bard returned 30 percent of correct answers and even while giving incorrect responses, made them confidently, like “The correct answer is-“, which is a common feature of AI language. In comprehension, Bard maintained a 50 percent success rate. Overall, Bard scored 1200 points, which would place a student in the likes of Howard University, Michigan State University and San Diego State University.
As for Bard’s primary competitor, CHATGPT AI (GPT-4), “scored better than 90% of humans” with a score of 93 and 89 percentile in reading and mathematics respectively.
OpenAI announced that GPT-4 exhibits “human- level performance” on “a number of professional and academic benchmarks”.
Faced with the recent setbacks and falling short of it’s competition, Sundar Pichai said in a podcast, “We clearly have more capable models. Pretty soon, we will be upgrading Bard to some of our more capable Pathways Language Model (PaLM) models, which will bring more capabilities; be it in reasoning, coding, it can answer math questions better,”. Google has also been fast to deny accusations claiming that they use Microsoft-owned OpenAI’s ChatGPT to train Bard AI.
Pichai also said that part of the reason for the slow development of Bard is a sense of caution within the company. He told The New York Times in a podcast that “AI is the most profound technology that human beings will ever work on” and that “AI will get to the essence of what humanity really is”.
Since Bard AI is an “experiment” and because Google has been pretty vocal about this aspect, parent company Alphabet Inc. and Google have ‘wiggle-room’ to improve upon the existing flaws of Bard AI.
“It was an experiment. We tried to prime users to its creative collaborative queries, but people do a variety of things. I think it was slightly lost. We did say we are using a lightweight and efficient version of LaMDA. So in some ways, we put out one of our smaller models out there, what’s powering Bard. And we were careful,” he noted.
