The Limitations of AI: Why It Doesn’t Work Properly Enough

MetaversePlanet August 9, 2023Last Updated: February 13, 2024

0 1 minute read

Artificial intelligence, today’s buzzword, has encountered a stumbling block in programming. Consequently, numerous companies have reevaluated their approach to its adoption.

Undeniably at the forefront of contemporary technological progress, artificial intelligence nonetheless comes with its challenges. Recent studies have highlighted a few significant drawbacks.

Programming: AI’s Achilles Heel?

The Limitations of AI: Why It Doesn't Work Properly Enough

A study led by Purdue University took an in-depth look at the AI tool, ChatGPT, and made a surprising discovery: the tool provided incorrect answers to more than half of the software programming questions asked. With an effectiveness rate of only forty-eight percent in the realm of programming, this highlights the need for a reassessment of its use in this field.

In the study, researchers submitted 517 common questions from Stack Overflow to ChatGPT. The responses were then evaluated by a group of volunteer developers. The outcome? Only forty-eight percent of ChatGPT’s answers were accurate, while a significant seventy-seven percent of its responses were found to be overly verbose.

Despite the inaccuracy of its answers, many participants appeared willing to utilize the AI’s responses. The tool’s ‘diplomatic’ nature and its ability to articulate responses eloquently contributed to users incorporating its answers into projects, regardless of their precision.

However, this tendency could lead to considerable issues in large-scale projects over time. Consequently, tech giants such as Apple, Google, and Meta have decided to restrict developers from using artificial intelligence in their work.