The Illusion of AI Authority: How ChatGPT Repeatedly Misinterprets the Bosnian Pyramids

Abstract

The emergence of large language models like ChatGPT has transformed how the public and scholars interact with scientific knowledge. This study evaluates the consistency, accuracy and epistemological behavior of two generations of ChatGPT—version 3.5 (May 2023) and GPT-4o (May 2025) in response to a scientifically grounded, yet publicly controversial topic: the Bosnian Pyramid of the Sun. Despite improvements in language generation, both AI versions repeated unsupported claims, provided contradictory geological analogies and failed to acknowledge peer-reviewed evidence regarding the pyramid’s geometric precision and cardinal alignment. The comparison reveals limitations in machine learning's ability to meaningfully “learn” from expert input or correct misinformation over time. While GPT-4o showed a more refined tone and a greater willingness to concede error, it ultimately echoed the same core biases as its predecessor. These findings raise concerns about the use of AI as an authority in controversial or emerging scientific domains.