Over just a few months, ChatGPT went from correctly answering a simple math problem 98% of the time to just 2%
The vastly different results from March to June and between the two models reflect not so much the model’s accuracy in performing specific tasks, but rather the unpredictable effects of changes in one part of the model on others.
“When we are tuning a large language model to improve its performance on certain tasks, that can actually have a lot of unintended consequences, which might actually hurt this model’s performance on other tasks,” Zou said in an interview with Fortune. “There’s all sorts of interesting interdependencies in how the model answers things which can lead to some of the worsening behaviors that we observed.”
The exact nature of these unintended side effects is still poorly understood because researchers and the public alike have no visibility into the models powering ChatGPT. It’s a reality that has only become more acute since OpenAI decided to backtrack on plans to make its code open source in March. “These are black-box models,” Zou says. “So we don’t actually know how the model itself, the neural architectures, or the training data have changed.”
read here