The results have been expected for weeks, although until now OpenAI had not provided details about which problems the model had solved or exactly when they would be published. In September, the company said its model had “resolved more than 100 long-standing open problems across most areas of mathematics.” The released papers and other details also include some summaries of the model’s reasoning, estimates of the compute used, and stats about the number of problems attempted. OpenAI claims the “average result” used the equivalent of three hours of ChatGPT Pro thinking.
AGMAI, or the Advisory Group on Mathematics and Artificial Intelligence, published its first recommendations in late September, urging AI labs to release mathematical results promptly and through established academic channels where possible, while disclosing details such as the name of the model used, prompts, and compute costs. The group also implored AI companies to “refrain from treating the release of mathematical results as marketing vehicles to promote their models,” a practice they said inflicts significant harm on the mathematical community.
Here’s how OpenAI described its process for this release:
For this release, we’re publishing the results in a GitHub repository, with protocols for paper revisions and citations. We’re continuing to explore other community-hosted alternatives for this release which meet the committee’s guidelines. For future releases, we are committed to further improving the quality of the papers via the citations, mathematical exposition, and presentation of the results for better understanding
