Highlights
OpenAI Data Practices and Allegations Regarding GPT-4o
OpenAI is facing scrutiny once again due to claims regarding its data practices. Recent allegations indicate that the company may have trained its latest model, GPT-4o, on copyrighted and paywalled content without the necessary permissions. These claims have emerged from the AI Disclosures Project, a non-profit organisation focused on monitoring AI activities, established in 2024 by media leader Tim O’Reilly and economist Ilan Strauss.
The AI Disclosures Project has released a study that raises significant concerns by asserting that OpenAI’s GPT-4o model shows a considerable recognition of copyrighted books from O’Reilly Media, despite a lack of a licensing agreement between OpenAI and the publisher. The report indicates that GPT-4o exhibits markedly greater recognition of paywalled O’Reilly book content compared to previous models such as GPT-3.5 Turbo.
The research employed a technique referred to as “membership inference attack” or DE-COP, designed to ascertain if the model could accurately differentiate between texts authored by humans and those generated through AI paraphrasing. If an AI model can effectively tell the difference, it suggests that it may have access to the original text, indicating that such content was likely included in its training data. The study analysed 13,962 paragraph excerpts from 34 O’Reilly publications, concluding that GPT-4o identified more paywalled content than GPT-3.5 Turbo, achieving an AUROC score of 82%, whereas the latter only surpassed 50%.
In spite of the striking results, the co-authors, including AI researcher Sruly Rosenblat, conceded that there could be limitations in their approach. They noted the possibility that users might have copied and pasted paywalled segments into ChatGPT, potentially leading to indirect inclusion of the content. Additionally, the study did not assess OpenAI’s latest models, such as GPT-4.5 and the reasoning models o3-mini and o1, which leaves uncertainties regarding whether these models have similar data sets.
A Broader Industry Issue
The findings of this report contribute to the mounting legal challenges faced by OpenAI, as the company confronts several lawsuits alleging copyright infringement and misappropriation of data. OpenAI and other prominent AI organisations have been pushing for more lenient regulations on the use of copyrighted materials for model training, contending that these practices should qualify under fair use. It is noteworthy that OpenAI has already established licensing agreements with news publishers, social media platforms, and stock media libraries to secure necessary data. The company has also recruited journalists to refine the outputs of its models.
The AI Disclosures Project underscores a larger systemic issue which could affect the quality and diversity of online content. The study contends that utilizing copyrighted data without remuneration could hinder income for professional content creators, thereby threatening the variety of available content on the internet. The project calls for enhanced accountability and transparency in the training processes of AI firms and advocates for policies ensuring that content creators are compensated when their data is utilised.
While OpenAI remains steadfast in defending its data practices, the findings from the AI Disclosures Project have undoubtedly escalated the discourse surrounding copyright and data ethics in the swiftly evolving AI sector. As legal disputes persist, the challenge of reconciling innovation with intellectual property rights remains an open question.
