-
OpenAI runs an internal effort called “Project Lily,” where contractors read real ChatGPT conversations to help improve the tool.
-
The company removes usernames and uses a privacy filter, but it admits the filter can still miss personal details.
-
Reviewers sometimes see a “user memories summary,” which may include private context, like where a person lives.
OpenAI now faces fresh questions about user privacy. A new report shows that hundreds of contractors read real ChatGPT conversations. This work falls under an internal project called “Project Lily.” The findings raise concerns about how much privacy chatbot users actually get.
How Project Lily actually works
At first, 404 Media uncovered the project through an investigation. According to the outlet, contractors review user prompts and ChatGPT’s replies. Their goal is to help the chatbot improve over time. The review process can cover entire conversations, not just single messages. Some of that material includes sensitive or personal details.
OpenAI strips usernames before contractors see any chats. The company also runs a privacy filter to remove personal information first. However, OpenAI itself admits this filter can miss certain details. Some reviewers even receive a “user memories summary.” This summary can hold information from a user’s past chats. It may include personal context. In some cases, it can even hint at where someone lives.
Contractors judge ChatGPT’s answers using set criteria. They check if the answer follows the user’s request. They also check if the answer stays accurate. Reviewers look out for excessive flattery or answers that feel too humanlike. The project therefore aims to sharpen ChatGPT’s responses. It is not designed to let staff casually browse private chats.
OpenAI’s own privacy documentation backs this up. According to the company’s privacy policy, conversations from services like ChatGPT can train its models. This happens unless a user turns the setting off. Users can disable this through the “Improve the model for everyone” option.
This sits inside the Data Controls settings. OpenAI also states that its privacy filter applies to chats when that setting stays on. The filter aims to cut down personal data before it reaches training material. OpenAI further explains this process in its help center article on chat training data.
Social media reacts strongly
News of Project Lily spread fast online. Many users began questioning how private their chatbot conversations really are.
On X, user @nudevise responded directly to the report. This user urged people to check the privacy settings on every app they install. They suggested turning off any setting that allows data sharing.
Another user, @FosterHeritage, took a more mocking tone. This user joked that Project Lily sounded like a friendly name. Yet it actually describes a system where many strangers can read personal chats.
A separate user, @cintiaramiduart, raised a different point. This user asked how long human reviewers have been reading AI conversations overall. That specific claim does not appear in the 404 Media findings. It should be treated as a separate discussion from the Project Lily report itself.
Other AI companies face similar questions
OpenAI is not the only company doing this kind of review. Anthropic, the company that built Claude, follows a similar pattern. Anthropic confirms it may use chats to improve its models too. This only happens when users allow it.
According to Anthropic’s privacy page, the company removes links to user identities before any review takes place. Anthropic also limits which staff members can access these conversations. Only people involved in model training get that access. The company adds that chats flagged by its safety systems may get reviewed separately. This review supports safety checks rather than general training.
Anthropic explains further in its support article on sensitive data and chat visibility. The article outlines exactly who can view a user’s conversations and under what conditions.
The Project Lily report has reopened a bigger conversation. It centers on how much privacy people should expect from AI chatbots. Human review can help these companies fix mistakes and improve accuracy.
The privacy debate also extends beyond AI platforms, with governments introducing new rules to protect people online. New Jersey, for example, has signed a kids code to strengthen online privacy and safety, adding another layer of protection for younger users.
Still, knowing that real people might read your chats changes how users may behave. Many people will likely think twice before typing sensitive details into any AI tool. The debate over AI privacy is far from finished. As more users learn how their chats get used, companies may face growing pressure. That pressure could push them toward clearer rules and stronger user controls.
