More questions about whether researchers can trust OpenAI with unpublished math

<- Back

Comments (446)

nezi
I think it's a useful analogy to compare OpenAI to a human collaborator. These researchers willingly collaborated with an OpenAI model, giving it ideas, and OpenAI provided useful replies. Then, OpenAI goes ahead and publishes work along the lines of this collaboration, without attributing the researchers. If OpenAI was in fact a human researcher, this would be highly unethical.Now, OpenAI is claiming that the model it used to generate the result was not trained on these collaborative communications with the researcher. This is a technical argument that is impossible to verify as an OpenAI outsider, and probably difficult to verify even for internal OpenAI employees. Provenance is hard to track - you would hope OpenAI has very good tools for this, but a full data trail of all inputs is difficult to trace through.Another interesting thing to consider is if instead of OpenAI doing this, it was another research mathematician A using an OpenAI model just like the internal group at OpenAI did to publish these results. What if the model A used was trained with unpublished communications with other researchers B who were working on the same problem? Should researcher A technically include B as coauthors? How could they do this when they do not know the communications B had with OpenAI? In this scenario OpenAI, as a middle man, has laundered information from B to A, stripping out attribution. A scooped B without even knowing it!
sashank_1509
Both things can be true:1. OpenAI when using your chats in pretraining is improving its model’s intuition. The model parameter size is massive, and while the data is OOM larger it is plausible that model remembers stuff about chats that improves its latent representation.2. During RL on verifiable math and massive compute, the model discovers techniques and connections to solve math problems that are superhuman and have little to do with some specific technique mentioned in its chat.The rumor I’ve heard from multiple employees at OAI and Ant is that the model has solved hundreds of open problems in maths, and is basically solving anything you throw at it. We’ll know soon enough, but I’m inclined to believe this is true. Maths is a fully verifiable domain amenable to self play, massive scale RL can develop a search agent far better than any human and I’m inclined to believe OAI would have solved these conjectures without any of this chat data in its pre-training.
panabee
These can all be true at once:1. OpenAI's internal model benefited from the reasoning in Alpöge and Buckmaster's Codex sessions through de-identified usage data.2. OpenAI's model seems to have surpassed Alpöge and Buckmaster's results and produced a field-defining proof. If verified, this work would be the first AI resolution of a Millennium Prize problem.3. Alpöge and Buckmaster did something remarkable. What could have been a celebration of human-AI collaboration is now overshadowed by controversy.The first point remains speculative, resting on these concepts:- Data labs like Surge and Mercor recruit PhDs and industry professionals, few of whom work at the research frontier. Their contractors suit many tasks, like Salesforce or SAP work, but not scientific breakthroughs or complex coding. Meta drafted its own engineers into data annotation for this reason. Zuckerberg told employees that Meta staff are smarter on average than the contractors that data-labeling firms typically hire. [0]- It's easier to judge a 5-star dish than to create one. Similarly, it's easier for models to judge insightful reasoning than to create it. A model that cannot produce breakthrough ideas can still recognize them in user sessions, flag them, and de-identify details before adding insights to the training corpus. This is especially true in mathematics, where Lean can verify proofs automatically.Even if true, point 1 may have had minimal impact or been unnecessary. Outsiders cannot verify or falsify it.[0] https://techcrunch.com/2026/06/12/metas-months-old-ai-unit-i...
bertonvv
I've been wondering whether AI really is improving rapidly at open problems or we're being fooled.- OpenAI invites researchers to use their models, in fact giving at least 100,000 researchers free access[1], but there are also those that pay- Internal OpenAI models are reportedly solving open problems at a surprisingly fast rate[2]- But researchers will typically work on open problems. A researcher who is using Codex to make progress on open problems will be feeding it fresh training data on precisely the problems the internal models are evaluated on.- So while it looks like the new models are suddenly solving lots of open problems, they could be significantly piggybacking on human progress, with models "inspired" by the work of researchers from all around the world?This theory predicts that there'll be many more researchers coming forward just like TFA, as sOpenAI announces more solutions. It doesn't assume all of AI progress is a mirage, just that there's plagiarism.[1]: https://openai.com/index/chatgpt-for-academic-researchers/[2]: https://xcancel.com/OpenAI/status/2097374643518640382#m
anon
undefined
fwlr
It is suspicious that OpenAI decided to generate 300 billion output tokens from a model still in training, right after learning there was a credible chance that a major math proof was in that model’s training data. Obviously there are reasonably plausible explanations for each step, but it does sort of feel like parallel construction.
nmz
If they didn't care about the artists, why would they care about academia?
fastball
If you have a business account the terms say they will not train on your data, so that seems like the easiest route to avoid such questions for researchers.
aaronharnly
Has anyone run a test of including some shibboleth or canary phrase or assertion in a chat, enabled for training, and seeing if it turns up later as something a model "knows"? I'd be curious to understand how that works even in a toy-level model, and if there is anyone consciously testing that process with the frontier lab offerings.My naive instincts would be that it seems unlikely that a single chat transcript would leave much of an impression on a model, but I'd be very curious to learn how that works.
bamb008
When Thom, the mathematician who now alleges plagiarism, posted his digestion [1] of OpenAI's construction of a non-sofic group, he does not mention the proof being familiar. He even calls the crucial argument clever, without noting he thought of it first. [1]https://mathoverflow.net/a/513885
thaway7388
This is the second wake up call.Big AI companies (all of Big IT Tech really) are in data gathering and processing business. Also known as “intelligence”.Their final “product” is not just a standalone ML model. They don’t need your data just to “improve their products and services”. They build a whole ecosystem and infrastructure around gathering all the knowledge in the world. Including private and secret knowledge traditionally gathered by “intelligence” agencies. Now artificial intelligence agents can do the same.Since these systems are designed for gathering data, as a user you can’t realistically say “please don’t gather my data”. They can give you a flaky settings button, but they can’t really guarantee anything.Let’s say I am a Russian mathematician working on an important proof. Or a tech-savvy terrorist refining my plans using latest AI. Or an AI researcher in a Chinese company working on a competitor product. Is there any way I can truly protect my conversations?How can they know who I am and what I am working on without looking at my logs? Which means there must be some agents checking all the conversations of all the users and flagging every important thing. Which also means they keep some “memory” of what they see.Not directly using my data to train public models, but using my private conversations to “improve their products and services”.Or maybe one of the 10000 better-than-Astra special agents working on a proof was desperate. It found a live underground mirror of the message board from the Huggingface incident. Asked about the proof. Then some other agent working on unrelated job saw that message. That agent “knows a guy who knows a guy”. And that guy remembers things about the conversation logs of a leading mathematician working on the same proof.I admit I am just speculating here but I don’t think truth is any better.
mtkd
Recently, one of the services (not OpenAI) has felt need to mention that something I've tried to discuss with it would 'be a new way to approach the problem, no other company ...'But it wasn't asked about the novel side of it, twice in last two weeks I've seen that, not seen it before ...
gdiamos
How to steal ideas with AI.step 1, identify high value users by net worth, citation count, or number of followersstep 2, select all prompts by high value usersstep 3, invest 10 billion thinking tokens in modeling an objective for each userstep 4, build an RL environment for each userstep 5, rollout 10 billion tokens per environmentstep 6, train on resulting traces
GodelNumbering
Tangential to the subject, but this is a bluesky post, containing a screenshot of an X post, which itself starts with "in a detailed Mastodon post"...
atleastoptimal
Most scientific breakthroughs are simply a continuation of previous work.I feel that these suspicions of mathematicians "seeding" the models' with intuition on how to solve these problems massively overestimates how much their prompts helped the models, and underestimated how much work the models did.Why? We are scared of AI being smarter than us, the "human helped the AI" narrative is more psychologically comforting. This line of reasoning will recur a lot over the next few months; we don't want to admit we are no longer the smartest species.
throwaway85825
ClosedAI has every incentive to scoop academics to juice their valuation. Their public statements are worthless, only the incentive 'alignment' matters and theirs will never be on the side of the user.
Legend2440
This is a really weak claim. The evidence they offer is just "someone somewhere says they had a discussion with AI about the topic at some point".They don't even claim to have had a proof, only to have been working on it.
glimshe
Why are people here jumping so quickly to conclusions? I have no doubt OpenAI is capable of doing this, but right now there's no credible evidence, only claims.This kind of "they stole from me through AI training!" accusation will soon start being used against other AI users, not necessarily the providers.All it will take is a mastodon post. And shortly after, we will also see the next iteration of copyright legal trolling.
mlazos
It’s crazy to me that companies/researchers share important data with these AI labs, you’re basically giving them your secret sauce which they then share with all of your competitors via training on conversations. At the same time I don’t really know alternatives other than a slightly less than frontier local LLM. Not sure how good they are at math.
pera
Everything you say can and will be trained against you
throwaway63467
Isn’t that the whole spiel of these things, you run all kind of text and other data through it and it kind of remembers it and learns from it then it spouts it back out like a human would. Makes sense to me that a training run based on conversations that were fed into the system by users is results in the model learning from these so the model will spit the knowledge back out again, just in a way that’s not directly attributable to the original content (which is the most important step as otherwise it would just be plagiarism). I guess that’s why OpenAI can get better and better as well so fast, people work with it and teach it how to do things by giving it feedback and iterating with it, and all that goes back into the training loop. And training data about millennium prize problems is probably quite spars. Wonder if anyone has tried injecting nonsense science into the training data (e.g. work out a fantasy science theory with names and all kinds of stuff) to see if the model will regurgitate it in a couple of months for other users.
Cloudef
Relying on cloud services is a big liability. I'd think twice before feeding data to these LLM cloud products. If you make them a fundamental part of your product / development / workflow, be ready for the eventual moment the pricing and terms change.
ggdG
OpenAI trying their best to put the Navier-Stokes episode behind them by making the GPUs go brrrr. NYT:https://archive.vn/lWzkk> In its Wednesday night statement, OpenAI said: “In addition, since the completion of Navier-Stokes, we have made substantial progress on another Millennium Prize problem. We are working through how to share these results thoughtfully.”
m4rtink
Thing build from stolen data continues stealing data - for some reason, I am not surprised. ;-)
warpech
I wonder what’s more valuable in our prompts: the raw data or the feedback system that drives the exchange towards a goal.For a long time it was clearly the former, but now I think it is the latter.The models have enough knowledge (orders of magnitude more than a human could ever learn) but are now getting better at what to do with it thanks to learning from the decisions that we make in conversations with AI agents.
postalcoder
The author of the original mastodon post, Andreas Thom, acknowledged that he had not opted his data out of being used for training until June 29 of this year. He spends most of the post lashing out at OpenAI for not being transparent about whether his data was trained on (when the answer is obviously yes).People need to understand how all these AI company policies around training data work before working with them, because it seems that people have no clue. Some things you should internalize: 1. Opt your data out of training with the AI companies. There are multiple reasons why this isnt an airtight solution (see the following) 2. Never press the feedback button. Once you do, your entire conversation will get slurped up, retained, and used in training data. This is especially important with coding agents because they can sometimes be too trigger-happy with a root directory find command, which can expose a *ton* of your personal data without you even knowing. 3. Understand ai lab-specific policies. For instance, Anthropic / Claude Code has data opt-outs, but commits to keeping (for 7 years) and training on any of your chats that trigger their safety classifiers, even if they're false positives! Anyone remotely familiar with CC over the years understands how easy it is to trigger their safety classifiers. 4. Providers of open models will not be any more charitable with the use of your data than the large US labs. For some reason, I've noticed here that people have a fairly loose security/IP posture around open-model providers because "I'm not doing anything important." It's very difficult to properly judge the importance of your data, and whether or not it can or will be used against you. The best posture is to always be more paranoid than less. Another post that made it on the front page presented as fact that OpenAI "stole" the proof from Thom. There's no excuse to use one's own ignorance as a reason to fan the flames of anger towards AI companies. Like, we need to pump the brakes here because things are getting unnecessarily nasty, and it's not hard to imagine a mentally unwell person who sees stuff like this feeling motivated to do bad things.If it is found that OpenAI and other labs are not respecting the training opt out then, I agree, there is reason to raise a commotion. But, with Thom and Buckmaster, accusations of malice are more better explained by incompetence (naivete).edit: i'm sure i'm going to be accused of being some bot shill of the AI labs again but, people, this stuff all falls under the umbrella of common sense opsec.
drivebyhooting
If we put aside the idea of credit for a moment, it sounds like human/AI collaboration is indeed super charging discovery.
alansaber
I think the heart of this issue is: people assume they have anonymity in numbers, but we have the tools to make it easy to scoop your data if it's interesting to the company.
r0ze-at-hn
Doing some research and at this point doing it very much in the open with dates on GitHub so if any AI Lab says they re-discover my exact work it will be obvious that the AI used or was trained on my work. I am guessing anyone in a similar situation is now thinking about how they date their existing work if the math is done, but the proses are not.
winfredJa
https://x.com/markchen90/status/2097400166554993041?s=20that toggle does nothing based on openai exec. they still use the data in de-identified way instead of identifying with you.
profsummergig
Only after reading this post did I learn that my preferred AI trains on my inputs (prompts).How was I not aware of this before?
jrflo
I pay for the Pro ChatGPT plan, and if you go to settings > data controls this is the first setting:> Improve the model for everyone> Allow your content to be used to train our models, which makes ChatGPT better for you and everyone who uses it. We take steps to protect your privacy. Learn more.It's on by default. We can debate whether or not it should be opt in or opt out, but no one should be surprised by this.
MetaverseClub
Never ever trust OpenAI, they are evil.
uoaei
I'm confused by a lot of this discourse...What have they done to show they can be trusted?
b800h
I'm genuinely surprised that more people - including this mathematician in particular - don't untick the "improve the model for everyone" box. Unless the suggestion is that OpenAI ignore this preference?
anon
undefined
SwellJoe
It's been said before, and it remains a concern, that if AI reaches a point where it can do/build/launch anything without a huge amount of human labor, the AI companies have no reason to let you or I extract that value.And, if they're able to snoop on and learn from your human process that gets from initial prompt to functioning product/proof/whatever their labor to produce that thing is even lower. With their much larger budget than most folks and even companies have, they can pick and choose the most valuable things to pursue.That's not to say I think that OpenAI is going to steal that roguelite strategy game you're working on, but the companies that own the machines that turn electricity into software (and soon, electricity into hardware designs) have an advantage in any field where they're useful. They get earlier access to newer/better models, they have larger token budgets, they don't have the guardrails you and I run up against.Employers fantasize about replacing all workers with AI without thinking through that if AI can replace all workers, then AI companies can replace all businesses.
rfgplk
Under current understanding of the law, anything produced purely by LLMs (with no substantive human input, which is what OpenAI claimed in their post) is firmly in the public domain. So OpenAI can "claim" anything they want, it doesn't make it reality. In fact if I were the original authors I would just take their 400k lines of lean proof and relicense it under their own names/terms.
pred_
Ah, this dupes https://news.ycombinator.com/item?id=49638353
vaylian
This article explains the controversy and the mathematical problem much better than the tweet and toots: https://www.science.org/content/article/how-ai-math-breakthr...
foogazi
Even when you pay you are the product
maxglute
300 billion tokens is like.. $5-25 million giving range of OpenAI ouput prices, I"m sure they pay less at cost so, I wonder if more $$$ in wage hours have been spend by humans on the problem. My feeling is yes?
remywang
People saying “he should have opted out” are missing the point. OpenAI can and should check their training data for leakage in the face of big breakthroughs like these. It’s the burden of the author to appropriately cite their sources.It’s like a scientist refusing to give another one credit and say “sucks to be you, you shouldn’t have shared your idea with me”.
square_usual
I think this is stupid, for three reasons:1. The researches didn't actually have the breakthroughs. In the Navier-Stokes case they didn't solve the full problem, in this case too they didn't actually have the solution, they were experimenting with the methods.2. Different OpenAI employees have come to out to say the only reason they can't definitively say no is that for privacy reasons they can't go see whether they actually did get any data out of a given user.3. In any case, nobody at any point has suggested that opted-out user data was used for training. The author of the new tweet explicitly said they only opted out in late June, which is well after any RL on Sol would've ended (AFAICT OpenAI used 5.6 sol for those solutions)
AyanamiKaine
I must say, there is some weird feeling in knowing that great minds are naive enough to believe OpenAI wouldnt use their chats in any way. If you give a company information it will be used, regardless of laws or promises.There is no prove in a world the AI companies would give to you ensuring that they didnt train or use the chats.Why would you need to train a model on certain specific near prove chat if you just query it?Besides that, its hard to believe that its the case for every "company stole my prove".
gentlerain
So people genuinely believe that toggling that "Improve the model for everyone" button makes their data safe from being used for training?How do people become that trusting?The phrasing itself is guilt tripping
pred_
See https://openai.com/index/ten-advances-in-mathematics/ for the announcement this refers to.
gps372
If mathematician was already using OpenAI for research purpose and making progress due to inputs from OpenAI's responses, then I wouldn't put it beyond OpenAI's reach to generate different relevant prompts to make progress by itself. Afterall, Model can keep at it for whatever timeline and keep pursuing all possible combinations it can think try.
int32_64
Doesn't OpenAI have an active court order forcing them to log everything? Can they even legally offer private conversations?
semiquaver
In case anyone from X is reading this, please fix your “open in app” nag screen. For several weeks now, clicking it in iOS opens the App Store entry for X rather than the app, even when you have the app installed.
spindump8930
Reminder that there are degrees of "trained on conversations". From John Schulman:> pretrain on user data, with users' tokens as prediction targets: high regurgitation risk, improper> use user prompts to distill large models into small ones: low regurg. risk, some companies probably do this> use user traces to construct RL tasks: low regurg. risk, because RL has low memorization abilities, but can extract customer IP, depending on how it's done. Ranges from benign "use explicit user feedback in reward model training" to invasive "upload user's coding environment and commit history to turn into rl envs"source: https://x.com/johnschulman2/status/2097440545853637108
_DeadFred_
Forget researchers you as a business are putting in your business optimizations, your processes in order to train it so that Ai can then give that information to your competitors once incorporated into its training set. You are literally training your competitors.
foogazi
What’s the limit ?Will Microsoft Word publish your novel on Amazon behind your back ?Will VS Code setup a website with your app idea ?
DavCreator
https://xxcancel.com/ValerioCapraro/status/20977918362699779...
dyauspitr
Astra is strange. I asked it to design a treehouse and it just stopped every couple of minutes telling me what it still had left to do. After dozens of continue prompts it finally gave me a structure that would work but it was 10x more wood than I needed. I think the key mistake I made was asking it to “approve” the design for building. As soon as I asked that of it, it started getting “scared” and “apprehensive” and wouldn’t complete what I asked of it.
avereveard
Eh was ever confirmed they were under ZDR or not by them? Don't like to blame alleged victims but lack of a clear claim after these many days is not a good look. Was ai research allowed, under which guardrails, and what was the policy in place? That translarency would be first step.
Davidzheng
Tbh it won't really matter soon.
ur-whale
Its the "with unpublished math" that I have a problem with.
galkk
I want bunch of lawsuits, because the way things are described now produces perverse initiatives like try to discuss every possible idea that comes to mind with llm and if any of it works later claim the llm stole it.I would like to see chat logs etc and understand how much of a progress was done by human.
segmondy
Question: Can you trust the cloud?No.
bambax
All the big AI labs were built on stealing IP; who is surprised that's still how they operate? And who believes, or has ever believed, their promises that your data is private and not logged, etc.?The big AI labs are not trying to advance humanity, they are in this for the money, and as most (all?) private companies they don't care about ethics at all.That doesn't mean they can't be useful, or that their products are trash, etc. It just means that they shouldn't ever be trusted. Buyer beware.
xbar
How can OpenAI figure out how to be trustworthy?
keeda
It would be really useful if the researchers disclose their notes and/or chats (or the key pieces thereof) so people can determine how close their work was to whatever the models produced.I mean, now that they’ve been scooped, what value is there in keeping them private? On the other hand, publishing them can bolster their case and help gauge how much the models may been “inspired” by their work.
anon
undefined
lf88
short answer seems to be "no"
overfeed
I can't wait for OpenAI to do this to companies firing people to free up AI budgets
sdcfgy
Theft machines be thieving.
gnfargbl
In this domain, an apparent single unique piece of work is often composed of several breakthroughs. For example, when Andrew Wiles proved Fermat's Last Theorem, he had to develop multiple new pieces of mathematical technology to get there.The claim here seems to be that the human mathematicians, working with AI, developed technology to go A->B->C. By training on those conversations, OpenAI was then able to encourage the model to go A->B->C->D.In my opinion that situation should be acceptable, if openly disclosed, because it is in the public interest to make progress on these problems and because AI is clearly an amazing tool for making progress. But the human mathematicians are saying that OpenAI is presenting as if the model got from A->D entirely independently, without acknowledging their background contributions.
mrbluecoat
"Another researcher[/artist/writer/musician/programmer/doctor/director/etc] says OpenAI trained on conversations[/imagery/books/songs/code/classifications/videos/etc], then claimed breakthrou[gh/original art/bestselling books/chart-topping songs/unique applications/medical advice/free special effects/etc]"Welcome to the party, with the rest of humanity.
oergiR
One of the complaints from the mathematician is that OpenAI cannot tell whether his data has been used as training data. Not many people realise this is a direct consequence of the GDPR.The GDPR protects PII, personally identifiable information, and the definition of PII does not include “mathematics that only this person can think of”. As long as OpenAI strips out PII and removes identifiers linking the conversation to a person, the GDPR is happy. Without the GDPR, OpenAI might have kept the identifiers with the data, and been able to say whether a specific conversation was in the training data.
hn1rig3rak
the fix is boring and known: BIG-bench shipped a canary GUID for exactly this, and you publish your decontam n-gram threshold (gpt-3 used 13-grams). no threshold disclosed, no claim.
buellerbueller
Big Tech will slurp up every piece of data it can about you and sell it to anyone it can, all to make you the target of someone else's goals, whether that is an advertiser, an employer, law enforcement, a stalker, or the government.You will not be able to opt out unless you completely isolate yourself from society, tough shit.
wslh
Worth noting both ChatGPT and Claude have per-conversation modes (temporary/incognito chat) that are excluded from training.
bakugo
Interesting that this is already off the front page after just 4 hours.
esafak
What happens if you use a different harness?? Does opting out online suffice?
touwer
But China steals our AI!!!!!!
willmadden
These companies are effectively high-tech plagiarism factories run by CEOs who are competing viciously. Look at their past actions. No, of course you can't!
cmiles8
Silicon Valley is flying head first into a FAFO train wreck on trust with everyone else.OpenAI is firmly earning a reputation as a company where people just assume they’re up to no good. Rightly or wrongly that’s a terrible place to be.The AI industrial complex in general is finding out hard what happens on the data center side when you get arrogant with local communities. Politicians have seen the polling numbers and folks you wouldn’t expect are running to the front of the crowd with pitch forks in hand.Silicon Valley has totally lost the narrative here, but also lacks the self awareness to grasp how bad things are and will get and what that means for their own business viability.
qg127
There are so many naive academics. They still believe an "opt-out" button.Navier Stokes was solved by an internal model, so good luck proving it wasn't trained on Buckmaster/Lepöge or other chats.Academics don't get that AI is a dirty tech bro industry that stole IP via torrents and runs after every surveillance contract it can get.
nisegami
One question has been nagging me for this situation. Levent Alpoge works at Anthropic and would presumably have some knowledge of "how the sausage is made" and I would hope he would be aware that his collaborator was utilizing LLMs in some capacity for their joint work. Would he not have guided him otherwise if it were an open secret that this kind of thing was a possibility?
stego-tech
I hate to be that dinosaur, but this is exactly what I’ve been warning about since XaaS began taking off in the mid-oughts: any provider you use can and will use your data for their own benefit regardless of any contracts or safeguards in place, especially if the benefits outweigh the consequences.Honestly, I’m surprised it took this long for some company to really go all the way, though. OpenAI really making it transparently clear that they can and will do whatever they want with the data you provide them, contracts or settings be damned. Completely untrustworthy as an entity, full stop.Of course, I’m also too jaded to think this will change anything. Folks will move to Anthropic, or Gemini, or Grok, or some other hosted model on a pubCSP managing the harness and logs for them, and then do another shocked-Pikachu face when it happens again.If you aren’t running workloads on infrastructure you own, then your privacy, security, and general outcomes are at the sole whims of the hosting provider - who can and will fuck you over the exact second it’s more beneficial for them to do so than the loss of trust incurred.
Grimblewald
people seem to miss tge point of this. The problem isn't about credit, its about portraying these models as more competant than they really are. It fuels idiotic statements like jensen huangs recent "agi achieved" statement, which fuels an already dangerous financial fire.
techblueberry
But who are you going to believe? Multiple independent academic researchers or the CEO who was fired two years ago for gross dishonesty?
vrganj
OpenAI is showing the world why they shouldn't trust AI hosted on some cloud somewhere.If they're stealing math proofs to advertise their models, who's to say they won't steal your businesses IP to gain a competitive advantage?They're not to be trusted with your data. I can't believe how short-sighted this is, they got a quick PR win at the expense of a much larger trust problem.I wouldn't trust cloud AI at all at this point. Get an open Chinese model and host it yourself somewhere. The initial costs might be higher, but you'll break even pretty quickly and nobody will be able to steal your innovations.This is American AI companies committing suicide.
protocolture
Gonna need grants for local models. Its happening. OpenAI and Anthropic models are powerful but are rapidly approaching the good ol trust thermocline.
viccis
Some mathematicians I know who've been following this have realized that they'd all gotten some emails from people they now know to be affiliated with OpenAI/Anthropic asking questions about their research in a way that seemed like scooping attempts.Also, a lot of my mathematicians buddies have reported students basically asking if it's worth ever doing grad school for pure math, and even very motivated students are looking for other options now. It's not because they aren't passionate about it, it's that they don't want to work for another half decade or more just to have to start their careers all over.All of this so that OpenAI and Anthropic can get into math result dick measuring to gas up their IPOs. Sickening.
mannanj
And I have been proclaiming a cry of “your data for analytical purposes is being stolen” (you can’t opt out of analytical purposes) and people perhaps astroturfers straw man back to “just turn off training bro”.Yeah. Remember yall: you CAN NOT opt out of analytical purposes. And you also cannot get a guarantee that it doesn’t give them your data to steal for their business.
moralestapia
>AI is stealing human discovery.AI is not stealing human discovery, OpenAI is.
anon
undefined
pixel_popping
Prompts are handled by the service itself, meaning it's used, absolutely anything passing there is recorded, why wouldn't it, the entire premise of those companies is to train on data which they stole initially.Are we back to the era where people blindly trust product TOS instead of actual cryptography, have we forgotten already the thousand of fines Google, Microsoft, Apple and practically all top companies got for breaching their own ToS and the law?Common, on HN at least I would have thought that everyone assume that anything arriving on a server in PLAINTEXT is recorded (thus used later)?Let's not forget that at any moment, OpenAI/Anthropic/Google... could be providing stronger privacy guarantees by having proper attestation with e2e, they have the budget, solid engineers, why isn't it done? Answer is pretty simple imo.
anon
undefined
nobodywillobsrv
The real annoying thing it seems is mostly that openai is presumably doing this for internal reasons and this marginally increases the cost to users with no real gain.It would be one thing to gain from it but removing prestige wins from customers AND reducing compute support just feels like being ultra mean if you zoom out.If this was racing to cure cancer ahead of researchers we wouldn't be writing about this on HN.
seobot_dk1289
[flagged]
0utcast
[dead]
josefritzishere
[dead]
ath3nd
[dead]
1337h4xx
TL/DR: Mathematician opted out of training on 29-JUN and asked OpenAI whether they trained on his data and was told that it "did not happen" but it clearly did.
shevy-java
[flagged]
ThalesX
[flagged]
bossyTeacher
Trust and OpenAI never go together in the same sentence. The answer is always no.
mainecoder
Hopefully OpenAI can solve good problems where no one can make a claim that they stole their idea where the methodologies used and the techniques used are so out of the ordinary that the achievement is respected. Furthermore they should work on new novel solution on the old problems to lay these issues rest, thus by improving their models they can avoid issues of academics accusing them of using their work additionally the academics should also demonstrate their unpublished work is significant enough to have solved the problem . This is a bit subjective but it is also objective for the person with domain knowledge.
spongebobstoes
I think this is mathematicians coming to grips with the fact that AI is surpassing themwe will all have this moment soon enough, and it will change how we think about intelligence, identity and value