HN

Comments (389)

• sleepyguy
  http://archive.today/2OWwO
• Menu

  swalsh
  There's issues where bad people with bad intentions use AI to do bad things. But to an extent the guardrails already built-in are sufficient. The real danger in AI is how it impacts society, our economy, and our perception of self worth.I'm not afraid of rouge agents nearly as much as I'm afraid of us building a permanent underclass of people dependent on whatever scraps the people who devalued their labor decide the grace them with. I'm afraid of the security apparatus that will be built to keep this underclass in line.
• Menu

  perarneng
  The economy isn't ready. Basically, all companies right now are sprinting toward increasing profit margins by trying to do more with fewer people. The problem is that, on this scale, they will lose income because unemployed people make terrible consumers, they will only buy what is strictly necessary. Consequently, society is in a race to the bottom, which could lead to extreme suffering and potential revolutions in many countries. Democracies will likely have it easier in general because everyone shares the responsibility, in a sense, through voting. However, the USA, which is extremely polarized—will have a tough time because animosity between the political sides is already at an all-time high, even without an economic collapse. China, on the other hand, will also be in a difficult situation. If the West collapses financially, consumption of Chinese-produced goods will drop drastically. This will create massive unemployment in China, causing public rage to increase. In a non-democratic country, it is much easier for the population to point the blame in a single direction.
• Menu

  turing_complete
  We don't need to live "side by side" with AI. AI is not alive, it's a technology we use. This is like talking about living side by side with your toaster.
• iDon
  The risk is humans using AI to control / exploit / coerce / injure other humans. The risk of AIs being given enough agency to threaten humans comes after that - they'll only have the agency we give them (being "alive" or "conscious" is not the near-term risk).The article lists diplomatic actions which might help to manage the risk, starting with "An agreement between America and China" - they all sound like impossible dreams. We had ~80 years of relative peace and prosperity in which to construct a framework of unity to face challenges like AI (and global warming, which until GPT I thought was the bigger risk); international unity is weaker than ever. In geopolitics and defence, capability of other nations is the concern rather than intention; the capability curve of LLMs is heading off our charts. We're backed into tight corners on nuclear proliferation, global warming, and I can see LLM-enabled conflicts (cyber warfare, infrastructure terrorism) pushing us over those other edges. Our democracies seem weakened, and I expect LLMs will empower those using social media to create conflict and control opinion. We're familiar with the cycle of inventing new technology which benefits people, then seeing how long before people invent ways to misuse it. There is a possibility here that LLMs could be used to solve the problems we are juggling, but I struggle to imagine that people won't misuse it even faster.The article is a start on thinking and talking about managing our risks. The best outcome would be it is so well managed that, like the Y2K "bug", people say "after all that hoopla, nothing happened". I'm not seeing a smooth path to there.
• Menu

  jpease
  Humanity hasn’t been ready for the current ignorance explosion.
• Menu

  tedggh
  Why should we assume AI can rapidly turn into super intelligence when physical and critical resources like energy and materials remain under human control?
• Menu

  stuaxo
  If the economist is writing it then it's probably not happening.
• assimpleaspossi
  >>Giving birth to a superintelligence would be the most consequential moment in human history—and it is likely to be irreversible, as any “off” switch humanity might design will probably fail.Nothing will be able to stop you from pulling the plug.
• Menu

  truthbe
  Like my grandfather used to say "They let anyone on TV now", They let anyone be called an expert now.Love talking about 'recursive self-improvement' of a prediction engine. What about 'recursive self-degradation'?
• Menu

  Animats
  How far are we from reliable AI? One that could, for example, handle 80% of office jobs without screwing up more than humans?
• Menu

  Schlagbohrer
  Why is someone from Planet Labs writing this?Why not, for example, Geoffrey Hinton, the scientist who won the nobel prize for LLM architecture? Or a sci-fi author?
• jwpapi
  Transcription hasn’t become better for 2 years. It was better than before, but there is a ceiling.
• Menu

  cortesoft
  I couldn't read the whole article, but just from the part I could read:> Strikingly, this concern is being openly voiced by the very people who have the strongest incentives to project confidence rather than alarm: the founders of the largest ai laboratories.I don't know, they also have an incentive to make their technology seem transformative and powerful, and saying that your technology has the power to cause a massive catastrophe is a way to promote that idea.
• Menu

  hypfer
  Reminds me of all of those ethics debates back when fully self driving cars were basically already here ca 10 years ago.The ones talking about how hard it would be to chose which person to run over.Additionally, I find it hard to believe that this would be a case of the future just not being distributed evenly.But sure. The AI labs relying on hype stating 15-50% risk of building a magic entity is certainly a reliable number.
• Menu

  ilamont
  The first priority should be an agreement between the two heavyweights of ai: America and China. Donald Trump and Xi Jinping should affirm the principle that humans must remain stewards of ai systems until adequate frameworks for reliability and security have been built.This is naïve. The goals of both men have nothing to do with protecting humanity, but rather furthering their own personal agendas.In Trump’s case, it’s all about amassing more wealth and power.For Xi, it’s realizing an ethno-nationalist dream where China under the CCP is at the center of world power, the independent nation of Taiwan as well as disputed border areas that are currently controlled by India and Russia and the Philippines are annexed by China, and Xi’s eternal legacy is remembered as the savior of the Chinese people.International cooperation and touchy-feely rhetoric about saving humanity from AI have no place in either man’s worldview.
• Menu

  al_borland
  > Experts in artificial intelligence estimate...The predictions of these "experts" have been drastically wrong for the last 3 years. At what point does someone lose their "expert" title?
• Menu

  seydor
  Oh I'm ready. I ve had enough of useless tech that evolved to a global network where people vent their frustration about technology. Let's do some cool stuff instead
• Menu

  nopinsight
  "Ten years from now, I think we will realize that we were standing in the foothills of the singularity now... I believe that we're only a few years away from [AGI], maybe 2030 plus or minus a year...I think [AGI] will be an enormous transformative technology, it's going to effectively be a new human era...We can feel this year, I would say, even though I've been working towards this for 30 years, I think this year with the way the agents are working and tool use, it started to become really useful, still early days of it, but genuinely useful in people's workflows...And it's not any one thing, it's several different technologies, several use cases, several things that I thought were maybe a bit further out, turned out to be now, that are coming together that make me feel that in aggregate.I think society needs to hear that because we don't have long to prepare for what that means." -- Demi Hassabis, CEO of Google DeepMind & Nobel laureateSource with interview clip: https://x.com/deredleritt3r/status/2062223035940139253EDIT: Ten years ago, most of the skeptics in these threads would have said the success of AlphaFold 3 was impossible.
• Menu

  g-b-r
  > Fermi asked why, given the apparent abundance of planets suitable for life, no evidence of other technologically advanced civilisations had been detected. One disquieting possibility is that intelligent life routinely reaches a technological threshold and fails to navigate it,The Fermi paradox could actually also be taken as an evidence that it's rare (at least) for artificial intelligences to take over a civilization and sprawl and survive for very long times
• Menu

  g-b-r
  > Strikingly, this concern is being openly voiced by the very people who have the strongest incentives to project confidence rather than alarm: the founders of the largest AI laboratoriesOr to have AI regulated in their favor
• Menu

  CamperBob2
  Humanity doesn't fully leverage the intelligence it already has, so I think people are overestimating the disruption ahead.
• PowerElectronix
  This is the kind of articles that make the last, biggest fool buy the top of a bubble.A good chatbot, image/video generator, coder, etc is not going to enslave us all (not in actual sense, although maybe in the instagram sense) or propel us into a golden era.They are nice tools for some pretty specific tasks and not much more. And we are only seeing the heavily subsidised version of the tools.
• Menu

  g-b-r
  Is it really likely that a "recursive self-improvement" capability would lead to a great acceleration of AIs capabilities?Isn't the preponderant bottleneck in improving the models the need to train them at scale to verify the hypotheses, and the time and cost that it takes?Or does someone think that they could get magically able to predict big improvements without training?
• ur-whale
  https://archive.ph/2OWwO
• Menu

  alecco
  Another CEO with AI psychosis [1]. LLMs are not true AI, they lack common sense (or whatever it's properly called). LLM-based systems still need somebody with deep domain knowledge at the wheel to keep them from doing stupid things. It's like an alien-made bicycle that gets you to the speed of sound if you are an Olympic cyclist.Still, LLMs are extremely powerful pseudo-AI [2] and will bring a pseudo-singularity. But the impact is still scary if a tiny fraction of humans are augmented 1000x. And as better models become exponentially more costly, only the money people will be able to afford the new models. This is a very likely scenario and scares me to the point I dropped all my projects to work on affordable LLM-based tools to make the difference at most 10x instead of 1000x.To my elder relatives I explain it like: imagine we are farmers in the 17th century and suddenly out of nowere John Deere tractors, combines, etc. become available. But they cost more to run than all you and your fellow farmers have, so only a tiny handful of rich people take over everything.[1] https://en.wikipedia.org/wiki/Chatbot_psychosis[2] https://en.wikipedia.org/wiki/Chinese_room
• croes
  > Experts in artificial intelligence estimate the risk of an ai-caused catastrophic event at 10-50%Because something AI did or because a human in the loop blindly trusted the result of AI or the promises of AU companies?
• Menu

  thunderbong
  Honestly, this article has too many words which don't mean anything.'AI Experts', 'superintelligence', and hand-waving doom scenarios.As an example-> Within a couple of years, possibly much sooner, AI may achieve so-called closed-loop recursive self-improvement (RSI): the capacity to rewrite its own code to become more capable, without human intervention. Should that happen, the result could be an intelligence explosion of a kind for which there is no precedent and no map.I've heard these same objections in my lifetime about the internet. And I've read similar arguments against TV, radio, the phonograph, and the printing press.Honestly, this is getting extremely tiring. Every new invention that has happened has affected the world in both good and bad ways.Ultimately, what counts is what we do with it.
• Menu

  esafak
  I think it is so reckless and unethical that AI companies are not spending good money on answering the political, social, economical, moral questions that they are raising.
• caporaltito
  Yet again another journalist who read too much science fiction and should stop overestimate a bunch of rocks conveying electrical pulses. Or is this simply to get more clicks?
• Menu

  gorszon
  Can these snake oil salesmen stop with this Sci-fi stuff please? Yes, Machine learning, and transformer architecture are great stuff, but lets not pretend this is sci-fi, please.
• Ericson2314
  Oxford Brain
• anon
  undefined
• mhd
  On the same page (for me): "Silicon Valley needs to get God"Well, seems it already did.
• Chrise_N
  We'll be fine. We've seen this one before, this is just new packaging.
• Menu

  jdw64
  I want to name the AI era as the corruption of the elite.The rhetorical structure of talking about uncertain risks and then trying to concentrate the authority to manage those risks in their own hands sounds utterly ridiculous to ordinary people like me.It's just a simple hypothesis that AI will become uncontrollable to humans once it becomes superintelligent.Isn't the fact that a reinforcement learning agent improves itself in a specific domain completely different from it recursively improving its own code in a 'better' way? It's just a tool to create a justification for regulation and control using sci-fi fear.The comparison between nuclear power plant risk and AI risk is also absurd. Where exactly can you define and measure the probability of AI exterminating humanity? It's as unquantifiable as 'I, human JDW64, will become a successful programmer.' What is the measurement standard? Why dress up AI researchers' concerns as objective probabilities? Is it because numbers make it look logical?The current US-China relationship is in the middle of an AI arms race. The US is strengthening export controls to limit China's AI development, and China is building its own ecosystem. In this situation, I don't understand the idea of cooperating for the common safety of humanity. RAND is an organization that presupposes cooperation—isn't it just a well-written research proposal from an institution that wants to position itself for that role?Isn't the claim that 'government must step in' ultimately about protecting their own interests? 'A strong government that will protect us' is an authoritarian government. If they were East Asian, they would understand that such regimes have always been used as tools for surveillance and control.And I don't understand why Fermi's paradox is being brought up here. Why package a software problem as something that inevitably requires strong control? Whenever I see articles like this, I think about what 'intelligence' really means. This person would probably be called 'intelligent' by others. But no matter how I look at it, the holes in the argument are too obvious. It really makes me think that there are different tiers of intelligence.
• Menu

  themafia
  A similar Luddite view formed in the 1990s. It turns out humanity constantly beats the expectations of economists. It makes you wonder if the "science" of economy is almost entirely bankrupt either in morality or imagination or same dangerous combination of both.> Humanity simply does not have a strategy to ensure it remains safe through RSI.Turn off the power. It's pretty simple. Leave it to an economist to forget about input costs. Your "super intelligence" only matters if it's actually more energy efficient than a human being and for a million years of evolution humanity is a much harder target to beat than this author seems to realize.
• elorant
  Economist has drunk the AI koolaid. Pretty pathetic for a publication of their esteem.
• surgical_fire
  Humanity isn't ready for the amount of bullshit being written on a daily basis to pump the upcoming IPOs.
• add-sub-mul-div
  > Experts in artificial intelligence estimate the risk of an AI-caused catastrophic event at 10-50%.I don't think it's that high but can we put that aside and focus on the 100% chance that it's being used to enshittify every part of our lives?
• cat_plus_plus
  Models consume a lot of memory and power to slowly generate autocompletions of existing content one token at a time. Letting this text control important things in real world is a human decision and control of nuclear weapons or penetration testing agents should be well regulated. But these should already be well regulated without AI. So how about we focus on drawing down nuclear arsenals, which is a present danger one idiot (or faulty AI) away from world shattering consequences rather than fearmongering about unknown future. Before effective AI regulation can be drafted, we need to anyway know more about AI specific dangers rather than humans committing already very common crimes like hacking for ransom with new tools?
• anon
  undefined
• smitty1e
  > "recursive self-improvement (RSI)"My aching joints enter the chat...
• avazhi
  AI labs do not have an incentive to downplay the risks and if you can’t understand that you’re naive and shouldn’t be writing Economist articles.AI labs have every incentive to overstate the risks so that they can get lucrative government contracts, especially since it’s clearly not profitable going the public consumer route. And if you’re Anthropic you’re even more incentivised to overstate the risks because at this point in the game regulation hurts potential competitors more than it hurts you.
• danaris
  This article posits "recursive self-improvement"—which is just a more-technical name for the supposed Singularity.There is no evidence that we are currently on a path that leads to the Singularity.There is not yet any evidence that the Singularity is more than science fiction.It is effectively an article of religion—the Rapture, but for techbros. Indeed, it is what some of them are pinning all their hopes on; after all, if they can recursively-self-improve their way to artificial superintelligence, then and only then could the absurd investments of companies like OpenAI actually prove worthwhile.
• shevy-java
  Of course humanity is not ready for skynet. But there are rebel forces - the no AI using humans. They may be a dying breed but at the least they put up a fight. Many other humans already have become servants to skynet (version 16.0 now). Next step for them is the neuralink chip. Some billionaire with twitching right arm gestures owns that, doesn't it ...(Also, Cameron was not quite right with regards to skynet. It is much much dumber but also more effective than displayed in his movie really. Kind of a weird combination if you look at the current AI slop out there.)
• lousken
  As the AI gets "smarter", it is gonna make the society dumber so it will balance itself out /s
• kurthr
  I wanted this to be thought provoking, but for a physicist to open with, "the acceptable risk of catastrophic meltdown for a nuclear power plant is roughly one in a million", just seems sad. It is a phrase without meaning. Is it per plant? there aren't a million. Is it per year? notably have been at least 2 major ones (arguably 7). A meltdown (or loss of containment) just isn't that bad, if it doesn't affect ground water or lead to atmospheric fallout. We're turning 3 Mile Island back on to power an AI datacenter! The AI super-intelligence apocalypse envisioned (ignoring the likelihood) is inherently global.That said, the rest of the analysis and proposal also greatly disappoints me. The idea that the current administrations of US and China could do anything constructive seems hilarious. They're so paranoid and self-serving they couldn't come together, even if there was an alien invasion. Then the idea that LLM safety is somehow as easily traceable as nuclear isotopes and bomb tests seems equally ludicrous. I am sad.
• agrijakhetarpal
  "humanity" and intelligence explosio" in the same sentemce! Muhaha! Boy, have you been on just a couple of tourist flights / visiting major tourist hotspots?Most people out there are literal Normies, and operatelike a robotic vacuum cleaner, or NPC.