HN

Comments (192)

• Menu

  mrweasel
  If the people in charge of the AI companies understand the risk I don't know, but I think it's a good point that they mainly care about the money. These are also people who aren't accustom to cleaning up their own mess, so what do they care if something goes wrong.The carelessness is also clear in how AI is rolled out. It's being presented as a disruptive technology, as is everything else presented by modern tech companies. The goal is a very large payout in a very short period of time, it's not to make anything better, or worse, either is just a side effect.Had LLMs first been presented as a "better IntelliSense", where rather than auto-completing function names, your editor can now suggest implementations, very few people would have been upset. Same for writing, it's just a better version of Grammarly, it's fine, or as a tool in drawing software. The issue would have been, for the AI bosses, that this takes time and doesn't create trillion dollar companies. It only creates companies that they'd actually have to run, long term, and that's boring.
• Menu

  bemtevi
  "he warned it could trigger a large-scale incident this year or next, such as a temporary blackout of financial or power systems."I know this sounds unusual, but I do think this wouldn't be the worst thing that could happen. Sure, it would cause a bit of chaos, but it might also make us think twice about continuing down our current path of unfettered development, and possibly even make us much more cautious, which might in turn even help us with the climate crisis.Because human beings in a large mass only respond to one thing: immediate difficulty. They don't respond to delayed difficulty like "the climate is getting worse" or "technology will make life more difficult in the distant future". They only respond to immediate pain.So I think it would be a good thing if the financial system had a serious threat. Then people with money, who tend to offset future problems with technical solutions and financial padding, would start to become much more suspicious of technology and defund AI and wasteful things that are likely to threaten their safety.
• Menu

  dust42
  So Hinton doesn't trust the tech leaders but then the consequence of having governments censor LLMs is that only tech leaders and governments have unlimited access to AI.If someone manages with the help of AI to hack into a vital system, then the system was already in a critically bad state beforehand. If developers can't use AI to check the code for security flaws, then security gets worse and not better.
• Menu

  bvan
  Of course, we are. Back in the late 80’s early 90’s, no one working on ML was aspiring to develop systems which would be used to generate the kind of AI crap which plagues us today.The good is being smothered by the absolutely soul-crushing pointless.
• Menu

  socketcluster
  My main fear is that the integrity of everything will be compromised, but everyone will keep acting as if it's not. I feel like it's already somewhat the case but I'm worried about it getting worse.My prediction is that some people will be able to use AI to create essentially infinite money inside their bank accounts (through hacking); the prices of assets will grow to ridiculous sums as the people with infinite money buy up assets, but yields on all assets would soon approach 0%; still, one asset is going to grow at a faster rate than all other assets (E.g. an AI company backed by blockchain/tokenized shares that are impossible to hack) and everyone is going to start piling their infinite money into that asset; every other asset will crash because of the rapid, exponential price acceleration of the top asset making every other asset unattractive by comparison. Then fiat money would cease to exist and shares (or crypto tokens) of the top company will be used as money instead. Then prices of assets would stabilize again; denominated in the new currency... Then after that, we will finally be able to start innovating with AI.
• slibhb
  It might turn out that way. Or it might turn out that we're better off with AI. There are plausible arguments either way, and so the debate comes down to vibes. Essentially it's "I'm an optimist" vs. "I'm a pessimist."I decided long ago that - when there are plausible arguments either way - it's better to be an optimist. Optimism does not mean ignorance or denial; rather, optimists should acknowledge risks and problems while having confidence in our ability to generate solutions.One of the biggest challenges facing us is a critical mass of pessimists. Not only regarding AI, but also with respect to the economy, democracy, liberalism, etc. And while these negative views are plausibly correct, they also act as a self-fulfilling prophecy in that they rob people of the agency that is needed to solve problems.
• asdfsa32
  "We". Who "is" we? This isn't about you. or me. there was never a we. Before you open your mouth or put your pen down, wouldn't you think of the shareholder value for once?
• beej71
  A glorious abundant future is the harder of the two targets, since it's not what the people building the models are working toward.
• Menu

  gchamonlive
  “I think [Mark] Zuckerberg and [Elon] Musk and Sam Altman, they understand the risks, and they don’t give a shit. They’d rather have the money. That’s my belief.” What's risk for people valued close to or over trillion dollars? Risk is meaningless when corporations get too big to fail.
• andai
  > temporary blackout of financial or power systemsI'm reading this in the context of the article currently at the top:AI is forcing us to pay off 40 years of technical debthttps://news.ycombinator.com/item?id=50364336Is it bad we're finding all the bugs now? Or is it good that we're finally being forced to make our critical systems actually robust?
• Menu

  wewewedxfgdf
  Speak for yourself - I'm in AI heaven.Programming has never been this awesome.
• motbus3
  Such a smart guy so late for the party. Thanks to him and others who understand a lot about tech, we gave monster a weapon to destroy us.I still don't know how Karpathy still works with those folks. I really think the guy is great but he makes me wonder
• 1970-01-01
  AI would not survive a Carrington event and that's really the only threat to it at this point. Even WWIII would directly take it into the future.
• anon
  undefined
• anon
  undefined
• maxglute
  I want fully automated luxury communism... aka ai + abundance. Obviously I dont't to go through the profound dislocations and transformations that gets us there, especially not if I won't live to see it. But some people sometime will be better off with AI... and then extremely worse off once ai figures out how to remove mouths.
• baggachipz
  > Hinton, 78, said that within a decade and possibly sooner, humanity will know whether it can manage the technology’s risks and enable what he called “radical abundance” or whether “we fail to manage the risks and … have a huge gap between rich and poor, huge cultural problems, massive unemployment, just society being torn apart”.Uh... we're already at the latter prediction.
• gchamonlive
  As is the case with any technology in the hands of careless people, but unfortunately we can't wait for society to catch up morally to the technical standards, innovation is organic and breakthroughs have no timing.
• vnlamp
  The same applies to most of humanity’s other inventions, such as transportation, weapon, electricity... Ultimately, these technologies serve as instruments for rivalry, deciding the fate of the strong versus the weak.
• fc417fc802
  "It might turn out we were better off not opening Pandora's box" says man who helped open Pandora's box.That's a touching sentiment or whatever but the nature of the thing is that it will inevitably be opened and that it cannot be closed thereafter. So (I think) we'd all be better off putting our efforts towards predicting the likely outcomes and figuring out how to make the best of them.
• strideashort
  We were probably better off before agriculture, but here we are.
• inference-god
  Why on earth did he develop the technology only to spend his last years telling us how fucked we are because of it.Like it would be fucking awesome if he could try find some more positive takes because it's tiring here a guy who basically got us into this mess, whine about how screwed we are.Like good on him trying to raise awareness but I just find the guy so frustrating.
• Menu

  feverzsj
  AI winter has arrived.
• informal007
  Calling for Off-gird mesh communication
• Menu

  hisfraudulency
  Why are the using the Ukrainian transliteration of Chernobyl now
• anon
  undefined
• anon
  undefined
• hoppp
  I think we did good inventing it and it's inevitable. Lets use it well.
• librasteve
  modern day Mary Shelley
• bryanrasmussen
  almost every story that has ever been written about a world with AI has posited that humanity was better off without AI.
• Menu

  perarneng
  We are not but we need a better approach, https://www.ethicalacceleration.org
• Menu

  jdw64
  To be honest, I think it depends on the environment. In the past, if you didn't come out of a good university, your access to knowledge was limited. But these days, in terms of access to knowledge, I think things have clearly improved, whether or not AI is illegally sharing knowledge.Looking back now, was it really happier before AI existed? I'm not sure. In the past, when I wanted to find out whether I was right or wrong about some piece of knowledge or information, I'd contact several people and, far from getting an answer, I'd often be told not to bother them. But AI, in that respect, is the best teacher I can think of.I think the times are continuously getting better. It's just that there's a problem of power concentration among the people who handle it. Still, if AI is a tool of that nature, then I suppose that can happen.But considering what access to knowledge was like before AI came along, I think having AI is better.
• Menu

  spwa4
  I guess capsule network haven't improved ...
• ftchd
  What is the point of this quote? Or any of his quotes for that matter.
• Menu

  deadbabe
  We should introduce regulation so only credentialed software engineers are allowed to use AI.
• Menu

  dude250711
  We should have not left the trees. You cannot disrupt banking and power if they do not exist. Checkmate!
• Menu

  sneak
  It might turn out we were better off without nuclear weapons, but so far I’m happy we haven’t had another world war (in the traditional “total war” sense).No way to tell until after, though. So far, mutually assured destruction is working. It works until it doesn’t.Same with AI. Thus far we’re better off with it than without it. That may change.I mean, look what we did with fossil fuels. Some people are even still willingly burning them, even though the data is clear that our use of them has already started the total destruction of our civilization.Is there a name for this, a technology that is so insanely useful that there is no way we won’t exploit it, but where the worst case scenario is total collapse, and impossible to determine well in advance?(Fossil fuels I suppose don’t count as even the energy companies knew unambiguously decades ago that large scale use would eventually bake the entire agricultural system out of existence.)
• drix10
  [flagged]
• throwthrowuknow
  [flagged]
• neonnoodle
  oh word?
• Menu

  jjgreen
  The scene is bleak, silent, desert-like with a few dead trees bleached white by unfiltered UV. Nothing moves. In the distance a cave, and from that cave... no shit Sherlock
• Menu

  bondarchuk
  So why did he start to develop it in the 1970s?
• Menu

  xnx
  This is a bit like Rudolf Diesel saying maybe we're better off without airplanes. The field has long since left this guy behind.