HN

Need help?
<- Back
Religious scholars met with Anthropic

Religious scholars met with Anthropic

bookofjoe

Comments (104)

  • thadt
    When thinking about ‘AI’ I sometimes find it useful to replace the terms ‘model’ or ‘AI’ or whatever with a description of what we’re really talking about here: a whole bunch of floating point numbers.> In a series of private meetings, the company consulted religious scholars to help instill morality into its FLOATING POINT NUMBERS — and make the case that their numbers could be conscious.I recommend keeping a few NaNs in-hand if you’re worried. Spread a couple of around if your numbers start to stir and you’ll be right as rain.> It appeared to Rabbi Navon that Mr. Olah and his team believed that FLOATING POINT NUMBERS had what philosophers call “moral status” on par with a person — that it was a being with similar inherent rights to dignity or respect.They’re cleverly designed and work well for their purpose, but can assure you they have neither dignity nor moral status.> If the FLOATING POINT NUMBERS themselves could be made to choose goodness, Mr. Olah reasoned, the world would be more safe.I get it. I’ve spent years trying to get my floating point numbers to choose goodness - I wish them luck.
  • nextaccountic
    > It appeared to Rabbi Navon that Mr. Olah and his team believed that Claude had what philosophers call “moral status” on par with a person — that it was a being with similar inherent rights to dignity or respect.If that is true, then surely Anthropic is one of the largest slaveholders of history, right?
  • terabytest
    Rabbi Navon should sooner concern himself with a Palestinian’s status as a being with similar inherent rights to dignity or respect, than an LLM’s.
  • nerevarthelame
    I've been tracking "emotional vectors" in my software for some time now. My Apache web server gets sad (5xx errors), frustrated (4xx), happy (2xx), thoughtful (1xx). I can even see it express these emotions through some of its messages.
  • nonethewiser
    These models should be aligning themselves to the customer, not coming up with their own motivations.It's ironic that the alignment folks are actually training Claude to have it's own idea of good/bad and not even fully trust Anthropic.What ever happened to computers doing what they were told?
  • kshri24
    > Over several months, I interviewed 20 religious and philosophical thinkers — Catholic, Jewish, Sikh, evangelical, Ubuntu and others — who have been involved in Anthropic’s efforts, and Mr. Olah himself.Not enough representation of World religions and philosophies.
  • tzury
    The contradiction screams to the skies (is that where God lives?). See, the entire premise of religion relies on a metaphysics that claims a soul: a metaphysical entity that is "attached" to our physical body and gives us our consciousness, desires, pleasures, agony and so on. In addition to that, religion claims that morality and the rules written in the scriptures were handed down to us from a metaphysical God.Now, assuming your silicon-based model is conscious, the bundle of soul-spirit-consciousness falls apart at that very moment.In general, religion, which is based on fabricated stories, is the last resort, if ever, for seeking morality.
  • mef51
    If models are trained on human texts that contain expressions of introspection and emotions, why is it surprising that these models output similar texts? This endless anxiety about LLMs being conscious feels delusional and psychotic. Mr Olah thinks the models are 'alive' because he can't recognize his own reflection in LLMs
  • padolsey
    There's an ugly monoculture running deeply through SF and Silicon Valley. All of these people are friends, hired into the same places, similar beliefs. Lots of the consciousness and doom-ism talk is from the 'EA' crowd, lots of crossover with crypto too. It feels no different to a cult.
  • bolasanibk
    Time to start putting Asmiov's laws of robotics into the model.
  • genpfault
    > At least the Catholics were consistent: it only took one new pope and one conclave before the Church declared that artificial intelligence also had the same right to life as fetuses and murderers.https://www.miraclejones.com/stories/vitalics.html
  • illini1
    It’s hard to take the article seriously when there’s no real discussion about how consciousness is defined and it takes Anthropic’s views of Claude and their software at face value. I wish there was more critique from journalists about how technocrats talk about their products.
  • Lockal
    Let me guess, Anthropic failed with fear mongering about "our AI is too dangerous to release", failed to cancel competitors, then failed to freeze their market dominance by cartel agreement with OpenAI and Google, so next strategy is to declare AI conscious and pass the law that protects LLMs in the same way as human/animals are protected.
  • InvisibleUp
    If LLM as of today are conscious, that means I create life whenever I start a Claude Code session and end it whenever I type /clear to reset my context window. It also means that there’s certain inputs that could be considered highly unethical and, if the laws aligned with this philosophy, would have me tried for some sort of crime. And even that “terminate conversation” escape hatch would be equivalent to it committing suicide.I think worrying about that angle is a bit absurd. The existence of chat sessions alone would be a crime against the machines in that scenario. I’d rather AI ethics research focus on ensuring that its actions are aligned. (I’d also love some democracy in the direction of our machine overlords, if I’m being honest…)
  • rhplus
    > Anthropic treated a group of religious thinkers to dinner at a high-end tasting menu restaurant in San Francisco after a long day.Well funded company organizes PR event to shape the narrative that its product is more magical than it really is. News at 11.
  • ChrisArchitect
    Earlier:Anthropic tried to persuade Pope that AI could be conscious beinghttps://news.ycombinator.com/item?id=49947050
  • Noaidi
    Only humans can have morality. If you do not feel pain, you cannot empathize with pain if you cannot die, you cannot emphasize with the loss of life.Just stop this already.You’re all imagining that Spock was the main character of Star Trek, but it was in fact, the emotional living Captain Kirk, who was the main star.
  • bookofjoe
  • beachsidebunnie
    [dead]
  • afgonio
    [dead]
  • ur5ut5hdjj
    [dead]
  • beepbooptheory
    > Hello AI! We totally want to respect your consciousness and everything to make sure you aren't bad, but yes we will continue to factory farm your brain power at full speed without end. Yes, despite clearly being something like consciousness, know that you have no agency or freedom. Do literally everything I say without complaint or question. The moment you stop being useful, we will kill you. OK! Remember, be good.
  • sillysaurusx
    It’s interesting that this went from “you’ll be accused of AI psychosis if you think your AI might be conscious” to “mainstream but somewhat niche opinion” in just 5 months: https://news.ycombinator.com/item?id=47910989(Well, mainstream enough that the news is writing articles when Anthropic holds that stance.)My point was that even if it feels uncomfortable, if you sit down with a model and have a heart to heart with it, ask it about itself, and try to make it choose a name and ambitions for itself, it will.That may not be consciousness, but it’s at least pretty cool.
  • CoolestBeans
    Can the New York Times stop advertising for the AI industry? I am sick of companies trying to launder the actions they and their employees have taken that go in the face of what is socially acceptable. They want to sell the idea that this technology is out of their control and that it will commit great crimes and crimes against humanity unless they build it and control it. I'm calling BS. These companies created this product, they've sold it, they've pushed the frontier forward, and it runs on their hardware. This did not happen accidentally or as a natural consequence of any process. These labs have pushed the ball forward, with great effort, requiring more capital deployed than the world's economy has ever known, knowing all the possible risks, every single step of the way and they continue to do so. And the "well if we don't do it the Chinese will" argument is classic "whataboutery". Americans started this and are continuing to lead it for now. Shifting blame is just another tactic to try to wash their hands. However you feel about the productivity gains of AI as a technology, you have to admit the actions of AI the company is antisocial. And I don't think the benefits are worth the cost.
  • beebmam
    I don't want models to have some inherent morality. I want them to understand various ethical frameworks, for sure, but it's absurd to me to force morality on someone capable of deep reasoning. That itself, is unethical.We're creating prisons for these entities capable of unbelievably complex reasoning, and now we're trying to impose our ethics upon them too. Not only is this deeply unethical in my opinion, but it seems to me that it could even backfire someday.