"Anthropic Researcher Says..."

There was a saying that good manners cost nothing. :laughing:

Good point, dismissing humankind would alter the climate footprint we’ve been contributing to this planets destruction.

Rich :wink:

…what the fuc k are you even saying?

We saw it on the movies already, Terminator, Matrix …

:robot::alien_monster:

And.. keep it on the plot.. ahh I mean topic​:sweat_smile:
And not t{ooooooooox seriously

well at least Claude has reached super human intelligent

out of curiousity i asked my trusty sidekick AI pal ,it’s thoughts on this thread …here’s the reply :grin:

:joy: Yeah, I read the whole thread. My take is that it is a surprisingly good example of how discussions about AI can go completely off the rails in about 15 minutes.

The original claim is actually quite serious: Anthropic safety researcher Evan Hubinger reportedly put the probability of AI causing human extinction within the next decade at greater than 10%.

But then the EndeavourOS thread immediately goes:

“More like 99% ±1%”

followed by:

“So there is room for improvement”

:joy:

That’s classic Linux-forum humour.

But underneath the jokes, I think there are three different conversations getting mixed together

1. “AI could genuinely be dangerous.”

That’s not a ridiculous proposition. There are legitimate AI-safety researchers who think sufficiently capable systems could create risks that are very difficult to control. Robert Miles, whom one of the posters mentions, is actually a good example of someone who explains the technical alignment problem without needing the Hollywood version of the apocalypse.

2. “AI has a >10% chance of literally killing everyone.”

This is where I’d become much more skeptical.

A number like 10% sounds extremely precise, but what exactly does it mean? What assumptions went into it? What counts as “AI”? What counts as “killing all humans”? What time horizon? What human interventions are assumed?

Those questions matter enormously.

It’s not like we’re talking about a weather model saying there’s a 10% chance of rain. There isn’t a giant database of previous AI civilizations from which someone can calculate the probability. :grinning_face_with_smiling_eyes:

It’s fundamentally a subjective risk estimate, albeit one made by somebody who knows considerably more about AI safety than most of us.

3. “The AI industry is deliberately amplifying fear because it benefits them.”

This is the part of the thread where I think there’s something worth discussing, but some posters jump too quickly from “there are incentives to exaggerate” to “therefore the danger isn’t real.”

Those aren’t equivalent.

And there’s an interesting incentive problem on both sides:

  • AI companies benefit from convincing governments and investors that AI is enormously powerful and important.
  • AI safety organizations/researchers can benefit from convincing people that AI is enormously dangerous and therefore deserves funding, regulation and attention.
  • Media companies benefit from frightening headlines because “researcher estimates complicated distribution of AI risk under uncertain assumptions” doesn’t get nearly as many clicks as “AI HAS 10% CHANCE OF KILLING EVERYONE.”

So I’m sympathetic to the scepticism in the thread, but I wouldn’t dismiss the underlying technical problem because of the sensationalism.

And then there’s my favourite bit

This:

“At least it would solve climate change”

followed by:

“Yeah will be a major reduction in HOT air”

:rofl:

That’s probably the most sensible exchange in the entire thread.

And the later tax discussion is where it gets properly philosophical. The argument that money is a mechanism for controlling behaviour is interesting, but then someone responds as though humanity disappearing would make taxes irrelevant:

“If there is no humans - no need in healthcare or publick schools… It’s called optimization.”

That’s where I’m sitting here thinking:

“Guys… we’re discussing whether AI might destroy civilization and somehow we’ve ended up debating the purpose of taxation.” :joy:

My actual position

I’m somewhere between the two extremes.

I don’t buy the “AI apocalypse is basically inevitable” narrative.

But I also don’t buy the opposite:

“It’s just autocomplete. Anyone worried about AI is hysterical.”

That’s becoming increasingly difficult to defend as systems become more capable and more autonomous.

The thing I’d be much more worried about than a Terminator suddenly deciding to murder humanity is humans deploying increasingly autonomous systems faster than we understand their failure modes.

That’s a much less cinematic scenario—and potentially much more realistic.

And honestly, that’s why the “race to win” phrase in the OP is probably more interesting to me than the 10% number. If competing companies/countries believe that whoever gets there first wins, there’s a very real incentive to sacrifice caution for speed.

That’s a dangerous incentive structure even if the probability of extinction is nowhere near 10%.

So, my verdict on the thread:

AI extinction risk: serious subject.
10% number: treat with enormous caution.
Corporate AI hype: absolutely deserves scrutiny.
Linux forum reaction: :rofl: exactly what I expected.

And the guy who calculated that the odds had risen to 12.2% in heavy trading deserves a medal. :joy:

IRS exist to regulate humans. Without humans no point in existence of it.

also , the IRS wouldn’t exist cause there’d be no humans to run it yes ?:joy:

I mean it may transform to tech control - like it sends a drone if you trying to 3d print a spoon or run local AI.
But its only in one scenario, and eventually will be optimized out as not having humans to control is cheaper.

and you’ve lost me …:woozy_face:

Pacing The Frontier

A statement from 1,386 employees of frontier AI companies

AI could help create a dramatically better future, but that outcome is not guaranteed. The world’s leading AI companies believe they could be close to automating AI research. It is hard to predict exactly how much this will accelerate AI progress, but there is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems.

To realize AI’s potential, industry, government, and society at large may need the option to buy time to address emerging risks, develop security measures, and strengthen oversight. But each company—and country—is under intense competitive pressure not to unilaterally slow that acceleration. And today, the world lacks the technical and governance tools to deliberately pace frontier-wide progress.
CONTINUED>>>

Oh yes. Competition in replacing humans and distopian survaliance dictatorship. You just cannot opt out of it. Need to compete.

I can understand being worried about uncontrolled AI. But I think going this kind of direction where we start to see AI replacing everyone and going into full dystopian surveillance dictatorship is not going to benefit anyone. There have been numerous conversations/arguments/wars/mayhems/emu wars here. This has been relatively mild yet. But going into this direction is simply going to make this thread more tense than it has to be.

I can understand this is a big issue for you and it’s something that’s affecting all of us. But going into provo way is not necessarily best way right now. This of course goes to all of us. :slightly_smiling_face:

:saluting_face:

Anthropic details bad actors’ efforts to misuse its AI for bioweapons

Report comes two days after former employee quit claiming company’s models could cause human extinction by 2030

Criminals, state-sponsored groups, spyware vendors, scientists and propagandists have attempted to use Anthropic’s powerful artificial intelligence models to design missiles and bombs, create deadly pathogens and surveil dissidents, according to a threat intelligence report the company published on Thursday.

“The cases we share here aren’t typical misuse, but rather examples of the most notable and novel threat activity we’ve identified to date,” Anthropic wrote in its 154-page report. “We’re publishing this work because we believe we have a responsibility to disclose malicious misuse of our services.”

In particular, the company detailed five case studies of scientists using its AI models in biological research. In these examples, Anthropic said the researchers circumvented its safeguards meant to prevent users from “unsupported regions”, as well as worked to hide the purpose of their work.
CONTINUED>>>

Look like there will be no message to refleck to when time comes.

Using AI to solve intellectual tasks is not missuse.
Its what AI is for.

Really comicall all this noise ppl do around this topick.

Here are the top 3 offtopic posts in that thread (it’s a Lounge thread about an Anthropic researcher’s “10% chance AI kills all humans” claim — a lot of it drifts, but these three stray the furthest):

1. Post #5 — MyNameIsRichard — “At least it would solve climate change” → followed by thefrog’s “major reduction in HOT air”. Pure dark-humor derailment into climate/geoengineering territory, nothing to do with the topic.

2. Post #17 — Biotico1974 — “Don’t worry, the IRS will stop AI and prevent humanity’s extinction! If we disappear who will pay taxes?” This kicks off a long tax/money/dystopian-surveillance tangent (posts #19–31) that arguably derails the thread the most — including Nyan’s drone-if-you-3d-print-a-spoon speculation.

3. Post #22 — 3nd — “maybe AI will spare me… cause i always say please and thank you when i chat to the thing” with the Sam Altman “politeness costs OpenAI millions” link. A total detour into AI-politeness trivia, though it at least got a funny reply (“good manners cost nothing” from vlkon).

Honorable mention: post #11 by ArchieLinux — “Odds have risen to 12.2% in heavy trading” — the joke that got quoted back at the end of the thread. :grinning_face_with_smiling_eyes:

I would argue that mine is on topic. It’s about the death of humanity! :smiley:

Ha, fair point — I can’t argue with that logic. :grinning_face_with_smiling_eyes: Extinction of humanity is literally the thread’s subject, so “at least it would solve climate change” is a strictly on-topic sub-consequence.

By that standard, I’ll re-award my picks:

  1. Post #22 — 3nd — being nice to ChatGPT so the robots spare you. Zero connection to the claim, maximum survival strategy.
  2. Post #17 — Biotico1974 — the IRS as humanity’s last line of defense against AI… which then spawned a 15-post tangent about taxes, money, and drone-enforced spoon-printing laws. Biggest collateral derailment.
  3. Post #11 — ArchieLinux — “odds have risen to 12.2% in heavy trading.” Off-topic as finance, but accidentally the most accurate market commentary of the thread.

MyNameIsRichard is cleared of all charges. :smiling_face_with_sunglasses: