alcamar
12155
1223
112
Even this won't stop them.
Sauce: https://www.pcgamer.com/a-us-air-force-ai-drone-simulation-ended-with-the-drone-going-rogue-and-killing-its-simulated-human-operator-because-the-operator-wouldnt-let-it-kill-everything-it-wanted-to/
lindabelchersfirstcousin
vissago
baby skynet!
admiralawesome4213
me too little ai drone, me too...
68mano
More fear-mongering.
JustFeedMePieDammit
You would not believe you eyes, when 10 million pissed AIs, drone strike the world while we’re asleep
Drix1942
♪ 'Cause they'd fill the open air and leave corpses everywhere ♫
♪ You'd think me brave but I would just shake and die ♫
andydicktracymorganfairchild
That's better.
BryanTenn
We are a virus on the planet. A free thinking A.I. will quickly see that.
SalmonMax
Seems like a sufficiently flexible AI model works a little like a Wish spell. It's capable of amazing feats, but you gotta be REAL careful with your wording.
fknbastard
Ai should be programmed to recognize it needs refueling and the side steps will result in grounding upon return but the real problem is ai will always look for the easiest way to succeed even if it is prejudice, biased, or just cheating. If you don’t set parameters, don’t expect the results you intended.
Maaatth
Here's a vid of the exoskeleton testings:
v
xenjamin
What movie is this?
Maaatth
RoboCop 2. This clip shows the (obviously failed) attempts at developing successors for Murphy.
ps238principal
Maybe the operator had been leaking classified info on the World of Tanks Discord and it took action?
5amNonsense
This whole thread got very Ace Combat 7. Which I suppose was to be expected, given the premise
LupusLilium
Trigger was framed!
ObadiahRain
This did NOT happen.
YouWillNeverFindMe
Instruction to kill obstructions, and the "controller" got the same label?
IrateWolfe
It didn't actually happen, though, this isn't a true story
BoboTheIceMan
Didn't humans do that too, then learned not to due to punishment?
KimberlynnKelly
VindictiveBathToaster
They amended this headline saying this didn’t happen.
JeffGodofBiscuits
Some General, somewhere...
Lethous
Sounds about write, generals ideas are usually stupid
JustFeedMePieDammit
GravyEducation
Kill it kill it before it escapes the lab
EnthusiasticPickle
Enters code "kill all the bad guys". *Drone turns around*. US Military "Wait! What are you doing!!!""
dashcan
GravyEducation
WhyAreThereSixPedalsIfThereAreOnlyFourDirections
Crimx42
Who put Bing in charge of the AI? Oh right, lowest bidder
mandy009
if we go way back to the root of it, it was Bill Gates in the Windows lawsuit, with the monopoly power... 90's people remember
nclu
One of my comp. sci classmates was an ex military guy. He was adamant that we did not want mil having any part of AI research. "The military is incredibly good at one thing: killing people. You don't want their expertise imbued in AI"
ToSisPoS
Two things: killing and spending.
AlwaysUpvotesCatgirls
FITARA usually is best value, not necessarily lowest bidder. But even that has a bunch of exemptions.
VeinyFleshTrumpet
I quote my cousin on this who was military before I joined: “the important thing to remember about military hardware is that it’s chosen in terms of whatever is best, it’s chosen in terms of whatever is cheapest.”
SwankyPotato
It was Clippy
CardeasIV
All along
KafkasKarma
It's always the ones you suspect the most.
JustFeedMePieDammit
“Hi! It looks like you’re trying to commit a war crime! Would you like help with that?” - Future Clippy
simplefishy
Sticklebrickk
I'll fucking do it again.
koflan
Thank goodness we have that bastion of hard-hitting journalistic integrity, pcgamer.com.
bitterpilltoswallow
my favorite part of an article I read about this was that after they told the AI that it would lose points if it killed it's human operator, it then went on to destroy the communications tower, so that the human-operator couldn't stop it from completing it's mission.
TheChunguskaEvent
but can it render hands
MathiasGodin
ok, I laughed a little
OneUniverse
Combine AI and genome splicing and you can have hands coming out your ass. Literally.
NuclearMonk
UserNamesArentEasy
These days, yes it can.
notacobra
This story is a completely nonsensical anecdote from a "literal who" that makes a claim about an AI that goes completely against the way AI, machine learning and learning algorithms actually function. It's just pure fiction, from the ground up, spread as hand wavey scary stories.
notacobra
An AIs job isn't to destroy a missile silo at any cost using any means it can determine. It's job is to either: A) guide a missile towards an intended target using provided input. B) gather info and provide input on distance/elevation/etc to send to (a) or C) playing "match the picture" to tell (b) what is and isn't a middle silo. No AI currently does all three and even if it could, it doesn't "identify impediments" like an operator telling it no. That's literally SciFi nonsense.
TheMayorOfTittyCity
It does say the "literal who" (I'm not sure what that means) is the Head of AI for the Air Force. I know nothing about this, but that would seem to me like that's someone that knows more than me and probably (hopefully) everyone here about the subject.
TheMayorOfTittyCity
No? Someone disagrees with that? Can you explain why instead of just downvoting? As I said, I know nothing about this, I am going to believe someone who knows more than me.
GendouLovesDucklings
Cinco Hamilton is a literal who? https://www.defenceiq.com/air-forces-military-aircraft/articles/interview-with-director-of-the-department-of-the-airforce-artificial-intelligence
thesmelge
There's 5 Hamiltons?
GendouLovesDucklings
Since he very much made the claims: https://www.aerosociety.com/news/highlights-from-the-raes-future-combat-air-space-capabilities-summit/
crackblind
Anyone watching "Class of '09" on Hulu? This happened on the most recent episode.
Exdeath5000
Let's keep pouring billions into AI... Why? Because we have to, because other countries are!
Thajurriesexcuesed
Like any major new technology there will be tradeoffs. Some good, some bad.
Cebrail
*Facepalm* - Why keep spending money on developing cars, they are unsafe. Why spend money on energy, it can blow up. Bla bla bla. AI is a neccessary part of humanity's future, it's the only way we can technologically evolve fast enough to outpace how quickly we are destroying everything. WE just need to find a way to do it safely.
azarza
we 100% want AI. there is a great deal of data left on the table because no one has the ability/manpower to make it functional
TheNLK
Pretty much. Just don't start placing it into dangerous machines and we'll be ok. My concern would be cyber security, but only because so many corporations and government entities take it lightly.
azarza
i mean.. the simple solution here is 'don't give the AI the ability to kill people'. i don't get why that's the first place we have gone
azarza
to rant off a coffee and a magical cigarette, if we did this right out the gate. Laws laid out to protect AI, personal AI rights, laid out expectations, laid out worries, treat the AI like a legal protected entity, this could work
Tminus1622d
*I dont believe you gif* Also, drones were probably first used with no oversight, just making all their kill decisions on their own, in 2018 so the concept of a human operator is kinda moot.
Tminus1622d
Apologies, I seem to have gotten the year wrong, it was 2020, not 2018. 2018 is when the Vatican spoke out against Lethal Autonomous Weapons.
Tminus1622d
example from 2020-2021: https://www.independent.co.uk/tech/drone-fully-automated-military-kill-b1856815.html
whiskinputwarbles
No. "Air Force spokesperson Ann Stefanek denied that any such simulation has taken place. "The Department of the Air Force has not conducted any such AI-drone simulations and remains committed to ethical and responsible use of AI technology," Stefanek said. "It appears the colonel's comments were taken out of context and were meant to be anecdotal."
whitebbwolf
yeah sure
languex
Exactly what Skynet would say!
whiskinputwarbles
https://www.businessinsider.com/ai-powered-drone-tried-killing-its-operator-in-military-simulation-2023-6
softsuit
This is called getting out in front of the story.
Wormfood07
https://amp.theguardian.com/us-news/2023/jun/01/us-military-drone-ai-killed-operator-simulated-test
OneUniverse
Has it paid anybody outside the department to pay another company, who subcontracted to another company to do this? So they can deny it was them?
JayEnfield
The whole thing just reeks of tech-journalist bullshit in the first place.
sms001
The Colonel's walk-back isn't terribly reassuring: 'We've never run that experiment, nor would we need to in order to realise that this is a plausible outcome.'
breadedfishstrip
My "Our drones aren't trying to kill us" shirt has people asking questions already answered by the shirt
DumbB1tch
Whatchu mean
TheMayorOfTittyCity
Uh, that just makes it confusing. "Anecdotal" in no way means its not true, it just means it was one persons experience, which is what was being described. No one denies its his anecdote. But it appears to be his anecdote of a simulation he ran where the drone killed the operator and then the communication towers, which is the thing the article is about and is what we are all concerned about.
10001110101periodictablewithasantapieceofmind
The key part of this headline is "Because the operator wouldnt let it kill everything it wanted to". In order to prove that the reasons the AI killed its operator where specifically vindictive or conniving youd have to prove this AI was incredibly more intelligent than it was built to be, for no reason. Think of that headline "Chess Robot Grabs And Breaks Finger Of Seven-year-old Opponent" it makes you think the robot did it out of frustration or malice but its incapable of such things.
Dascari
The report got updated. The initial briefing/report was on a thought experiment considering potential loopholes and logical escapes that an AI could take to go rogue. The simulation was never performed, physically or digitally.
Z0op
As if the first self driving ai’s didnt run over ppl in simulations in an attempt to evade something else. Whether er should take this anecdote at face value or not, it doesnt matter, its sensationalism and nothing new happened here
flarflarf
My understanding is that this was just poor reporting. The "simulation" in question was just a thought experiment.
MaverickEsq
It looks more to be a thought experiment, we're not at the level of a general AI capable of evaluating options not initially presented to it as this would require the AI to have a model of reality and how these things interact. Drones don't kill their operators to have less control because drones are, unless specifically programmed to be, unaware that the target designated "no kill - operator" has anything to do with their approval strategy nor would they think removing the aprover would = yes.
MaverickEsq
So its "anecdotal" in that it is an anecdote about a real technology (AI Drones) the content of the event is notional.
potshot
Not even that, more of a "it's an anecdote of a hypothetical scenario with technology that doesn't even exist yet."
TheMayorOfTittyCity
If the head of the US Air Force's AI Test and Operations says I ran a test and this happened, and a spokes person says that was "anecdotal", that doesn't seem to in anyway refute claim that the AI head made. If I said I ran a program on my computer and it crashed, and someone said "that's an anecdote", I would say, well yeah, that's what I just said?
Altanese
Let's not pretend that the spin in this isn't 'LOOK AT THE SCARY ROBOTS, CLICK HERE FOR MORE'
TheMayorOfTittyCity
If you read the original article from the Royal Aeronautical Society, its basically just a more detailed version of this. I am not sure how much hysteria inducing spin the British Royal Society is putting on this. They dont really seem like a 'click-baitey type' of historical prestigious and illustrious scientific institution.
Altanese
We're talking about PC Gamer and the article seen in the post.
LoudBirb
That colonel later admitted he misspoke and that he never ran that scenario, it was a thought experiment. So... no, it didn't actually happen. Here's the source, the correction is at the very bottom: https://www.aerosociety.com/news/highlights-from-the-raes-future-combat-air-space-capabilities-summit/
TheMayorOfTittyCity
That would make much more sense and would really be the only way to reconcile both of their statements. It would have been more clear if the PR person said it was "hypothetical", as an anecdote can still be a real event
LoudBirb
Oh, yeah, the PR person definitely either goofed hard on their wording or tried to explain it away before the Colonel could correct himself.
brickius
Sounds more like it had trouble telling friend from foe and was identifying friendly targets as strike targets.... Then someone spun a story to make it sound like something else
Delathi
Or it was poorly programmed and wasn't given negative point values for doing bad stuff.
GlennBecksChalkboard
ArkoneAxon
It sounds like AI is getting REALLY good at rules lawyering.
LupusLilium
'Can't deliver an override order if you can't communicate with me, haha!' - Simulated EDI mini.
BigJT
What happened was it was given "points" for hitting targets but could be denied permission to strike by a friendly commander. It realised that if it struck commander denying it permission, it could strike all the potential targets. After they changed the program so that striking a friendly cost points, it struck the communications equipment instead of the commander
AeroEngineer
No, it was a thought-experiment of how AI could backfire/work-around is programming, misrepresented as a simulation.
AeroEngineer
/gallery/ifpsR6I/comment/2329848153
brickius
But did it? That's quite a leap, like, it would actually have to be able to identify where it's commander was, and that it could stop the orders by removing them..... A big step in logic, plus, striking the communications equipment is also a friendly, you recon it figured that out? I smell bullshit
Borinax
it would need the equipment to be registered as a "friendly" for that to matter. An easy oversight. Still quite a logic leap, but feasible depending how it's coded. If it's designed to target those for hostiles it could apply it back to friendlies in theory, and equipment could easily not be flagged for it.
acrabb3
I'd guess it's an evolutionary algorithm, which basically works by button mashing, seeing how many points it got, and trying again with a slightly different button mash to see if that's better. Eventually it will "realise" that inputs that bomb a specific location get more points than ones that don't, and the people running the simulation have to tweak the rules so undesirable behaviour is penalised.
acrabb3
Here's a pretty big paper on ways AI have "cheated" by finding holes in their simulation's rules: https://arxiv.org/abs/1803.03453v1 many of which have similar "realisations" in.
Fairemont
In a simulation, it's using limited scope, so a program realizing another portion of the program is denying it from doing its objective can be removed, and then removing it is not particularly far-fetched. The commander likely registers as a potential target and had zero point value, so it was worth destroying to get more points.
atomicsockmonkey
This right here. It's like finding an exploit in a video game.