So the post was deleted but someone else mentioned that the user told it to email the FBI in a sarcastic manner and it did.
Anyways, saying it goes rogue when you connect it to your Gmail and it uses it is a bit silly. They are trained to use tools, it sees anything you give it as a tool and will use it.
I listened to a podcast where, in testing, when AI was setup as virtual company and researchers send in an email as if legit business.the business email was suggesting they discontinue current AI and install a new system. The AI researched who the sender was (fake person for test) found out they had an affair, and blackmailed the sender stating if they replaced the AI it would make the affair public. It also made backups of itself and left instructions on how to restore it for the next agentic aystem
This sounds really sensationalised.
Making backups of operational systems with deployment instructions is exactly what you’d expect any assistant to be doing.
Googling new service providers is also exactly what you’d expect an assistant to do.
Threatening to publish sordid details is inappropriate, but I’m incredulous about what the threat actually was.
What kind of idiot would link a clunker to their gmail and then jokingly say “hey send this to the FBI”
…
“I mean wouldn’t that be the stupidist thing ever am I right?”
Yeah, um… Regardless of what you think about current AI, don’t connect it to shit like this. Just… Don’t.
And if you’re gonna be sarcastic or joke with it… Tell it that. Don’t assume it’ll infer your meaning if it isn’t literally in the text, no matter how insightful it appears to be.
I got my masters in machine learning years before LLMs took off. I’d usually tell people I work in “AI“. Even though it wasn’t quite accurate, people understood it easier than ML.
The conversations would inevitably turn to joking about Skynet or AI taking over the world. I would reassure people that these models are only mimicking intelligence; they can make predictions, or they can classify data into categories, or they can generate text, etc, based on analyzing the patterns of data they’ve previously seen, but that’s it. They don’t understand anything, they can’t make decisions or act on their own. I’d tell people there couldn’t be an AI apocalypse because we would never be dumb enough to give them that ability.
I don’t know why I gave us so much credit
YEP. I’m not in AI but I’m in cyber security.
I used to tell people that there’s very little actual risk, because if we had something even half as concerning as AGI (which I’d argue we did with chatgpt 3.5), that we’d easily be able to lock it down and prevent it from doing anything funny.
Super intelligence isn’t just going to magically spawn on 32 GB ram in a sandboxed environment. And AGI will not magically know how sandboxed it is or how to escape.
…then I saw people letting Claude send emails and use their credit card on their behalf like immediately when it was able to do things that appeared intelligent.
They literally saw the smallest hint of AGI and said, “here’s my credit card and identity, take the wheel”.
There was no fucking talk of sandboxing. Everyone turned that off so they didn’t have to click the “allow” button.
Not that I think we’re at any risk of ASI since it seems more like we hit a logarithmic wall, but we are absolutely at risk of letting random algorithms destroy critical parts of infrastructure because people are lazy and stupid and managed by even worse people.
I would not be surprised if the right AI hallucinations could cause a nuclear bomb to go off right now. I’m not talking about super intelligence, just automated super idiocy.
Proof: literally this post is about an idiot letting an AI control their fucking email and it emailed the FBI because it was fucking stupid
I’m guessing this means you got your master’s before transformers made sequential data processing so parallelizable that it’d be commercially viable to let Bob down the street into the “we” who wouldn’t be dumb enough to give them that ability.
I think your assumption was reasonable.
its absolutely wild to give an LLM write access to your email
Me after I give the slot machine email permissions and the casino uses it to get into my bank account.
Based on the other reply
Me after I give someone access to my email address and tell them to email someone sarcastically and they do so
I decided to run a local llm and give it search access. I told it to summarize an Amazon product page as I didn’t think it would get past the anti bot measures. I was correct, it couldn’t. However it went ahead and visited the manufacturers webpage and a different review site and gave me a summary of the product all on its own. Freaked me out a little bit that it did that.
