Malicious AI injection isn't cute

Today I noticed a LinkedIn post from James Gosling. Here is what he said in comment on the Ars article: "On the one hand, this stunt is overdone and somewhat childish. It could have been far worse. But it is also hilarious. The part I don’t get is that in the furor, where’s the outrage at the lax security in AI? Unauthorized prompt injection should not be possible. After decades of fighting to build safe computer systems, these tools are far too casual."

Maliciously injecting code to delete files into your repo is not childish or hilarious. It is malicious and should be illegal. It is no different from writing a virus into your code that fucks with a users machine.

That "we only fuck with AI users" in no way makes it okay.

Further, this response by James in not acceptable. The way LLMs work there is no definitive way at all to prevent injection. We've seen it time and again. No matter what is done it cannot be made safe.

Encouraging asshats to poison the world to ruin AI is not cute. It's not funny. It's fucked up and unacceptable. I agree actually that AI is bad overall and that we should get rid of it, but poisoning your source code is NOT an acceptable solution.

It's concerning that James, who I otherwise would expect is an intelligent individual, is implying that AI developers are lax in security for their systems. That's fucking ridiculous and he should know better than to say that. Sure some don't do as much as they could, but there is no fix for this. The way LLMs work all you can do is make it more difficult to inject.

LLM security isn't lax overall. It's very restricted. It just can't actually be locked down because of how LLMs worked. Shame on you James for implying anything else.