- New research shows that AI tools are being used and abused by cybercriminals
- Computer pirates are creating tools that exploit legitimate LLM
- Criminals are also training their own LLM
It is undeniable that AI is being used by cybersecurity and cybercriminal equipment, but Cisco Talos’s new investigation reveals that criminals are becoming creative. The last development in the IA/Cybersecurity panorama is that the LLM ‘uncensored’, the Jailbreak and the LLMs designed by cybercrime are taking advantage of the objectives.
It was recently revealed that Grok and Mistral AI models were promoting Wormgpt variants that generated malicious code, social engineering attacks and even providing piracy tutorials, so it is clearly becoming a popular tactic.
The LLMs are built with safety and rail characteristics, ensuring a bias and minimum exits that consist of human values and ethics, as well as to ensure that chatbots do not participate in harmful behavior, such as creating phishing malware or emails, but there is work around.
Jailbroken and uncensored
The so -called uncensored LLM observed in this investigation are versions of the AI models that operate outside the normal limitations. This means that they can perform tasks for criminals and create harmful content. These are quite easy to find, according to the investigation, and are easy to execute, with only relatively simple indications required.
Some criminals have gone one step further, creating their own LLM, such as Wormgpt, Fraudgpt and Darkgpt. These are marketed for bad actors and have a lot of dire features. For example, FraudGPT claims to create automatic scripts to replicate records/cookies, write pages/letters of fraud, find leaks and vulnerabilities, and even learn to code/hack.
Others navigate around the security characteristics of legitimate AI models through chatbots ‘jailbreaking’. This can be done using ‘obfusion techniques’, which include base coding64/ROT-13, using different languages, “L33t SP34K”, emojis and even Morse code.
“As IA technology continues to develop, Cisco Talos expects cybercriminal improving family attacks.
