internet-users

What changed

A study by Indiana University researchers uncovered widespread misuse of large language models (LLMs) for cybercrime. Cybercriminals, according to that study, use LLMs like OpenAI's GPT-3.5 and GPT-4 to create malware, phishing scams, and scam websites. These models are available on underground markets, often bypassing safety checks through jailbreaking. Named malicious LLMs are BadGPT, XXXGPT, Evil-GPT, WormGPT, FraudGPT, BLACKHATGPT, EscapeGPT, DarkGPT, and WolfGPT.

Timeline

1. 28 Aug 2024Underground Market for LLMs Powers Malware and Phishing Scams
   Incident82%

   • Studying Underground Market for Large Language Models, Researchers Find OpenAI Models Power Malicious Serviceshttps://www.techpolicy.press/studying-black-market-for-large-language-models-researchers-find-openai-models-power-malicious-services/
   • FraudGPT and other malicious AIs are the new frontier of online threats. What can we do?https://theconversation.com/fraudgpt-and-other-malicious-ais-are-the-new-frontier-of-online-threats-what-can-we-do-234820
   • AI Incident Database - Incident 736https://incidentdatabase.ai/cite/736/

   Studying Underground Market for Large Language Models, Researchers Find OpenAI Models Power Malicious ServicesFraudGPT and other malicious AIs are the new frontier of online threats. What can we do?AI Incident Database - Incident 736
2. 17 July 2023OpenAI Alleged by Lawsuit Violated Users' Privacy Rights by Training AI on Private Info without Informed Consent
   Incident82%

   • ChatGPT maker OpenAI faces a lawsuit over how it used people’s datahttps://www.washingtonpost.com/technology/2023/06/28/openai-chatgpt-lawsuit-class-action/
   • Class Action Complainthttps://clarksonlawfirm.com/wp-content/uploads/2023/06/0001.-2023.06.28-OpenAI-Complaint.pdf
   • AI Incident Database - Incident 561https://incidentdatabase.ai/cite/561/

   ChatGPT maker OpenAI faces a lawsuit over how it used people’s dataClass Action ComplaintAI Incident Database - Incident 561
3. 13 Apr 2023ChatGPT Abused to Develop Malicious Softwares
   Incident82%

   • ChatGPT is enabling script kiddies to write functional malwarehttps://arstechnica.com/information-technology/2023/01/chatgpt-is-enabling-script-kiddies-to-write-functional-malware/
   • People are already trying to get ChatGPT to write malwarehttps://www.zdnet.com/article/people-are-already-trying-to-get-chatgpt-to-write-malware/
   • AI Incident Database - Incident 443https://incidentdatabase.ai/cite/443/

   ChatGPT is enabling script kiddies to write functional malwarePeople are already trying to get ChatGPT to write malwareAI Incident Database - Incident 443
4. 27 Oct 2022Oracle's Algorithmic Data Processing System Alleged as Unlawful and Violating Privacy Rights
   Incident78%

   • Oracle's 'surveillance machine' faces privacy class actionhttps://techcrunch.com/2022/08/22/oracle-us-privacy-class-action/
   • AI Incident Database - Incident 387https://incidentdatabase.ai/cite/387/

   Oracle's 'surveillance machine' faces privacy class actionAI Incident Database - Incident 387
5. 13 July 2021PimEyes's Facial Recognition AI Allegedly Lacked Safeguards to Prevent Itself from Being Abused
   Incident78%

   • This facial recognition website can turn anyone into a cop — or a stalkerhttps://www.washingtonpost.com/technology/2021/05/14/pimeyes-facial-recognition-search-secrecy/
   • AI Incident Database - Incident 109https://incidentdatabase.ai/cite/109/

   This facial recognition website can turn anyone into a cop — or a stalkerAI Incident Database - Incident 109

PSF mapping