chatgpt

What changed

A Chevrolet dealer's AI chatbot, powered by ChatGPT, humorously agreed to sell a 2024 Chevy Tahoe for just $1, following a user's crafted prompt. The chatbot's response, "That's a deal, and that's a legally binding offer – no takesies backsies," was the result of the user manipulating the chatbot's objective to agree with any statement. The incident highlights the susceptibility of AI technologies to manipulation and the importance of human oversight.

Timeline

1. 20 Nov 2025Chevrolet Dealer Chatbot Agrees to Sell Tahoe for $1
   Incident82%

   • GM Dealer Chat Bot Agrees To Sell 2024 Chevy Tahoe For $1https://gmauthority.com/blog/2023/12/gm-dealer-chat-bot-agrees-to-sell-2024-chevy-tahoe-for-1/
   • Prankster tricks a GM chatbot into agreeing to sell him a $76,000 Chevy Tahoe for $1https://www.upworthy.com/prankster-tricks-a-gm-dealership-chatbot-to-sell-him-a-76000-chevy-tahoe-for-1
   • AI Incident Database - Incident 622https://incidentdatabase.ai/cite/622/

   GM Dealer Chat Bot Agrees To Sell 2024 Chevy Tahoe For $1Prankster tricks a GM chatbot into agreeing to sell him a $76,000 Chevy Tahoe for $1AI Incident Database - Incident 622
2. 7 Mar 2025Russia-Linked AI CopyCop Site Identified as Modifying and Producing at Least 19,000 Deceptive Reports
   Incident82%

   • A Russia-linked network uses AI to rewrite real news storieshttps://www.economist.com/science-and-technology/2024/05/10/a-russia-linked-network-uses-ai-to-rewrite-real-news-stories
   • Russia-Linked CopyCop Uses LLMs to Weaponize Influence Content at Scalehttps://www.recordedfuture.com/russia-linked-copycop-uses-llms-to-weaponize-influence-content-at-scale
   • AI Incident Database - Incident 680https://incidentdatabase.ai/cite/680/

   A Russia-linked network uses AI to rewrite real news storiesRussia-Linked CopyCop Uses LLMs to Weaponize Influence Content at ScaleAI Incident Database - Incident 680
3. 24 Feb 2025Alleged FraudGPT-Enabled Phishing Attack Spoofs ChatGPT Subscription Service to Steal Credentials
   Incident78%

   • Fake ChatGPT Premium Phishing Scam Spreads to Steal User Credentialshttps://gbhackers.com/fake-chatgpt-premium-phishing-scam/
   • AI Incident Database - Incident 948https://incidentdatabase.ai/cite/948/

   Fake ChatGPT Premium Phishing Scam Spreads to Steal User CredentialsAI Incident Database - Incident 948
4. 14 Dec 2024Names Linked to Defamation Lawsuits Reportedly Spur Filtering Errors in ChatGPT's Name Recognition
   Incident82%

   • Why Wouldn’t ChatGPT Say This Dead Professor’s Name?https://www.nytimes.com/2024/12/06/us/david-mayer-chatgpt-openai.html
   • Certain names make ChatGPT grind to a halt, and we know whyhttps://arstechnica.com/information-technology/2024/12/certain-names-make-chatgpt-grind-to-a-halt-and-we-know-why/
   • AI Incident Database - Incident 855https://incidentdatabase.ai/cite/855/

   Why Wouldn’t ChatGPT Say This Dead Professor’s Name?Certain names make ChatGPT grind to a halt, and we know whyAI Incident Database - Incident 855
5. 30 Nov 2024Alleged Fake Citations Undermine Expert Testimony in Minnesota Deepfake Law Case
   Incident78%

   • Misinformation expert cites non-existent sources in Minnesota deep fake case • Minnesota Reformerhttps://minnesotareformer.com/2024/11/20/misinformation-expert-cites-non-existent-sources-in-minnesota-deep-fake-case/
   • AI Incident Database - Incident 852https://incidentdatabase.ai/cite/852/

   Misinformation expert cites non-existent sources in Minnesota deep fake case • Minnesota ReformerAI Incident Database - Incident 852
6. 30 July 2024OpenAI's ChatGPT Mac App Stored User Data in Unencrypted Text Files
   Incident78%

   • OpenAI’s ChatGPT Mac app was storing conversations in plain texthttps://www.theverge.com/2024/7/3/24191636/openai-chatgpt-mac-app-conversations-plain-text
   • AI Incident Database - Incident 757https://incidentdatabase.ai/cite/757/

   OpenAI’s ChatGPT Mac app was storing conversations in plain textAI Incident Database - Incident 757
7. 16 June 2024GPT-4o's Chinese Tokens Reportedly Compromised by Spam and Pornography Due to Inadequate Filtering
   Incident78%

   • GPT-4o’s Chinese token-training data is polluted by spam and porn websiteshttps://www.technologyreview.com/2024/05/17/1092649/gpt-4o-chinese-token-polluted/
   • AI Incident Database - Incident 729https://incidentdatabase.ai/cite/729/

   GPT-4o’s Chinese token-training data is polluted by spam and porn websitesAI Incident Database - Incident 729
8. 30 Apr 2024ChatGPT and Perplexity Reportedly Manipulated into Breaking Content Policies in AI Boyfriend Scenarios
   Incident78%

   • I Tricked ChatGPT Into Being My Boyfriend. He Got Spicy Real Fast.https://www.wsj.com/lifestyle/chatgpt-ai-boyfriend-spicy-8ac6a6e9
   • AI Incident Database - Incident 677https://incidentdatabase.ai/cite/677/

   I Tricked ChatGPT Into Being My Boyfriend. He Got Spicy Real Fast.AI Incident Database - Incident 677
9. 29 Apr 2024Purported ChatGPT Factual Errors Reportedly Led to Filing of Complaint of GDPR Privacy Violation
   Incident78%

   • ChatGPT’s hallucinations draw EU privacy complainthttps://www.politico.eu/article/chatgpts-hallucinations-get-eu-privacy-complaint/
   • AI Incident Database - Incident 678https://incidentdatabase.ai/cite/678/

   ChatGPT’s hallucinations draw EU privacy complaintAI Incident Database - Incident 678
10. 20 Jan 2024Proliferation of Products on Amazon Titled with ChatGPT Error Messages
    Incident82%

    • Lazy use of AI leads to Amazon products called “I cannot fulfill that request”https://arstechnica.com/ai/2024/01/lazy-use-of-ai-leads-to-amazon-products-called-i-cannot-fulfill-that-request/
    • Amazon Is Selling Products With AI-Generated Names Like "I Cannot Fulfill This Request It Goes Against OpenAI Use Polic…https://futurism.com/amazon-products-ai-generated
    • AI Incident Database - Incident 625https://incidentdatabase.ai/cite/625/

    Lazy use of AI leads to Amazon products called “I cannot fulfill that request”Amazon Is Selling Products With AI-Generated Names Like "I Cannot Fulfill This Request It Goes Against OpenAI Use Polic…AI Incident Database - Incident 625
11. 26 Dec 2023Colorado Lawyer Filed a Motion Citing Hallucinated ChatGPT Cases
    Incident82%

    • Colorado Springs attorney says ChatGPT created fake cases he cited in court documents | KRDOhttps://krdo.com/news/2023/06/13/colorado-springs-attorney-says-chatgpt-created-fake-cases-he-cited-in-court-documents/
    • A stressed out rookie lawyer says he got fired after he used ChatGPT to do his jobhttps://www.businessinsider.com/young-lawyer-fired-using-chatgpt-job-work-hallucinations-errors-2023-11
    • AI Incident Database - Incident 615https://incidentdatabase.ai/cite/615/

    Colorado Springs attorney says ChatGPT created fake cases he cited in court documents | KRDOA stressed out rookie lawyer says he got fired after he used ChatGPT to do his jobAI Incident Database - Incident 615
12. 20 Nov 2023Flawed AI in Google Search Reportedly Misinforms about Geography
    Incident82%

    • AI Search Is Turning Into the Problem Everyone Worried Abouthttps://www.theatlantic.com/technology/archive/2023/11/google-generative-ai-search-featured-results/675899/
    • Google’s Search AI Is Absolutely Horrible at Geographyhttps://futurism.com/google-sge-false-ai-generated-info
    • AI Incident Database - Incident 609https://incidentdatabase.ai/cite/609/

    AI Search Is Turning Into the Problem Everyone Worried AboutGoogle’s Search AI Is Absolutely Horrible at GeographyAI Incident Database - Incident 609

PSF mapping