Support is wild in an age of AI
A developer recounts bizarre support requests triggered by AI hallucinations, including a user asking for a feature that never existed based on a fake screenshot from ChatGPT, and MS Copilot inventing non-existent transformation steps. These incidents highlight the risks of customers relying on AI for technical support.
Article intelligence
Key points
- A customer requested a feature that never existed, backed by a hallucinated screenshot from ChatGPT.
- MS Copilot fabricated three out of four transformation steps when asked about one-hot encoding.
- Customers increasingly turn to AI for support, risking misinformation that looks credible.
Why it matters
This matters because a customer requested a feature that never existed, backed by a hallucinated screenshot from ChatGPT.
Technical impact
May affect model selection, inference cost, product capability, and evaluation benchmarks.
Recently a customer emailed me:
Please consider re adding the option to convert from one time zone to another as was available in [Easy Data Transform] version 1.x. See the attached screen dump.
Easy Data Transform has never had a time zone conversion feature, so that is a bit strange. Although . But the screenshot really set off alarms bells as, despite saying “Easy Data Transform v1.11.2” in the title bar, that is not our software!
I thought that someone was trying to pass of their product as ours and did a search. But I couldn’t find any reference to another piece of software called “Easy Data Transform”. I emailed the customer to ask where they had got the software from. The reply came back:
The screenshot was from a ChatGPT output – maybe hallucinating.
Wow. Not only had ChatGPT hallucinated the feature, but also a fever dream screenshot of the user interface, with the non-existent feature. It looks like a real screenshot at first glance, but the icons are a giveaway if you look closer.
For reference, this is what the actual user interface looks like:
ChatGPT has got some of the input types, transforms and menus correct. But otherwise it looks quite different.
I had another brush with AI hallucinations when I asked MS Copilot how to perform ‘one hot encoding’ in Easy Data Transform. It came back with a very plausible and confident sounding answer, including this summary of the transforms required:
Just one problem – only 1 of these 4 transforms actually exists (Split Col). It hallucinated the other 3!
Customers are increasingly typing questions into AIs, rather than reading documentation or asking on a forum. That is good news if it means that the customer gets a quick and accurate answer without troubling busy developers. But it is very bad news if they are getting incorrect answers, especially when these answers look plausible and are confidently presented. It is galling enough that AIs are stealing all our web traffic, without them giving our customers bad support advice as well! Wild times are ahead.
Ps/ Time zone conversion and one hot encoding are now available in the latest version of Easy Data Transform.
More
Share on Tumblr
Email a link to a friend (Opens in new window) Email