I minus the A ?

I know skepticism about AI and "2+2=5" style AI stories abound and I remain a "fan with tons of huge doubts". But I have to say this little experiment is making it harder. Here's what happened:

We have a thing where you can type in someone's name and get back any recent news about them that might influence your willingness to do business with them. For example type "Hank Green Vancouver crime stories since 2025" and get back a link to a CBC story about Hank having robbed a bank. Maybe you decide not to hire him after all.

Systems to do these things exist and we offer it via a third party supplier, who does a fairly bad job. So we were wondering if AI could do better.

I was pretty surprised that Gemini, Perplexity and Deepseek all consistently came up with totally fabricated stories. I was even more surprised at Gemini's explanation:

"Gemini is a "next-word predictor." If it doesn't find a news story in its training data, it doesn't say "I don't know." It simply predicts what a news story about that person might look like."

"When an AI generates a URL, it isn't "finding" a link; it's imagining what a URL for that story should look like (e.g., https://bbc.com/news/person-name-event). These links almost never exist in reality."

"You need to tell the model that its job is to be a librarian, not a storyteller. Use a System Instruction like this: You are a factual news aggregator. Use ONLY the provided search results. If no recent news exists for the person, state 'No recent news found.' DO NOT invent stories. If you cannot provide a verified URL, do not provide a link at all."

Yeah, I knew AI is fundamentally a next-word predictor but I had been operating under the assumption that it predicted the next word based on FACTS, not what a response "might look like".

Needless to say, many experiments to adjust the prompt and also use a feature called 'Grounding' failed to make a meaningful difference.

It's not about making "honest" mistakes. It's about INTENTIONALLY MAKING SHIT UP as a normal operating mode, not an edge case.

WHAT. THE. F ---. 

Comments

Post a Comment

Popular posts from this blog

Change of Venue

Well, I have my doubts that ANYBODY will read this, but here I am: my old blog at progressdeveloper.wordpress.com has become "read only" for reasons that are beyond my control and that I will not bore you with. I might try to scrape it all and stick it somewhere else, but I doubt it. I will probably just let it live in the never-never land of 99.999999% useless information content until someone at Wordpress decides to retire it. (OK, I will bore you just a little. No, I did not lose the password. But they won't let me in anymore because I don't have access to it's registered email address anymore. I am annoyed at Wordpress, enough to try this platform instead [it's part of Google], but not enough to rant about it. These things happen).
Read more

Arrived

 Touchdown in YZP, quick shopping trip in a small Super Valu adjacent to YZP, 20 minute ferry ride (we were 2nd last car on - ferry is tiny and only runs 1x/hour). AirBnb is about 30 seconds drive from Skidegate ferry and is super nice. 3 BR and I think just 1 bath. Huge kitchen. Great view. I think we will be very comfortable. David H is, as expected, unbelievably pleasant and quite fun, but on top of that, seems to know about all sorts of topics that I barely know about and interest me to a greater or lesser extent. How UK's time in India ended. India v Pakistan. Supreme Court in the UK. Whether judges should be elected. He has lived in Africa. He also is super into art as both an observer and a creator. I think the trip will be at least quite good based solely on his involvement!
Read more

Haida Gwaii Part 2

Here I am at YVR Gate C32, waiting for our flight to Sandspit. We're spending 9 nights (7 on HG, one on a ferry, and one in Port Hardy); we being my oldest sister Janet, and a family friend, David (yet another David! Good grief). As you might recall, I went to HG with ex-work-David in May or so of last year. On that trip we were only on the north (Graham) island. I wasn't able to accompany ex-work-David on his zodiac tour of the south (Moresby) island, where, apparently, resides all the "indigenous stuff". (The north island is pleasant but not all that different from "our" gulf islands, at least in my opinion). So I'm going back to fill in the gap of my HG experience with the south island. I'm not hugely interested in Haida art (I think David is, and Janet for sure is), but I'm somewhat interested in FN topics and trying to do my bit to improve my knowledge / empathy towards "them".
Read more