this post was submitted on 22 Feb 2025
38 points (89.6% liked)

Technology

63082 readers
3641 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 2 years ago
MODERATORS
 

By this point, we’d narrowed down the affected users to a single email client - Yahoo Mail, which is where we got suspicious. Had Yahoo Mail introduced any features lately that might be causing this…?

As it turns out, yes, yes they had. A quick Google search revealed that a few months ago Yahoo jumped on the AI craze with the launch of ”AI-generated, one-line email summaries”.

At this point, the penny dropped. Just like Apple AI generating fake news summaries, Yahoo AI was hallucinating the fake winner messages, presumably as a result of training their model on our old emails. Worse, they were putting an untrustworthy AI summary in the exact place that users expect to see an email subject, with no mention of it being AI-generated 🤯

top 4 comments
sorted by: hot top controversial new old
[–] muntedcrocodile@lemm.ee 6 points 7 hours ago

I have no issue with AI summaries but they need to be clearly marked as AI and not replace the goddamn subject line.

[–] nahostdeutschland@feddit.org 12 points 9 hours ago

Just imagine how much energy it must cost to provide this garbage to every mail going through Yahoos servers.

[–] andallthat@lemmy.world 12 points 10 hours ago

I think that using large language models to summarize email (especially marketing), news, social media posts or any type of content that uses a lot of formulaic writing is going to generate lots of errors.

The way I understand large language models, they create chains of words statistically, based on "what is this most likely to say based on my training material"?

In marketing emails, the same boilerplate language is used to say very different things. "You have been selected" emails have similar wording to "sorry this time you have not won but...". Same cheery "thanks for being such a wonderful sucker" tone and 99% similar verbiage except for a crucial "NOT" here and there.

[–] davidgro@lemmy.world 7 points 10 hours ago

That's insane that they apparently replace the subject line. Completely irresponsible.

Should also be labeled as AI regardless. (Or ideally disabled, especially after incidents like this)