480
AI Models from Google, OpenAI, Anthropic Solve 0% of ‘Hard’ Coding Problems
(analyticsindiamag.com)
this post was submitted on 22 Jun 2025
480 points (99.0% liked)
Programming
21127 readers
370 users here now
Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!
Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.
Hope you enjoy the instance!
Rules
Rules
- Follow the programming.dev instance rules
- Keep content related to programming in some way
- If you're posting long videos try to add in some form of tldr for those who don't want to watch videos
Wormhole
Follow the wormhole through a path of communities !webdev@programming.dev
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Outside of everything else discussed here, this is something I disagree with on a fundamental level, flawed tests are worse than no tests, IMO.
Not to get too deep in to the very contentious space of testing in development, but when it comes to automated testing, I think we're better off with more rigorous^[Validating tests through chaos/mutagen testing; or model verification (e.g. Kani)] testing instead of just chasing test coverage metrics.
I never said you should use flawed tests. You ask AI to write some tests. You READ THEM and probably tweak them a little. You think "this test is basic but better than nothing and it took me 30 seconds. You commit it.
I guess, it would be useful as a kind of fuzzer with a high price to performance ratio.
Making tests to try and find vulnerabilities.
For normal functionality testing though, better off making it yourself.