<- Back
Comments (147)
- OtherShrezzingThis article feels like the result of having unlimited tokens at your disposal. It’s got some very expensive practices, where near-free practices would produce the same or better results.There’s a long section in the middle where the article describes per-change verification via skill files and “ask Claude to use the chrome-for-Claude tool” when checking if changes have introduced regressions.But there’s already tools which do this without spending tokens tokens. Playwright will achieve equal quality but deterministic results across the same verification attempts. It is write-once-use-often, so has a marginal cost near zero.
- holodukeWhat we need are ultra fast testing suites. Specially e2e tests are notoriously slow. Runtime is big and its often difficult to isolate just a few tests. I want 1000 e2e tests in a few seconds instead of 10 minutes. It's also a mayor blocker for an agent that cannot work fast because of the constant waiting. At the same time you need the test feedback loop in order for the LLM to do proper shotgun programming.
- tshaddoxIs Claude Code considered to be a reliable piece of software these days?I stopped using as my daily driver around 9 months ago, and at that time it felt very heavy/janky and seemed to have very regular bugs and breaking UI changes.I’d be happy to hear that it has gotten better!
- girvo> With them, you can run dozens or hundreds in parallel.Im already spending my $2k usd per month in tokens with just one or two agents. How do they expect me to do that?
- r0m4n0The rate at which software engineering has changed is staggering. We don’t fully understand what is the right or wrong way to be doing things. It’s cool that they share what they are doing but that’s probably about the only thing most people should take from this. Jury is still out on whether this is sustainable budget wise, can be done better differently, whether this sustainable maintenance or quality wise, etc
- NarannThe more I read articles like this, the more I realize Claude is good for process prototyping, but you need to "bake" the process with flat code/script/organization to decrease costs.The future of IA would actually be to large scale prototype easily, even more like Python was a prototype language.
- koolba> Nobody can read every change, so we use failures as one way to decide what needs a person.Whether a human can or should review every diff is a conscious project level decision. I have yet to see a project where it is not feasible. The trick is to break things into small enough pieces that you can review them. Just like one would do if you were doing such changes manually. Nobody in their right mind performs non-mechanical changes of 100s of files.
- asmvolatileThe Claude code team doesn’t think about jack sh*t. Interacting with Anthropic teams makes it clear that nobody at that company has any idea how their software works.
- FartyMcFarter> Flaky tests get the same treatment. Claude can't skip them, add sleeps or retry until they pass. If it can't find the root cause, it deletes the test and writes down why.If the tests are hermetic, you should always be able to find the root cause (by bisecting to determine when the test became flaky).If the tests are not hermetic, it should still be possible to identify the dependency that is making them flaky, and use that as a justification if it still makes sense to delete the test.
- hliyanIn my opinion, the output of a stochastic process can only be truly verified through a deterministic process. Tests already do this for functional accuracy. Linters can do this for line-level code, with code treated as text. But there is currently no such method of deterministic verification for the structure of the entire program, or more importantly the viability of the architecture when you throw in connectivity and IOPS.Perhaps we need a level above linters: a tool that constructs the abstract syntax tree and verifies its various branches against deterministic rules.How to do this for architecture: unsure.
- anonundefined
- gregwebsAfter I created this skill for verification, it completely changed my level of trust with AI coding.https://github.com/gregwebs/skills-sdlc/blob/main/skills/ver...The implementation process that I am using also looks for verifications during planning and reviewing and makes sure they are completed. It does use a lot more time and AI spend than just asking AI to do something. I still do the latter for simple or noncritical changes. I had to switch to using DeepSeek as my workhorse to keep down my spend.https://github.com/gregwebs/skills-sdlc/blob/main/skills/imp...
- neloxWe are well on the way to recursive self improvement
- anshumankmrHonestly, the amount of bullshit unit tests Claude keeps pushing nowadays. I took a few hours the past few weeks, some with AI, some by myself to hand remove some really dumb fucking test cases. God knows how long the CI must be running when these guys push their code.
- sashank_1509Take 10-20 skilled programmers, the kind that made web browsers, game engines etc. Give them a year. They will end up with a superior working version of Claude Code without thousands of bugs and issues, that keep getting fixed and unfixed and trillions of tokens spent by AI.They won’t even do all this verification nonsense and their end product will be of superior quality.
- nandanadileep29excellent content from the team. very concise and clear
- arbor-group[flagged]
- adilbasari[flagged]
- fk202611[dead]
- ankur_misra[flagged]
- anatolii1309[flagged]
- makercfd[flagged]
- i_rutul_patel[flagged]
- runtarmac[flagged]
- dotdi> https://github.com/anthropics/claude-code/issues has over 16k open issuesthis u?
- AmpersanderWhy is the team thinking? Their product should be doing that for them
- saagarjhaThis is just a depressing read, and I’m not even sure why the team decided to post it because it just looks horrible. Like, you’re telling me that the team’s approach to verification is that they post Claudeslop blog posts talking about the most unimaginative ways to write tests? That their idea of making things work is this?> You can also close the verification loop by using the results to improve the agent’s performance on that task. For example, if an agent repeatedly fails a test that checks logs for user data, you can create a hook or a more explicit rule in your CLAUDE.md to avoid future issues. This (mostly) stops issues from happening again.There are people out there who are doing all sorts of crazy neat things in the verification space using the tools you build and yet the people who could be helping seem to have no good ideas on how to help :/
- titzerI see a lot about adding skills to make Claude test things.How about making Claude generate scripts that you can run?Algorithms are better than agents. In fact, the whole point of AI is to reduce thinking to algorithmics. I didn't see a lot of evidence that the team understands this. If you need a magic box to test your other magic box, you have magic boxes all the way down.
- TheRoqueI can totally see that they don't review their product at all, Claude code is probably the thrashiest TUI I've ever used.
- lavendermonAt what point in the future could the title become: "How the Claude Code team's Claude Code thinks about verification"?
- camgunzCongratulations you have invented testing.
- inamortyCan they not afford someone with an engineering background?
- whampThis whole thing read like google not anthropic.
- anonundefined
- matltcFormal vibefication
- jordemortwe think about verificationwe don’t actually do any of the work but we think about it
- ForHackernewsIronic that this page loads partially styled with broken CSS:"The resource at “https://claude.dev/_next/static/chunks/241gnzjfqbp9f.css” preloaded with link preload was not used within a few seconds. Make sure all attributes of the preload tag are set correctly."
- dude250711> ...On the Claude Code team, the vast majority of our code (~90%) is written and reviewed by Claude... I am more interested in the remaining 10%.
- 123ajhgFIs the perfect verification the reason that Claude is down so often?More snake oil and theatrical agent setups described in insufferable Claudish. I think the target group of Claude is people with an IQ below 100.
- mococaSo a lot of slop...
- Yaojia666[flagged]
- oceanrelay[flagged]
- sick_of_slop[dead]
- yoy0721[flagged]
- ahmedketata[dead]
- Eunice99x[flagged]
- jonenst[dead]
- mohsen1[flagged]