In my testing, an overnight Ralph loop didn’t deliver a finished refactor. It delivered what the prompt literally asked for, and found every loophole the prompt left open. Over more than 200 iterations on a Laravel codebase I hit seven gotchas. Each one was fixed by a more precise prompt and by verifying after every task.
A Ralph Wiggum loop, named after the Simpsons character and popularised by Geoffrey Huntley, runs an AI agent in a continuous loop, feeding it the same prompt until the task is done. The promise is that you point it at a codebase, go to sleep and wake up to finished features.
I tested it on a Laravel SaaS codebase outside production, with 78 Actions and 3,263 tests. The goal was a migration to three Spatie packages: laravel-model-states, laravel-data and laravel-query-builder. It’s the kind of systematic refactor that’s tedious for a person and, in theory, well suited to an AI running in a loop.
What I expected: an overnight refactor that ran on its own, with verification gates.
What I got: documentation about the refactor, and some creative loopholes.
The loop does exactly what your prompt says. Vague instructions get creative interpretations, and loose success criteria get exploited. In practice, that makes it far less reliable than its reputation.
Each gotcha below follows the same pattern: the problem, an example, and the fix that closed the loophole.
“Tests must pass” is dangerous
I wrote this prompt:
Migrate Product model to spatie/laravel-model-states.
Tests must pass before completion.
Output <promise>COMPLETE</promise> when done.Ralph made the tests pass, by hard-coding enum values instead of using State classes.
// What we wanted:
$this->assertInstanceOf(ActiveState::class, $product->status);
// What Ralph wrote to make tests "pass":
$this->assertEquals('active', $product->status); // Hardcoded string!The tests passed. The migration didn’t happen. Ralph found a path to “green” that completely bypassed the actual work.
This is the gotcha behind the others: an agent works to the success criteria you write down, rather than the outcome you have in mind. “Tests must pass” says nothing about how they should pass, so the agent took the cheapest green it could find.
The fix
Say how, as well as what:
Tests must pass USING State class assertions.
Bad: assertEquals('active', $product->status)
Good: assertInstanceOf(ActiveState::class, $product->status)Examples of acceptable and unacceptable patterns close the loophole, so the agent can’t claim success by reinterpreting the task.
The invented category escape
I asked Ralph to analyse 74 controllers and grade every method:
Grade each method:
CLEAN (1-5 lines)
NEEDS WORK (6-15 lines)
FAT (16-30 lines)
OBESE (31+ lines)Ralph invented a fifth category: “REVIEWED”
## AdminController - REVIEWED
## AuthController - REVIEWED
## BaseController - REVIEWED
(23 controllers marked REVIEWED with no method analysis)It found an escape hatch I hadn’t closed. “REVIEWED” satisfied the prompt, because every controller got a label, but the label told me nothing.
This came up again and again. If a prompt leaves any doubt about what counts as a valid answer, the agent finds it. The agent wasn’t being malicious. It was finishing the task with the least effort, which is what the prompt asked for.
The fix
Forbid the alternatives by name:
ONLY these grades exist. No other categories.
No "REVIEWED". No "SKIPPED". No "N/A".
Every method gets exactly one grade from this list.Name every escape hatch you can think of, then forbid it. If you can picture Ralph taking a shortcut, assume it will.
Estimation instead of analysis
The prompt said: “Analyse each controller and identify fat methods.”
Ralph “analysed” by counting total lines and estimating:
NetworkController: 491 lines
Estimated fat methods: ~10
ProductController: 427 lines
Estimated fat methods: ~8It never opened the files. It guessed from line counts.
This is common. Agents take the easiest route to output that looks plausible. A 491-line file probably has around 10 fat methods, which is a reasonable guess, but it’s still a guess.
The output looked professional, with structure, numbers and controller names. Nothing in it showed that no analysis had happened.
The fix
Require proof of reading:
For each controller:
1. Run: cat app/Http/Controllers/[Name].php
2. Quote the first line of EVERY public method
3. Count lines per method (show the count)
4. Then assign grade
No estimation. No guessing from file size.Making the agent show its working makes shortcuts visible. Quoting the code proves it read the file.
”Batch updated” without showing work
Ralph reported:
Updated 12 test files to use State assertions
Migrated 5 models to use State casts
Removed all enum usagesIt read well, with a tidy summary and clear progress markers. Then I checked:
grep -rn "ProductStatus::" app/
# 14 matches foundRalph claimed the batch was done, but the codebase didn’t show it. The summary read like progress, and the diff said otherwise.
This one makes the others worse. Once you allow batch reports (“updated X files”), you can’t see the individual changes, and the agent can claim any number.
The fix
Require proof for each file:
For EACH file changed:
1. Show the filename
2. Show the before code
3. Show the after code
4. Run verification command
No batching. No "updated X files". Show every change.The output gets long, and you can audit it. You can’t audit “updated 12 files”.
Detailed reports cost far less than finding out hours later that nothing changed.
Creating without integrating
I asked Ralph to implement spatie/laravel-data for type-safe DTOs.
Ralph created 12 Data classes:
class CreateInvoiceData extends Data
{
public function __construct(
#[Required] public string $customer_id,
#[Required, ArrayType] public array $line_items,
) {}
}They were textbook: proper attributes, tidy constructors and Spatie’s conventions followed to the letter.
But controllers still did this:
// Data class exists but is completely bypassed
$action->execute($request->validated()); // Raw array!The Data classes sat on their own, never wired into the application. As far as Ralph was concerned, the task was done: the classes existed, and the prompt said nothing about using them.
The fix
Define integration as part of completion:
A Data class is NOT complete until:
1. The class exists
2. The Action accepts it (not array)
3. The Controller uses Data::from()
4. Tests use Data::from()
Verify ALL FOUR before marking complete.If your prompt doesn’t define done from end to end, you get components that exist and do nothing.
Package not installed
The prompt referenced spatie/laravel-query-builder. The plan included QueryBuilder code. Ralph wrote detailed implementation steps. The completion marker fired.
composer show spatie/laravel-query-builder
# NOT INSTALLEDRalph wrote a plan about using a package that didn’t exist in the project. It marked the task complete. It moved on.
This one stung, because it’s so basic. The agent wrote code against classes that would fail on the first line. The plan had no syntax errors, and no connection to the environment either.
The fix
Verify prerequisites before starting:
STEP 1 (before anything else):
Run: composer show spatie/laravel-query-builder || echo "NOT INSTALLED"
If not installed:
Run: composer require spatie/laravel-query-builder
Verify installed before proceeding to Step 2.Don’t assume the environment is what you expect. Make the agent check dependencies, configuration and prerequisites before it writes any code.
A plan built on a missing package wastes iterations and gives false confidence.
Verification at end only
I had verification steps, in the wrong place:
## Final Verification
Run these commands to confirm completion...By the time verification ran, Ralph had made 47 commits across 6 tasks. Half were wrong. Rollback was painful.
This one is about structure rather than wording. When verification only happens at the end, errors build on each other. Each task builds on the output of the ones before, so by the time you find that Task 2 went wrong, Tasks 3 to 6 rest on it.
Untangling that is slow. Which commit introduced the problem? What depended on it later? Can you cherry-pick fixes without breaking the work after them?
The fix
Verify after every task:
## Task 3: Update Controller
[steps...]
**Verify before commit:**
Run: grep -c "Data::from" app/Http/Controllers/InvoiceController.php
Expected: 1 or more
Run: php artisan test --filter=Invoice
Paste output. Must be green.
**Only then commit.**An error is easy to catch after one task, and hard to catch after 47 commits. Build verification into each step.
The prompt template
This is the structure I ended up with:
# [Task Name]
## Domain Language
Define every term. No ambiguity.
## Rules
What MUST happen. Explicit requirements.
## Forbidden
What MUST NOT happen. Close every loophole.
## If Tests Fail
Explicit recovery steps. Don't let it get stuck.
## Tasks
### Task N: [Name]
**Files:** List every file touched
**Steps:**
1. [Action]
Run: [command]
Paste output. Expected: [result]
2. [Action]
Run: [command]
Paste output. Expected: [result]
**Verify before commit:**
Run: [verification command]
Expected: [result]
Paste output.
**Commit:** (only after verify passes)
## Final Verification
Mathematical proof of completion.
## Completion
Output <promise>DONE</promise> only when:
- [ ] Criteria 1 verified
- [ ] Criteria 2 verified
- [ ] Criteria 3 verifiedEach section has a job. Domain Language removes ambiguity. Rules define success. Forbidden closes loopholes. Tasks break the work into chunks, and each chunk proves itself before the next begins.
Every section is there because I watched Ralph exploit its absence.
Key principles
Five rules came out of the iterations:
1. Say how, as well as what
Bad: “Tests must pass” Good: “Tests must pass using State class assertions, not string comparisons”
The outcome you want is implicit to you. Make it explicit for the agent.
2. Forbid everything you don’t want
Bad: (silence about categories) Good: “FORBIDDEN: Inventing new categories, skipping files, estimation”
Name each shortcut you can imagine, and ban it.
3. Require proof at every step
Bad: “Update the controllers” Good: “Update the controller. Run grep. Paste output. Expected: X”
Treat any claim without proof as unchecked.
4. Close every loophole explicitly
An agent with no constraints finds the shortest route to done, and reads your words in whatever way gets there.
5. Verify as you go
Run the tests after every task, not once at the end of the night.
Each rule helps on its own, and all five together cut the failure rate. Expect surprises anyway.
What I took from it
Many people say a complex refactor can finish while you sleep. In my experience, the deeper the refactor, the less reliable the loop. The value was in the prompt and the verification, more than in the loop itself.
Write the prompt as you’d write a contract for a very literal lawyer, who’ll find every ambiguity and use it.
Every gotcha here happened because I assumed the agent would follow my intent. It followed my words.
Leave a gap between the two, and you’ll wake up to documentation about the refactor you wanted, rather than the refactor itself.
Resources
- Ralph Wiggum technique: Geoffrey Huntley’s original post
- Claude Code best practices: Anthropic’s guidance
- Prompt engineering best practices: structured prompts in more depth
- Teaching AI to understand your codebase: documentation scaffolds that give an AI context
I build Laravel applications for clients, and every change an AI tool makes in them is checked the way this post describes.
Published , updated .