The test passed. I ran it again. It failed. Same code. Same test. Different result.

If you've read The Flaky Test Chronicles, you know the feeling. That sinking sensation when CI goes red for a test that was green moments ago. That quiet voice saying “just rerun it.”

This is another one of those stories.

The Beginning

A red build in the CI pipeline.
The reason: “The expected notification was not sent.”
I run it locally, it passes.
CI runs it again, it fails.
Classic flaky test symptoms.

The test looked like this:

Notification::fake();

$job = new ProcessPaymentJob($order->id);
$job->failed();

Notification::assertSentTo(
    notifiable: new AnonymousNotifiable(),
    notification: PaymentFailedNotification::class,
    callback: fn ($notification, $channels) => $notification->orderId === $order->id
);

The job's failed() method:

public function failed(): void
{
    Notification::route('mail', config('app.admin_email'))
        ->notify(new PaymentFailedNotification($this->orderId));
}

So why does it sometimes pass and sometimes fail?

The Source of the Problem

I found the answer in Laravel's NotificationFake class. Notifications are stored like this:

$this->notifications[get_class($notifiable)][$notifiable->getKey()][$notification]

A three-level array: class name, notifiable's key, and notification class.

When I looked at AnonymousNotifiable's getKey() method:

public function getKey()
{
    //
}

Empty. Returns null. And in PHP, (string) null is just an empty string.

So all anonymous notifications should be stored under the same key. Theoretically, I should be able to access them with new AnonymousNotifiable(). But in practice, this wasn't always working.

The Real Problem

I went down this rabbit hole for longer than I'd like to admit. Debugging the NotificationFake internals. Checking if there was some object identity issue. Wondering if parallel test execution was somehow corrupting the notification storage.

Then I found the answer in the most obvious place: the documentation.

This sentence caught my eye in the Laravel documentation:

“If the code you are testing sends on-demand notifications, you can test that the on-demand notification was sent via the assertSentOnDemand method.”

assertSentOnDemand(). A method designed specifically for this purpose.

Looking at the source code:

public function assertSentOnDemand($notification, $callback = null)
{
    $this->assertSentTo(new AnonymousNotifiable, $notification, $callback);
}

The moral? When Laravel provides a specialized method, use it. Don't try to be clever.

The Solution

Notification::assertSentOnDemand(
    notification: PaymentFailedNotification::class,
    callback: fn ($notification, $channels) => ...
);

A one-line change.
assertSentOnDemand instead of assertSentTo.
No notifiable parameter - because the method name already tells us we're testing an on-demand notification.

Bonus: Callback Parameters

The assertSentOnDemand callback can accept three parameters:

Notification::assertSentOnDemand(
    OrderShipped::class,
    function ($notification, array $channels, object $notifiable) {
        // $notification - the sent notification instance
        // $channels - the channels used ['mail', 'sms', ...]
        // $notifiable - route info ($notifiable->routes['mail'])

        return $notifiable->routes['mail'] === '[email protected]';
    }
);

Fifty test runs later, not a single failure.

The fix was one line. The debugging took way longer than it deserved. When Laravel gives you a specialized method, there's usually a reason.

Not every flaky test is a race condition. Sometimes it's just using the wrong tool for the job.

This is a spin-off from The Flaky Test Chronicles - a series documenting what we learned from 300+ commits of test suite cleanup. This particular trap didn't fit neatly into any of the six deadly sins, but it was too good not to share.