Idempotent consumers, without the ceremony

Every message broker I have used promises at-least-once delivery, and every one of them means it. Sooner or later the same message arrives twice: a consumer crashes after doing the work but before acknowledging, a rebalance hands a partition to someone else mid-batch, or a producer retries a send that had actually succeeded.

The usual advice is "make your consumers idempotent", which is correct and not very helpful. This note is the smallest set of changes that got me there, without a framework or a distributed lock.

What actually goes wrong

Take a consumer that reserves stock for an order. Processing the same OrderPlaced event twice reserves the stock twice. Nothing errors; the numbers are just quietly wrong until someone counts the shelf.

• The consumer commits the database change, then dies before acking.
• The broker redelivers after the visibility timeout.
• The second delivery runs the same business logic against new state.

The smallest fix: record what you have processed

Give every message a stable ID at the producer, and record that ID in the same transaction as the side effect. If the insert conflicts, the work has already been done and the message can be acknowledged and dropped.

@Transactional
public void handle(OrderPlaced event) {
    var messageId = event.messageId();

    // Same transaction as the side effect: both commit or neither does.
    int inserted = processedMessages.insertIfAbsent(messageId);
    if (inserted == 0) {
        return; // already processed — safe to ack
    }

    inventory.reserve(event.sku(), event.quantity());
}

The table behind it is deliberately boring: a primary key on the message ID and a timestamp so old rows can be pruned.

CREATE TABLE processed_messages (
    message_id   UUID PRIMARY KEY,
    processed_at TIMESTAMPTZ NOT NULL DEFAULT now()
);

Choosing where the guarantee lives

There are three reasonable places to put the check. I compared them on the things that actually hurt in practice:

What I would do again

Start with natural idempotency wherever the domain allows it, and reach for the processed-messages table for everything else. Exactly-once settings are worth understanding, but they solve a narrower problem than their name suggests.