Skip to content

What does it cost to run an AI agent in production? About €4 a month

Two Railway services, one cron, a webhook, a watchdog and a cost log. The real numbers from the Sunday slides agent's first weeks in production, including the Thursday it failed.

, 9 min read

Illustration: a slide above a heartbeat line and a bar chart of weekly build runs, with one failed Thursday marked in red

Part 1 was the story and Part 2 was the guardrails. This part is about operations: what runs where, how I know it ran, how fast it is, and what each piece costs. Every number below is measured in production from 8 to 23 October 2026, unless I say it's an estimate.

What runs where

Two services on Railway
plans-apiweekly-build
TypeWeb service, always onCron job, restart policy NEVER
DoesWhatsApp webhook, /health, slide plans over HTTPThursday build, Friday retry, Monday reminders
Schedulen/a0 8,17 * * 1,4,5 UTC; wake-ups that aren't due exit at once
Storage500 MB volume: chapter cache, cost log, conversation stateNone; it refetches what it needs
Runtimepython:3.12-slim, FastAPI on uvicornSame image

Both services deploy from main on push. Infrastructure is code (.railway/railway.py), and secrets are marked preserve(), so applying the config never writes or clears them. railway config plan must say already up to date before anything is applied.

Two choices worth pointing out:

  • The cron is idempotent by design. The Friday run checks whether Thursday's deck exists and exits if it does. That's why Friday runs show up as 2–12 second "builds" below.
  • Heavy WhatsApp work runs in a short-lived child process. The web service stays small, and the memory a slide rebuild needs is released as soon as it's done.

How I know it ran: Cronitor

A scheduled job that fails silently is worse than no job, because everyone thinks it ran. So there are two monitors, both on Cronitor's free plan, both alerting the coordinator by email:

  1. Job monitor for weekly-build: schedule 0 17 * * 4,5 UTC with a 30-minute grace period. The build reports run, then complete or fail with the reason. If it doesn't report at all, that's a missed alert.
  2. HTTP check on plans-api /health every 5 minutes, from Europe and the US. If the webhook is down, WhatsApp replies silently go nowhere, so this one matters as much as the build.

The telemetry client is deliberately tiny, and it can never break the build:

python
class Job:
    def __init__(self, url: str):
        self.url = url
        self.series = uuid.uuid4().hex  # pairs this run's events

    def ping(self, state: str, message: str = "") -> None:
        if not self.url:
            return
        params = {"state": state, "series": self.series}
        if message:
            params["message"] = message[:1500]
        with contextlib.suppress(requests.RequestException):
            requests.get(self.url, params=params, timeout=10)

The series id pairs each run with its complete or fail, so Cronitor can compute durations. Test builds don't report, so the dashboard only shows real Sundays.

The Thursday it failed

On Thursday 22 October at 17:01 UTC the build failed after 5 seconds:

text
RefreshError: ('invalid_grant: Token has been expired or revoked.', …)

The Google OAuth refresh token had stopped working. Here's what happened next:

  • 17:01 UTC: Cronitor emailed the coordinator within seconds, with the exception in the email. Nobody had to dig through logs to learn it was an auth problem, not a code problem.
  • That evening: I re-authorised the Google account, updated the three Railway variables and ran the build by hand. It completed at 20:13 UTC in 3 min 16 s.
  • Friday 17:03 UTC: the scheduled retry saw the deck already existed and exited in 4.4 s.

The slide preparer had their review on Thursday evening as usual, three hours late. Without the monitor, the first sign would have been an empty Sunday folder.

The lesson is an unglamorous one: the most likely failure of an agent isn't the AI, it's the plumbing around it. Expired tokens, a changed web page, a WhatsApp template paused by Meta. Each of those has a line in the runbook, and each produces an error message a non-engineer can act on.

Latency

Nobody needs the Sunday slides in milliseconds. What matters is that each step is fast enough for its human on the other end.

Weekly build (Cronitor job durations):

Weekly build durations (Cronitor)
RunDurationWhat it did
Thu 8 Oct2 min 39 sFull build
Fri 9 Oct11.5 sRetry; deck already existed
Thu 15 Oct5 min 05 sFull build (first week with songs)
Fri 16 Oct2.3 sRetry, no-op
Thu 22 Oct5.2 sFailed: Google token revoked
Thu 22 Oct3 min 16 sManual re-run after re-auth
Fri 23 Oct4.4 sRetry, no-op

Most of a full build is spent waiting on APIs, not computing. That means Google Slides batch updates, exporting the review PDF, and one or two Claude design calls of roughly 28k input tokens with images.

Web service (plans-api):

plans-api latency and resources
MetricValue
/health round trip, all probes (p50 / p90)≈ 0.31 s / 0.39 s
From Cronitor's EU probes (p50)≈ 0.36 s
From Cronitor's US probes (p50)≈ 0.25 s
Failed health checks1 of ≈ 4,500 (≈ 99.98%)
WhatsApp webhook server time (p50)24–38 ms
Memory (7-day average / peak)≈ 97 MB / ≈ 330 MB
CPU (7-day average)≈ 0.01 vCPU

The US probes are faster than the EU ones because the service runs in a US region while its users are in The Hague. At these response times nobody notices, but it's on the list.

The webhook acknowledges first and works later. Meta wants a fast 200, so the handler checks the X-Hub-Signature-256 HMAC (in constant time), queues a background task and returns:

python
if not hmac.compare_digest((x_hub_signature_256 or "").encode(), expected.encode()):
    raise HTTPException(401, "bad signature")
...
background.add_task(_process_replies)  # answer Meta now; act on the message afterwards

The background task waits about 8 seconds, then runs the work in a subprocess with a 30-minute timeout. It waits because a PowerPoint and the text that goes with it ("put these after the sermon") arrive as separate WhatsApp messages and should be handled together. Slide changes take a minute or two, so the agent says so first ("give me about 2 minutes ⏳") and then sends a picture of only the slides that changed.

Songs follow the doc with a check every 3 minutes. A hymn number typed into the Google Doc reaches the deck within 3–6 minutes. That polling costs about $0.22 a month.

What it costs

Measured on 12 October 2026, from the agent's own cost log and railway usage. $1 ≈ €0.86.

Monthly cost, measured 12 October 2026
ItemUnit cost (measured)Per month
Claude: weekly design pass (Sonnet)≈ $0.04 per round, 1–2 rounds per build€0.30
Claude: reading WhatsApp messages (Sonnet)≈ $0.018 per message (≈ 8.3k in, 0.14k out)€1.00
Claude: design after a change (sermon PowerPoint)≈ $0.04 per round, ≈ 6 a month€0.20
Claude: wording replies (Haiku) + song ideas$0.0005 per reply; ≈ $0.01 per Sunday€0.10
Railway: this project's usagememory is almost all of it€0.60
WhatsApp template messages≈ €0.03–0.055 each, ≈ 40 a month with the full roster€1.20–2.20
Google APIs, GitHub, Cronitor, Bible sourcesfree tiers€0
Total€3.40–4.40

The Railway Hobby fee ($5, with $5 of usage included) is shared with this website, so I don't count it here. If you do, the total is €7.10–8.10. The budget I set at the start was under €15 a month.

How it got there

The first estimate, before any optimisation, was €12.40–15.20 a month: uncomfortably close to the budget. The changes, in order of impact:

  1. Opus → Sonnet for the designer and for reading messages. That halved the per-token price, and the quality was fine for "is this readable from the back row?" and "what does this person want?".
  2. Design only the slides that changed. Each deck starts from last Sunday's already-designed deck, so most slides need nothing. The design pass went from $0.46–0.55 to about $0.08 per build.
  3. Smaller images (512 px) sent to the model. That's plenty to judge contrast and overflow.
  4. The child process for heavy work, which keeps the always-on service's memory, and so its Railway bill, low.

I also looked at what not to optimise:

Optimisations considered and rejected
IdeaSavingVerdict
Prompt caching for messages≤ €0.20 a monthMessages arrive minutes to hours apart, so the cache would mostly miss
Haiku for reading messages≈ €0.50 a monthNot now: messages come in five languages, often mixed, and understanding them is the part that must not get worse
Scale plans-api to zero, or Cloud Run≤ €0.60 a monthCold starts on a webhook, for pennies

Knowing what it costs, every month

You can't keep a budget you can't see. So:

  • Every Claude call is appended to llm_costs.jsonl on the volume: model, tokens in and out, and dollars.
  • Every job adds its total to the month and stores it on the church's Drive folder as a hidden property (claude_usd_2026-10). The cron service has no volume, so a local log file would vanish after each run.
  • Each job's log ends with "Claude: $…; this month so far: $…", and the coordinator's Monday summary ends with the month so far.
  • Test builds aren't counted, so the number is what real Sundays cost.

Like the monitor, the bookkeeping is wrapped so it can never stop a job. If Drive is unavailable, the total just isn't updated that time.

There was also a one-off cost: about $18 to build the song library from the scanned hymnal. That's a good reminder to give one-off tools their own API key with their own limit.

Degrading gracefully

Every external dependency has a defined "what happens if it's down":

What happens when a dependency fails
If this failsWhat happens
Anthropic API (outage, no credit)The deck is still built, just without the design pass. Replies are forwarded to the coordinator instead.
Bible source changes its HTMLSourceError, build stops, Cronitor emails. A regression script replays all 54 Sundays in the doc.
Google token revokedBuild fails fast with invalid_grant (see above).
Apps Script copy unavailablePowerPoint slides go in as pictures instead of editable slides, and the reply says so.
WhatsApp template paused by MetaReview not sent; the build log says why.
Readings not in the doc yetThursday reports "Not built: no readings". Friday retries.

What I'd do next

  • A usage limit on Railway and a spending cap on the Anthropic key. Not for the normal bill, which is tiny, but for a runaway loop.
  • An alert on low Anthropic credit, before it turns into a week without a design pass.
  • Move the service to an EU region, closer to its users.
  • A church-owned WhatsApp number instead of the current test number.

None of this is exotic. That's the point. The AI part of the agent got most of the attention while building it, but most of what makes it dependable is ordinary engineering: idempotent jobs, alerts with the reason in them, a cost log, and a runbook a coordinator can follow.

Filed under

  • Agents
  • Observability
  • Resilience

Related

  1. AI, , 6 min read

    Designing an AI agent that can't misquote the Bible

    How the Sunday slides agent proves it never changed a verse, why Claude only gets to propose design changes, and how the design pass went from 50 cents to 8 cents a build.

    • Agents
    • AI architecture
    • Python
    • +1 more
  2. AI, , 6 min read

    I built an AI agent that makes our church's Sunday slides

    Every week the same few volunteers spent two hours or more chasing sermon slides, copying Bible verses and fixing line breaks. So I built them an agent that does it on Thursday evening, for about seven cents, and still lets a person say no.

    • Agents
    • AI
    • Building

Next

Have a system worth arguing about?

If this note raised a question, or you would have designed it differently, I would like to hear it.

Start a conversation