WritingThree hours chasing a bug that turned out to be a timezone
PublishedApril 22, 2025
Reading time3 min read

Three hours chasing a bug that turned out to be a timezone

A lesson in not trusting your assumptions, reading error messages properly, and why UTC exists.

It's always a timezone.

I know this. Everyone who has spent enough time writing backend code knows this. And yet, last month, I spent three hours debugging what I was convinced was a database race condition, a caching bug, and briefly — embarrassingly — a corrupted Python installation.

It was a timezone.

What I was building

Part of the Aufeuo backend triggers time-sensitive events: send a notification at a specific time, expire a price after a window closes, that kind of thing. I'm storing timestamps in PostgreSQL and reading them back in Python.

Standard stuff. I've done it before.

The symptom

Events were firing, roughly, but not exactly when they should. Sometimes a few seconds early. Sometimes a minute late. The offset wasn't consistent, which was the thing that really threw me — a consistent offset suggests a conversion error, but a variable offset suggests something weirder.

My first theory was a race condition in the scheduler. I spent an hour adding logging, slowing things down, checking locks. Nothing.

My second theory was a Redis cache returning stale timestamps. I disabled the cache entirely. Still wrong.

At this point I've been at it for two hours and I'm staring at a database query result that looks like this:

# What I expected
datetime(2025, 4, 22, 14, 30, 0)

# What I got
datetime(2025, 4, 22, 14, 30, 0)  # looks the same, isn't

The repr() looked identical. The values were not.

The actual problem

Python's datetime objects can be either aware (they know their timezone) or naive (they don't). When you compare an aware datetime to a naive one, Python doesn't raise an error — it just gives you wrong results.

PostgreSQL was returning UTC timestamps. My application code was creating datetime.now() — naive, defaulting to local time (EST, UTC-5). When I compared them, Python was silently treating them as the same timezone.

The variable offset? That was daylight saving time doing its thing. Some comparisons were off by one hour, some by nothing, depending on which side of the DST boundary the timestamps fell.

The fix

Two lines.

# Before (broken)
from datetime import datetime
now = datetime.now()

# After (correct)
from datetime import datetime, timezone
now = datetime.now(tz=timezone.utc)

And on the database read side, make sure SQLAlchemy returns timezone-aware datetimes by setting timezone=True on the DateTime column type.

What I should have done differently

Read the types. If I'd printed type(timestamp) and checked the tzinfo attribute in the first ten minutes, I'd have seen a naive datetime immediately. I was so convinced it was an infrastructure problem that I didn't check the fundamentals.

Trust nothing you didn't create. Library functions, ORM reads, API responses — assume nothing about the timezone of any timestamp you didn't explicitly set. Check it. Log it. Be paranoid.

The error message I ignored: at one point SQLAlchemy threw a warning about comparing naive and aware datetimes. I dismissed it as noise. It was the answer.

Read your warnings.

← Back to writingApril 22, 2025