One Hour is Too Long (Or Is It?)

A while ago, a software engineer I know asked for my opinion on his startup’s CI. Their End-to-End (E2E) suite took about an hour to run, and the team was agonizing over whether to move it from every PR (presubmit) to a nightly job because it felt “too slow.”

An hour? One, single, hour? For an E2E test suite? My answer: Don’t touch it. In a startup, there are a million things with higher ROI than shaving minutes off an hour-long E2E suite, especially if your unit test coverage isn’t optimal yet. And certainly do not switch to running it nightly. Getting coverage that late will grind your development cycle to a halt, discovering bugs which may have been introduced by any PR in the past 24 hours(!!!).

When I posted about this on Twitter the pushback exposed an interesting aspect of software engineering culture: Many engineers felt that an hour-long CI was torture. They were waiting for the green checkmark before moving to the next task. Some even waited for the code to hit production before starting something new.

To me, this pointed at a deeper problem: it wasn’t actually about the time the CI took. It was about trust. The CI was failing late, the tests were flaky, and the developer workflow was strictly synchronous.

Since then, using AI agents to write code and execute tests asynchronously have become a widespread practice in software engineering, so perhaps engineers have gotten used to working asynchronously – but not necessarily in a good way.

Continue reading →