Engineering standards
A case study shows a result. Standards show whether it will repeat on your project. Below is how we actually work; every point is backed by an example from our own work rather than a phrase from someone else’s policy.
Decisions
Most failures in software are not bad code. They are confident decisions made without measurement.
Measure first, fix second
A guess about the cause of slowness is almost always wrong. Before optimising we reproduce the problem and get a number, and after the fix we get a second one.
The first paint of our own site took 6.4 seconds. The obvious suspect — the heavy 3D background — turned out to be secondary: the culprit was a splash layer the server rendered for everyone before it knew who had arrived. The fix brought it to 2.2 seconds. The breakdown, with the measurement method, is in the blog.
Architecture before code
Decisions about structure, data and system boundaries are made before the first line. Rewriting a document is cheaper than rewriting a service.
When in doubt, test on slow hardware
Development happens on fast machines; the product is used on all kinds. We test on a throttled CPU and a poor network, not only on our own desks.
A complaint about “lag on weak laptops” was reproduced with a software renderer: a scene frame took 175 ms against 17 ms without it. After the fix the first screen renders at 104 frames per second instead of five.
Code
Comments explain why, not what
What a line does is visible from the line. The comment carries the reason: why it is done this way, what was tried before, and what breaks if someone rewrites it the “obvious” way.
Next to an nginx directive in our repository sits an explanation that `add_header` inside a location cancels every header inherited from the server level. Without that line the next person removes the “duplicate” headers and silently strips protection from images.
A dependency has to pay for itself
Every library is someone else’s code in your product, someone else’s vulnerabilities and someone else’s upgrades. We do not add one for a single function.
The image processing tool is installed for the duration of a re-encode and removed from dependencies right after. The mail service that receives and answers letters runs on a single dependency.
Code is read more often than written
We write so that a year from now someone who did not write it — including your future developer — can work with it. That is not politeness, it is cost of ownership.
Verification
A bug is cheapest to catch before deployment and most expensive to hear about from a user.
Every commit passes four checks
Types, linting, build and a content check — before the change can reach the server. If it fails, it does not ship.
Every caught bug becomes a regression test
Fixing is not enough; it must not come back. Every serious breakage turns into a check that runs before deployment.
After we found that without WebGL support the site collapsed into a blank screen entirely, a script appeared that verifies three failure scenarios: a normal browser, a browser with no 3D context, and a browser with WebGL blocked.
Content is checked like code
A broken internal link, an article without a URL, a repeated anchor or an over-long search description fails the build exactly like a type error.
A flaky test is a bug, not weather
“Just run it again” is not an answer. If a test fails once in three hundred runs it has a cause: the particular interleaving the operating system picked that time.
We wrote an open-source tool for exactly this — unflake. It takes the schedule away from the OS: time, timers and the order of ready callbacks are drawn from a seed, so a failure reproduces byte for byte on any machine and the bug is shrunk to a minimal schedule. TypeScript, MIT, zero dependencies, published on npm.
A green run is not proof
A suite that always passes might be catching bugs, or might be looking at nothing at all — the colour does not tell you which. So the checking itself has to be convicted: shown to fail where it is required to fail.
That is how our two open repositories are held up. bulwark, an implementation of the Raft consensus algorithm, carries a bug museum: seven exhibits, each switching off exactly one rule of the algorithm, and the harness has to catch it — with a seed and with the name of the property it broke. adya, a transaction-isolation engine, holds every level to two directions: it never produces what that level must prevent, and it does produce what the next level prevents. Without the second half the first is empty: an engine that aborted every transaction would be “clean” at every level, and a checker that found nothing would agree with it. The same move is taken furthest in pnueli: a broken search reduction answers exactly what a working one answers — “no violation found” — so the exhaustive search is kept alongside it, slow and unable to be wrong, and the two verdicts must match.
Deployment
Building does not disturb the running site
The new version is built in a separate directory and swapped in with two renames. Users never see a half-built site.
Builds used to run over the live version, and for all forty seconds the site returned errors. The swap now takes milliseconds.
A health check follows every deploy
A build can succeed while the app fails on start: a broken environment variable, an error in server code. We wait thirty seconds for a response.
Rollback is automatic
If the new version does not answer, the previous one returns — with no human involved and no call in the middle of the night.
Change history
The log says why, not what
“Stability improvements” is not an entry, it is the absence of one. We write what the problem was, how it showed up and what changed; contentious decisions get their own explanation.
The version number describes the nature of the change
A fix, a new capability and a breaking change differ by version number, not by mood.
What was deliberately not done is written down too
The log has a “deliberately not done” section explaining why an apparently obvious change would be a mistake. It saves the next person weeks.
Handover
The code and the access are yours
Sources, infrastructure and documentation are handed to the client. We keep nothing without which the product stops working.
Server configuration lives in the repository
Not in an administrator’s head and not in a chat: a copy of the production configuration is stored next to the code so it can be read, reviewed and restored on a move.
Secrets do not travel through messengers
Keys and passwords are passed through a vault or your own perimeter, and on the server they live in files with closed permissions outside the repository.
What we do not do
Rules without prohibitions mean nothing: everyone agrees with “we write good code”. Here is what we refuse, even when refusing is slower.
We do not rewrite onto a new framework for novelty
Changing technology is justified by the problem, not by a release announcement. Every migration is months paid for by the client and new bugs in what already worked.
We do not hide bad news
A missed deadline and a bug we found in our own work are reported immediately. The breakdown of our own failure is published in the blog, with numbers before and after.
We do not take on what we cannot carry
A refusal on the call is cheaper for the client than a collapse three months in. Sometimes the honest answer is “this system is cheaper to rewrite than to repair”.
We do not leave “temporary” solutions unrecorded
Temporary outlives permanent. If something was done in haste, it is in the log with an explanation of what must be redone and when.
You can check this on the site itself
Everything described here is applied to this site first: measurements, regression checks, rollback on deploy, the change log. The write-ups with numbers are in the blog — read them before you write to us.