One Person, 160 Domains, One Nightly Job

A single developer maintains around 160 domains on this system, with one scheduled job doing the building, checking and measuring. The interesting part is not the number. It is which parts of a shop stop scaling with the client count.

Context

Most agency platforms are demonstrated on one well prepared example site. That proves the software can produce a good page and tells you nothing about what happens at site forty.

This system was not built as a product first. It was built to run a portfolio of PHP and JSON properties that one person owns and operates, and the product exists because the portfolio needed it. That order matters, because every rough edge was found on real sites rather than in a demo.

Methodology

One scheduled job runs nightly across every registered site. For each one it rebuilds the sitemap, rebuilds the machine readable files, then fetches every page in that sitemap and reads the rendered output back.

A receipt file is written only when that crawl passes with no failures, so a passing receipt cannot be produced by a partial run. A second pass then compares the registry of known sites against the hosting account and fails on any site that exists but is not registered.

The job is fail closed. A failure anywhere is accumulated, written to a log and returned as a non-zero result after the remaining sites are still checked, so one early break cannot be masked by later successes.

Findings

The work that scales badly and the work that scales well separate very clearly at this size.

Building, checking, structured data, sitemaps and measurement all stop growing with the site count once there is one build being repeated. They are a fixed cost paid in setting the system up.

Writing, judgment and client relationships do not scale at all. They cost the same per site at 160 as at 6, which is the honest answer to anyone hoping software removes the work.

The errors worth catching turned out to be the quiet ones. Pages that returned normally while being wrong, links pointing at a previous site, and a hiding instruction nobody could see. All three now have a script watching for them, and all three were found by reading rendered pages rather than trusting status codes.

What this is not

It is not a claim that one person can look after 160 client relationships. Client work is phone calls and invoices and someone changing their mind about a headline, and none of that is a scripting problem.

What it does show is the ceiling on the technical half. If the building, checking and measuring of 160 sites can run on one person's week, then the technical side is not what is capping a small shop at eight clients.

That is a useful thing to know, because most shops assume the opposite and go looking for staff before they go looking for a system.

The part that transfers to client work

Three habits, and none of them need a portfolio to be worth copying.

Pick one build and repeat it, even when a custom version would be slightly better for one site. Write the checks as scripts rather than as intentions. And make the job fail loudly, so a problem reaches you instead of waiting to reach a client.

The honest limits

Not every site in the portfolio is built the same way. The older ones predate the current system and carry their own quirks, which is exactly the twentieth site problem described elsewhere on this site, lived rather than theorised.

The nightly checks also test what the system can test. Structure, links, coverage, hidden pages, malformed data. They cannot tell anyone whether a page was worth writing, which remains a judgment nobody has automated.

And a crawler request is not a reader. The data layer records both and labels which is which, because blending them produces a number that looks better and means less.