HyperFaaS’s Compose files model workers, leaves, and routing controllers at different scales. The layout that shipped before PR #62 (92f3b2a) was honestly terrible: every replica that needed a custom image went through Compose’s build: path with its own tag, so a “full refresh” meant many separate Docker builds of the same Go binaries (for example eight worker images on the large profile, each a full compile), and the Dockerfile did not yet use BuildKit cache mounts for go mod and go build. It worked, but it was terribly slow. We have three setups: small, medium and large. Each setup has their own YAML file with a different number of workers, leaves and routing controllers.

What changed#

  • Compose: Removed per-service build: from the small, medium, and large YAML. Now all equal components share the same image.
  • Just: Our justfile recipes just ss / just sm / just sl can take optional targets (for example worker, leaf, routingcontroller) so we can rebuild only what changed, then bring the system back up.
  • Dockerfile: BuildKit syntax and cache mounts on go mod download and go build so module and compiler work survive across invocations. This is a HUGE improvement (can’t believe we didn’t do this sooner).

Latency I measured#

I benchmarked wall-clock time until all relevant images finished building: once using the old compose flow (parent of 92f3b2a, docker compose build on each profile’s YAML), and once using the new flow on main (three docker builds in parallel for worker, leaf, and routing controller). I cleared the BuildKit reclaimable cache between those two halves so the “after” run was not simply warmed by the “before” run. All on an AMD Ryzen 7 7730U (8 cores / 16 threads, up to 4.55 GHz) with 64 GiB RAM on Linux 6.19.

Profile Before (compose build) After (three parallel builds) Less wall time
Small 40.681 s 0.233 s ~99.4%
Medium 35.132 s 0.172 s ~99.5%
Large 39.904 s 0.144 s ~99.6%

A few words of context:

  • The before side was doing far more work than necessary (many redundant full builds plus a less cache-friendly Dockerfile).
  • The after times are still dominated by Docker reuse: only three images, a tighter Dockerfile, and a warm content store on a dev box mean most layers resolve almost instantly once cache mounts and layer graphs line up. That is we want for day-to-day work.

Takeaway#

If the same artifact appears under fifteen different service names, Compose will happily compile it fifteen times unless you stop it. Sharing one image per role, building deliberately, and letting BuildKit hold on to Go’s module and build caches makes it noticeably instant on the setups above.