Sizing & Scaling
Wagoe’s architecture (FC/IS + hexagonal ports) is meant to let you scale vertically, horizontally, or a mix — mostly by configuration rather than rewrites. This page is an honest map of how far that holds today: which knobs exist, which components are already safe to run as many replicas, and which still hold in-process state that you must account for.
Three axes
| Axis | Meaning |
|---|---|
Vertical |
One process, more resources. Bigger heap, larger connection/thread pools, more CPU. Pure configuration in Wagoe. |
Horizontal |
Many processes (replicas) behind a load balancer, sharing backing services (Postgres, Redis). Requires that no request-handling state lives in a single process. |
Functional decomposition |
Slice modules into separate deployables (microservices-style) that scale independently. A cross-module call that was in-process becomes a network call. Highest leverage, highest effort. |
Vertical scaling — by configuration today
All sizing knobs live in resources/conf/{dev,test,prod,acc}/config.edn plus
environment variables. Change the value, restart the process. No code.
| Knob | Where | Notes |
|---|---|---|
DB connection pool |
|
HikariCP. prod default |
HTTP server |
|
Jetty. |
JVM heap / GC |
|
Default |
Cache (Redis) pool |
|
Jedis pool. prod default |
|
Tip
|
Watch the multiplication: with N replicas each holding a pool of |
How the architecture enables horizontal scaling
Cross-module calls go through protocols defined in each module’s ports.clj
(enforced by bb check:ports). Core logic depends on a protocol, never on a
concrete adapter. That seam is the scaling lever: swap an in-process adapter for
a distributed one — Redis, a queue, a remote service — without touching the
functional core.
Several libraries already ship both adapters. The provider keyword is per library, not a convention — check it rather than assume:
:wagoe/cache {:provider :redis ...} ; | :in-memory
:wagoe/realtime {:provider :redis ...} ; | :in-memory (the default)
:wagoe/events {:provider :redis-streams ...} ; | :in-memory — no default, throws otherwise
;; libs/jobs has :in-memory, :redis and :db adapters, but no :provider key:
;; the application constructs :wagoe/job-queue / :wagoe/job-store directly.
This is the template every other seam follows: the protocol is the contract, the distributed adapter is "just configuration" once it exists.
With one condition that is easy to leave implicit. Swapping adapters is only
safe if both answer the protocol the same way, and a protocol does not enforce
that — a test across both adapters does. libs/cache had thirteen divergences
between its two adapters, each invisible to per-adapter suites that only ever
asked one of them, until a contract sweep enumerated the surface (BOU-288). Any
new seam here inherits that risk, so a second adapter comes with a sweep that
runs every method against both.
Horizontal readiness matrix
| Component | N replicas | Detail |
|---|---|---|
Cache |
✅ |
Redis adapter ( |
Jobs |
✅ |
Redis queue ( |
Auth / sessions |
✅ |
DB-backed, pure core ( |
Multi-tenancy |
✅ |
schema-per-tenant ( |
Email / external |
✅ |
Async via the jobs queue / stateless IO adapters. |
Event bus |
✅ |
Redis Streams adapter ( |
Readiness checks |
✅ |
|
Rate limiting |
✅ |
The config-driven |
Graceful shutdown |
✅ |
Integrant |
Realtime / WebSocket |
✅ * |
Replica-safe via |
Topologies
| Shape | When |
|---|---|
Single fat node |
Vertical only. One process, large heap and pools. Everything works, including realtime and the in-memory adapters. Rate limiting is the one exception: the |
N stateless web replicas |
The main horizontal mode. N copies of the uberjar behind a load balancer, sharing Postgres + Redis. Cache, jobs, auth, tenancy all scale. Caveats: activate |
Web / worker split |
Dedicated job-worker processes separate from web. |
Module as its own service |
|
The same image runs all four; the container argument selects the mode. Compose files, manifests and what has actually been brought up are in Deployment Topologies.
Production checklist
-
Use the Redis cache and jobs adapters, never
:in-memory, for more than one replica. -
Register all job handlers on every instance. A dequeued job with no local handler is re-enqueued so another instance can run it, and is dead-lettered only after
:max-requeue-age-ms(default 5 min) of going continuously unhandled — so a job-type that no worker registers costs five minutes of re-enqueue churn before it fails. (:max-requeues, default 10000, is a runaway backstop, not the budget: give-up is age-based so a slow handler-owning worker cannot lose a job to attempt exhaustion.) -
Enable rate limiting (
:wagoe/http :rate-limit :enabled? true) with an active Redis cache for a global limit across replicas. In:prodthis is not optional — the boot fails without a reachable cache.:wagoe/cacheis:inactivein the shipped prod profile; move it to:activefirst. -
Keep
replicas × maximum-pool-sizeunder Postgresmax_connections. -
Confirm your load balancer points health probes at
/health/ready(503-aware), not/health/live. -
For WebSocket: use
:provider :redison:wagoe/realtimeto scale across replicas (ADR-035). Sticky sessions / single-node are only required with the default:in-memoryprovider. -
Deploy Redis, and a load balancer for N replicas. The root
docker-compose.ymlis a single-instance dev stack (db, app, docs, dev-tools — no Redis); the replica and per-service references aredeploy/compose/multi-instance.yml,deploy/compose/per-service.ymlanddeploy/k8s/wagoe.yaml.
Functional decomposition (slicing services out)
The third axis: run a module (or a few) as its own process, scaled and deployed
independently of the rest. This is where the ports.clj seam pays off most — and
where the most net-new infrastructure is needed. It is not free "by config"
today, but the architecture is positioned for it.
What already enables it
| Asset | How it helps |
|---|---|
Per-module activation |
Modules are gated by |
The protocol seam |
Consumers depend on the protocol (e.g. |
Wire format ready |
Muuntaja (JSON / EDN / Transit) is already in the HTTP stack ( |
Remote-adapter template |
|
Clean data boundaries |
|
An acyclic dependency graph |
|
Context plumbing |
|
What must be built
-
✅ Generic remote-port adapter — shipped in BOU-90.
wagoe.platform.shell.rpc.client/remote-adapterreturns a value implementing any protocol by calling a service over HTTP, built from the protocol’s own:sigsso every module’s port works without a bespoke client. The counterpart…rpc.server/rpc-handlerserves a protocol from the process that owns it. correlation-id, tenant and auth ride the same headers the interceptor pipeline already uses. See The remote-port adapter. -
Network resilience — timeouts, retries and a shared circuit breaker are in the remote-port adapter (BOU-90, BOU-285). Service discovery is not: URLs are configuration, hardcoded or env-supplied. The
libs/externaladapters still use:throw-exceptions falsewith no retry/breaker of their own. -
✅ Break the allowlisted dependency cycles — done.
allowed-cycle-edgesincheck_deps.cljis#{};admin↔userandplatform↔{user,tenant,admin,workflow,search}were dissolved rather than allowlisted, so no module pair is blocked from separation by a cycle. -
✅ Async option — shipped in BOU-93 as
libs/events:IEventPublisher/IEventSubscriber/IEventHistory, with a Redis Streams adapter (at-least-once, consumer groups) and an in-memory one. A publisher does not know who is listening and is unaffected if a consumer is down — the complement to the synchronous remote-port adapter. -
Data ownership decision — schema-per-tenant assumes co-located modules in one Postgres. Across services either share the DB (pragmatic) or give each service its own; there are no distributed transactions, so split writes become eventual-consistency.
-
✅ Service launch mode — shipped in BOU-91:
java -jar wagoe.jar service <module>…boots a named subset plus the platform, and starts that module’s RPC endpoint when one is configured. See Running a module as a service.
Running a module as a service
Running configuration — compose files, Kubernetes manifests, the environment variables and what has actually been brought up — is in Deployment Topologies.
service boots only the modules named and the platform they need:
java -jar wagoe.jar service payments # one module
java -jar wagoe.jar service user tenant # several in one process
In the test profile service user runs 20 of the application’s 34 components.
/health answers, /web/login answers, /api/v1/tenants is a 404 — the tenant
module is not there to serve it.
Which keys belong to which module is declared, not guessed:
;; config.edn, under :active — merged over
;; wagoe.system-config/default-service-catalogue, replacing an entry of the same name
;; rather than merging into it
:wagoe/services
{:payments {:keys [:wagoe/payment-provider]
:rpc {:protocol 'my.app.payments.ports/IPaymentProvider
:component :wagoe/payment-provider}}}
The keys must be the ones the config actually emits. Naming a component that
does not exist does not fail — the real component is then claimed by nobody,
counts as platform, and runs inside every service. wagoe.service-launch-test
asserts every emitted key is either claimed by a module or listed as platform,
which is the only thing that catches it.
A key no entry claims counts as platform and runs everywhere. That is the deliberate failure direction: a catalogue that has fallen behind boots a service larger than it needs to be, rather than one missing a component it depended on.
With :wagoe/rpc in config, a service also starts the remote-port endpoint on
its own port, so the rest of the deployment can call it. Without it the boot
says so explicitly — a module running alone that nothing can reach is a process
doing no work, and it otherwise reports itself healthy.
The mechanics are in wagoe.platform.core.system-selection, and the one thing
worth knowing about them: refs to unselected modules are removed before the
dependency closure is taken. :wagoe/http-handler refers to every module, so
following its refs first pulls the whole system back in — and the result still
boots and still passes a health check, which is what makes the mistake worth
naming.
The remote-port adapter
A caller depends on a protocol, never on a concrete record. So a module can move into its own process without its callers changing — provided something implements that protocol by making a network call. That is this:
;; In the process that owns payments — serve the protocol on its own listener
(jetty/run-jetty (rpc-server/rpc-app pay-ports/IPaymentProvider provider
{:service-key service-key})
{:port 3001 :join? false})
;; In the process that consumes it — a value that satisfies IPaymentProvider
(def payments
(rpc-client/remote-adapter pay-ports/IPaymentProvider "http://payments:3001"
{:timeout-ms 5000
:service-key service-key
:cache cache ; enables the breaker
:context {:correlation-id id :tenant-id t}}))
(ports/create-checkout-session payments {...}) ; unchanged at the call site
rpc-app is a standalone Ring handler, not a route map for a module’s :api
or :web slot, and that is deliberate. The router rewrites both: :api paths
gain the version prefix and :web paths gain /web, so a client on the
default :path gets a 404 either way. :web is worse than merely wrong — a
POST there is CSRF-validated when CSRF is enabled, so the call is rejected 403
by a check meant for browser forms, for which a service-to-service caller has
no token.
Underneath the mechanics: this endpoint invokes port methods, and the public
listener is not where it belongs. A sliced-out service serves it on a listener
reachable only from inside the deployment — which is what the service launch
mode (BOU-91) starts. A service that serves it elsewhere tells the client with
:path.
The adapter is built from the protocol’s :sigs, so adding a method to a port
carries it across the hop with no client change. Each adapter is its own object
implementing that protocol’s interface — not a shared type extended in place,
which would make every adapter already built satisfy each newly adapted
protocol, so a payments adapter would answer satisfies? for ICache and send
cache calls to the payments URL.
What it does, and what it deliberately does not:
Context |
|
Wire format |
|
Transport errors |
Returned as data, in the |
Thrown exceptions |
Raised again on the near side, keeping the |
HTTP status |
A typed error keeps the status it has in-process, read from the same
|
Malformed requests |
Answered, not thrown. The endpoint is reachable by anything that can post to
it, so a body carrying no operation — or an operation that is not a name, or
the wrong number of arguments for one — comes back as Argument count is checked against the protocol’s own |
Retries |
Only failures where the call may not have completed, and only
Which failures count as never-executed is decided conservatively. A
The HTTP client’s own retry handler is disabled. Apache HttpClient resends
low-level I/O failures before this code sees an outcome, so An answer is never retried, whatever |
Exposure |
The server resolves an operation against the protocol’s own |
Authentication |
A service key in This is not a substitute for keeping the endpoint off a public listener. It is the part that can be enforced in code. |
Service discovery |
Not included — the URL is configuration. Hardcoded or env-supplied for now, as the MVP in this document assumes. |
Circuit breaker |
Opt-in: pass a The failure count is an atomic increment, not a read-modify-write: many callers hitting one outage at the same moment is what a shared breaker is for, and it is exactly when a read-modify-write loses increments — each reads the same value and writes back the same successor, so a burst advances the counter by one and the breaker never trips. A probe that fails reopens the window from that moment and releases its lease, so the outage is not forgotten on the original window’s schedule and the next window can still be probed. An invalid It protects the traffic after the burst, not the burst. Calls issued at the same instant all pass the check before any of them has failed. Trips on consecutive A refused call returns If the cache itself is unreachable the breaker fails open — the worst case is the behaviour of no breaker, which is better than a cache outage taking every remote call with it. |
Sliceability by module
| Module | Effort | Why |
|---|---|---|
payments |
Easy |
Zero internal Wagoe deps (only Maven). Already a self-contained provider. The natural pilot for the remote-adapter pattern. |
core, observability |
Easy |
Leaf / infra; no sibling deps. (Usually shared libs, not standalone services.) |
user, tenant, external |
With work |
Depend on platform + the in-process service assumption in middleware. The remote adapter exists and the cycles are gone; what remains is deciding data ownership and pointing consumers at a remote adapter instead of the local component — a wiring change in the application’s config, which is why |
admin, search, workflow |
Entangled |
|
|
Tip
|
Recommended path: prove the remote-port adapter by extracting payments as a
standalone service (zero internal deps, and the one the reference topology
uses), then |
Known gaps & roadmap
The architecture delivers the promise; these are the concrete pieces that make "scale by configuration" fully true. Tracked under the BOU-84 spike:
-
✅ Realtime Redis pub/sub adapter — shipped in BOU-85 (ADR-035). WebSocket is now replica-safe via
:provider :redison:wagoe/realtime. -
✅ Graceful connection draining — shipped in BOU-86. Configurable shutdown grace (
:wagoe/http :drain-timeout-ms) lets rollouts finish in-flight requests; JettyGracefulHandler+setStopTimeoutwired inwiring.clj. -
✅ Default rate-limit wiring — shipped in BOU-87. Config-driven
http-rate-limit-protectionis in the default pipeline; enable via:wagoe/http :rate-limitand it uses the Redis cache for a cross-replica limit (per-process fallback documented). -
✅ Jobs hardening — shipped in BOU-88. Missing-handler jobs are re-enqueued (bounded) instead of silently dead-lettered, an empty-registry worker warns at startup, and scheduled-job promotion is an atomic claim (
ZREM/swap-vals!) so a due job runs exactly once across workers. -
✅ Deploy topology reference — done (BOU-89). Compose + k8s with N replicas, Redis, a load balancer, the web/worker split and a module-as-a-service, in Deployment Topologies. No instance-id variable: nothing in the codebase reads one, so adding it would have been decoration.
For functional decomposition (the bigger bet):
-
✅ Generic remote-port adapter + RPC envelope — done (BOU-90), with the circuit breaker in BOU-285.
clj-httpclient implementing a module protocol over transit, context propagation, typed errors, retry bounded to calls that never executed. -
✅ Service launch mode — done (BOU-91).
wagoe.mainboots a named module subset as an independent service, with its RPC endpoint on its own port. -
✅ Break allowlisted dependency cycles — done (BOU-171/192/193/194/198).
admin↔userandplatform↔{user,tenant,admin,workflow,search}are gone;check_deps.cljallowlists no cycle edges, so the gate now fails on any cycle at all. -
✅ Event-bus port + adapter — done (BOU-93).
libs/events, Redis Streams and in-memory. Kafka would be another adapter behind the same ports, not a change to them.
What is left, in order:
-
Service discovery — remote-port URLs are configuration today. Fine for a fixed deployment, insufficient once services move.
-
Data ownership across services — schema-per-tenant assumes co-located modules in one Postgres (see the entry above).
-
Activate
:wagoe/eventsin the prod profile — it is configured intestonly, so the event bus is currently a test-profile capability in practice.