Integration Engineer
Integration & API Platform
A legacy-modernization stack built as one system: a contract-first SOAP service, a REST facade with Redis, a Kong API gateway, Kafka event streaming with schema governance, an MCP server that opens the platform to AI agents, a GraphQL facade, OpenTelemetry tracing across all of it, and a durable Temporal saga — over a real PostgreSQL inventory domain.
Navigate by integration skill:
The full request and event path. Click to enlarge.
What this is
The technologies that usually appear together in enterprise integration work — SOAP, XML schemas, service contracts, Redis, an API gateway, Kafka, and now MCP — built as one system rather than seven disconnected demos. A single write travels through every layer:
Kong → REST facade → SOAP service → PostgreSQL trigger →
outbox → relay → Kafka → consumers → read model
- Domain: runs on the inventory schema from the Databases & Data Platforms track — the same triggers and views, now exposed through service contracts.
- Verified, not just written: every phase has a runnable test script; results quoted below are from actual runs.
Contract-First SOAP Service
The XSD is hand-written first. JAXB generates the Java classes and Spring-WS generates the WSDL — neither is hand-edited or committed. The schema is the source of truth; the code is an implementation detail.
- Validation nobody wrote: there is no
if (!sku.matches(...))in the codebase. A validating interceptor enforces the XSD at the boundary, and those constraints are published to consumers inside the WSDL. - Typed faults: errors carry a schema-defined
InventoryErrordetail, so consumers branch on a stablecodeinstead of string-matching English prose. - Events, not consequences: the write path inserts only into the movement ledger — a database trigger derives the stock balance.
Verified: 15 tests, none requiring a database. Contract tests fail the build if an operation disappears from the published WSDL — proven by deliberately breaking it.
REST Facade & Redis
The strangler-fig pattern: new consumers get clean JSON while the SOAP contract keeps serving existing integrations unchanged. The API models resources, not a renamed copy of the SOAP operations.
- Cache-aside with invalidation: differentiated TTLs — products 5 minutes, stock 30 seconds — and explicit eviction on every write.
- Atomic rate limiting: a Lua script makes
INCR+EXPIREindivisible, closing the race that can leave a key with no expiry. - Idempotency keys: a client that times out and retries cannot double-count stock.
- Fail-soft: the cache is an optimisation, not a dependency.
Verified: cached 160 → wrote +20 → key evicted → next read returned
180 (not stale). 70 requests against a 60/min limit gave exactly 60×200
and 10×429. With Redis stopped, reads still returned
200 and writes 201.
Kong API Gateway
Everything Kong knows lives in one reviewable kong.yml — services, routes,
plugins, consumers, credentials. DB-less and declarative: no admin-UI
clicking, no config drift between environments.
- Auth left the application: the
key-authplugin rejects unauthenticated calls before they reach any service. Zero auth code in the app. - Deliberate asymmetry: the SOAP route requires no API key. Legacy consumers get throttling and observability now, authentication after they migrate — meeting existing consumers where they are.
- Shared-state limits:
policy: redis, becausepolicy: localsilently allows N× the limit on an N-node cluster. - Tracing: a correlation ID generated at the edge and carried through every hop.
Verified: 14 gateway tests passing — auth rejection, header transformation, rate limiting per consumer, SOAP proxying, and Prometheus metrics.
Kafka Events & Schema Registry
Events are published via the transactional outbox, not a dual write. The business change and its event commit in one database transaction; a relay publishes them afterwards.
- Why it matters: a naive
insert()thensend()can lose the event on crash, or announce one for a transaction that later rolls back. - Honest guarantee: delivery is at-least-once, so both consumers deduplicate — one in SQL, one in memory.
- CQRS read model: a daily-totals projection rebuilt purely from the event log.
Verified: replaying every event left the read model byte-for-byte identical, with duplicates ignored and no duplicate alerts fired.
MCP Server & LangGraph Agent
The platform's fourth contract, this one written for language models. An MCP server publishes the system as six tools, three resources and two prompts; a LangGraph agent on Azure OpenAI consumes it — and so can Claude Code, over the same stdio transport.
- Why it matters: the first three contracts were read by programs someone wrote and tested. This one is read at runtime by something that improvises — it retries, invents plausible arguments, and will call the write endpoint twice.
- The finding: every guard already built for human-written clients
held it without modification — the XSD's closed enumerations, the facade's
Idempotency-Key, and the gateway's key-auth and rate limiting. - Writes are gated: the graph pauses inside the tool call, checkpoints, and waits for a human. Only the write — interrupting before every read would make the agent unusable.
- Beyond tools:
inventory://contract/openapiserves the Phase 2 spec itself as an MCP resource — the same document governing the REST consumer and the LLM consumer. - Runs with no credentials: with no API key a scripted offline model still drives the real tools against the real gateway.
Verified against the live stack: 34 offline tests plus eight checks
against the running platform — the agent's correlation ID survives the gateway
unchanged, and two identical record_movement calls in one session produced
one database row (quantity 28 → 29, the second returning
replayed: true) through an idempotency key derived from the arguments
rather than the attempt.
GraphQL Facade
The phase that audits the rest of the project by breaking it. Moving query composition from the server to the client invalidates three mechanisms the earlier phases depend on — and each one needs a different replacement.
| Relied on | Why GraphQL breaks it | What replaces it |
|---|---|---|
| The URL as a cache key (Phase 2) | One URL, and the response shape varies per caller | Cache the entity, keyed by SKU |
| Counting requests (Phase 3) | One query can cost 1 backend call or 50 | Query cost analysis, in the server |
| Version numbers (Phases 1, 2, 4) | GraphQL has no versions at all | @deprecated plus usage tracking |
- Measured, not claimed: a backend-call counter is reset before each query and read after it, so the numbers below are outputs.
- Cost analysis done properly: the stock complexity calculator scores a 600-row list the same as one row — the exact query that hurts, rated as cheap. A list-aware calculator multiplies by the expected page size, and the estimate is honestly an estimate: you are choosing how wrong to be, and which way.
- What batching actually fixes: duplication and parallelism — not cost. With no bulk operation upstream, six distinct SKUs still cost six calls.
Verified against the live stack: 13 assertions plus 9 unit tests.
lowStock { sku } costs 1 backend call;
adding one nested field costs 7; a deeper selection is
refused having spent 0. All three are one HTTP POST to one URL, and
the gateway counted each as a single request against the same quota.
Try it: a read-only build of this same facade — one container, no database, scale-to-zero. Same resolvers, same validation, same cost analysis; only the data source is swapped, for a snapshot of the real database. GraphiQL opens on its own boilerplate, so paste one of these:
1 — six SKUs below their reorder point. One backend call.
{ lowStock { sku warehouseCode quantity deficit } }
2 — add one nested field. Same endpoint, same single HTTP request — seven backend calls.
{ lowStock { sku quantity product { name reorderQuantity } } }
3 — go one level deeper and the server refuses it at 2701 complexity, before executing anything.
{ lowStock { sku product { name stockLevels { warehouseCode quantity } } } }
Distributed Tracing
OpenTelemetry across every phase, into Jaeger. Phase 3's correlation ID told you which request you were looking at. A trace tells you what it did — the order, the nesting, and where the time went.
- The same bug, twice: the trace died at the SOAP hop, exactly as
the correlation ID once did —
WebServiceTemplateis not auto-instrumented, so nothing sendstraceparentunless you write it. Two healthy services, two unrelated traces, no error anywhere. - Free instrumentation: the Redis
get/setspans and the Lua rate limiter'sevalshawere never written by hand. That is the case for a standard over a bespoke logging convention. - Sampling, honestly: 1.0 on a laptop; the README says plainly why that cannot ship, and what tail-based sampling is actually for.
What the trace caught that a counter could not: Phase 6 measured its
N+1 as “7 backend calls” and was right — but the waterfall showed those calls running
strictly sequentially, while the batch loader used .parallel() and
both the code comment and the README claimed concurrency. The
.parallel() was deleted rather than the sentence quietly softened.
A counter can only confirm what you thought to count.
Durable Orchestration
A stock-transfer saga on Temporal, and the deliberate counterpart to Phase 4. Choreography answered how do parts of this system react to each other. This answers who is responsible for finishing.
| Phase 4 — choreography | Phase 8 — orchestration | |
|---|---|---|
| Owns the outcome | nobody | the workflow |
| State lives | implied by the event log | in the workflow, explicitly |
| On failure | each consumer copes alone | compensations unwind, in reverse |
| “Where is it now?” | replay the log and infer | a query |
| Adding a step | deploy a new consumer | edit the workflow |
- The state no event represents: query a transfer mid-flight and the
goods have left
WH-WESTand not reachedWH-EAST— they are in neither. Phase 4 could not have an event for that, because in transit is the gap between two events. - Failure is the interesting half: a rejection, a timed-out approval and an unavailable carrier all unwind the same way, because each compensation is registered immediately after the step it undoes.
- Retries meet idempotency from the other side: Temporal does not make
retries safe, it makes them certain — so every activity carries the same
Idempotency-Keythe facade has accepted since Phase 2. - The honest cost: the workflow knows every step, which is exactly what Phase 4 spent effort decoupling. Worth paying only when something must complete.
Verified against the live stack: 11 assertions covering all three
unwind paths — after a rejection and after a carrier failure, the stock is back in
WH-WEST to the unit, and the result still names the movement it made and
reversed rather than hiding it.
The idea the whole project builds toward
Phase 1 and Phase 4 are both contract-first. The XSD and the Avro schema are both hand-written, and both generate code. The difference is that only one has a referee:
| XSD (Phase 1) | Avro + Schema Registry (Phase 4) |
|---|---|
| Contract lives in your repository | Contract lives in a shared registry |
| Validates one message | Validates the schema change itself |
| Nothing stops a breaking edit | Registry refuses incompatible changes |
| Consumers find out in production | Producer finds out at deploy time |
The registry accepts an optional field with a default, rejects a required
field without one, and — surprisingly — accepts removing a required field under
BACKWARD compatibility. That last result is the point: BACKWARD
and FORWARD answer different questions, and choosing a policy means deciding
who is allowed to upgrade first.
Phase 5 adds a fourth answer to the same question. The XSD published a contract for programs, OpenAPI for developers, Avro for streams — and MCP for language models. What changes is not the schema but the consumer: the first three were read once, by someone who then wrote code. The fourth is read at runtime by something that improvises. That is the argument for building the platform first and the agent last.
Phase 6 supplies the fifth answer, and the most awkward one: GraphQL has no version
number to offer. It keeps serving a @deprecated field until telemetry says
nobody asks for it — which only works if somebody is actually measuring. A deprecated
field nobody tracks is a field with an apology attached.
Bugs found by running it
- PostgreSQL cannot infer a parameter's type inside an
IS NULLtest — optional-filter queries failed with a message that never mentionsIS NULL. - A distributed trace that silently died at the REST facade. The gateway generated a correlation ID and the facade dropped it; "one trace across every hop" was documented but untrue until it was tested.
- A cache that had become a hard dependency — every read returned
500when Redis went down. "X-Gateway: kong"producing the value" kong"— Kong keeps everything after the colon, whitespace included, and a presence-only test assertion missed it.- An MCP server that bound on top of the gateway — the framework
defaults to port
8000, which is Kong's proxy port here, so every tool call hit Kong's router and returned404. - A pattern is not a value set — the warehouse regex
WH-[A-Z]{2,4}admits far more codes than the database actually holds, so a model reading only the schema invented plausible ones. The contract now publishes the real list. - A silent wrong answer, found only by running it — an unknown warehouse
returns
200 []rather than404, and an empty array is indistinguishable from "stocked here, quantity zero". The agent confidently reported "no stock at WH-NYC" for a warehouse that does not exist. The tool now raises an error naming both possibilities. - A complexity limit that protected nothing — the stock calculator scores fields in the query document, so a narrow selection over a six-hundred-row list rated identically to one row. The query that actually hurts, scored as cheap. Security theatre until the calculator was made list-aware.
- A config validator that crashed inside its own error reporter — a
gateway plugin pointing at a service that did not exist produced
attempt to concatenate local 'k'and no hint of the real problem. - A test suite that competed with the system for its own rate
limit — Phase 8's checks polled through the gateway while the workflow's
activities did too; together they passed 20/minute, the activities got
429, and every transfer compensated. Correct behaviour, meaningless test. - A
.parallel()that never parallelised anything — the code claimed the batch fetches ran concurrently; the trace showed six sequential spans and no pool threads. Found only because Phase 7 drew the picture.