Running Both Providers in Parallel Without Your Application Knowing

The default instinct when swapping database providers is simple: update the connection string, flip a flag, and hope. That approach forces a hard cutover, and hard cutovers collapse the moment the new provider misbehaves under real traffic. According to a Caylent survey cited by IBM, only 6% of organizations finish their most complex database migrations on time. That number alone tells you the stakes without needing extra drama attached.

Why does the failure rate sit that high? Three root causes keep recurring: the new provider behaves differently once real production traffic hits it, schema incompatibilities slip past tests that looked thorough on paper, and once traffic has moved, rolling back cleanly turns out to be much harder than the migration plan assumed. The engineering approach is to run both providers at once. Keep the old one authoritative, let production traffic validate the new one in the background, and only cut over once confidence has actually been earned rather than assumed.

But running two providers in parallel creates its own set of questions. Which one is authoritative at any given moment? What happens the instant their results disagree? The application stays completely unaware of all this machinery because a deliberately built abstraction layer, not the application itself, manages which provider is authoritative and how disagreements are resolved. Answering those questions requires a deliberately built abstraction layer. That layer is the subject of this piece.

The three foundational patterns the abstraction layer can implement

Three patterns recur in real migrations, and each one trades consistency for simplicity in a different place.

Shadow writes, or dual-write, are illustrated cleanly by LaunchDarkly's documented migration from MongoDB to CockroachDB. Feature flags conditionally wrapped new code paths that read from and wrote to CockroachDB, while MongoDB stayed authoritative the entire time. CockroachDB's writes were treated as disposable, shadow requests generated by live production traffic rather than synthetic test data. Percentage rollouts let engineers turn shadow-write volume up or down on the fly, no deployment required. If the shadow-write logic lives inside the application, "transparent to the application" is already a broken promise. The abstraction layer has to own the dual-write itself, so application code never has to know a second provider exists.

Shadow tables build a synchronized duplicate of the data inside or alongside the production system, while production keeps running without interruption. It offers stronger consistency guarantees than dual-write and an easier rollback path than a blue-green deployment. InfoQ has documented large-scale cases at GitHub, Shopify, and Uber where this approach held up data integrity through the entire migration. It's not limited to provider swaps either. It fits microservice extraction and incremental schema refactoring just as well.

Asynchronous propagation with a single source of truth. Here, the old schema stays authoritative for every write, and changes propagate to the new schema asynchronously. A live Postgres schema migration account on dev.to describes this approach as a way to reduce the risk of database locks stacking on top of each other. The trade-off is lag: the new provider is always a little behind, and read traffic can only shift toward it once that lag is measured and judged acceptable.

Choosing the right pattern requires weighing how much synchronicity the workload actually needs, how much rollback risk the team can tolerate, and how much replication lag, if any, is acceptable. The proxy layer's design falls out of those answers.

What the abstraction layer must look like structurally

The application should only ever see one interface, such as a connection string, a driver endpoint, or an SDK client. That interface cannot change just because the provider behind it does.

The layer itself usually takes one of three forms. It might be a sidecar or reverse proxy process, a custom TCP or HTTP proxy that intercepts wire-protocol traffic and fans it out to both providers. It might be an adapter or repository layer baked into the application codebase, wrapping both provider clients directly, less transparent, but easier to instrument closely. Or it might be a dedicated dual-write coordinator service sitting between the application and both providers, owning write ordering and fan-out on its own.

Whichever form it takes, certain responsibilities have to live inside the layer and never leak into application code:

• Routing decisions: which provider gets which reads, which gets the writes, and under what conditions that assignment changes
• Write ordering: making sure the authoritative provider commits before the shadow provider ever sees the write
• Error isolation: a shadow-provider failure must never bubble up to the application as an error
• Result capture: storing both providers' responses so they can be compared later
• Traffic-shaping controls: dials an operator can turn without touching a deployment pipeline

There's one narrow sense in which this layer has to hold state. It needs to know, at all times, which provider is currently authoritative and what percentage of read traffic is flowing to the new one. That state belongs in something externalizable, a config store or a feature-flag service, so it can change instantly without a code push.

Shadow-write mechanics: keeping the authoritative provider safe while the shadow runs

The authoritative write has to complete and get acknowledged before the shadow write even gets dispatched, and getting this sequencing wrong undermines the whole approach. The authoritative write has to complete and get acknowledged before the shadow write even gets dispatched. Writing to both at the same time, in a way that lets the shadow succeed while the authoritative write fails, defeats the entire point of having an authoritative provider in the first place.

Dispatch can happen two ways. Synchronous dispatch, on the same request path, adds latency equal to whatever the shadow provider's write time is, so it's really only appropriate during low-traffic validation windows. Asynchronous dispatch, through a queue, lets the authoritative write return immediately while the shadow write processes out of band. That's the preferred mode for production traffic, though it introduces a replication lag the layer has to track continuously.

Failure has to be handled explicitly, not left to whatever the framework does by default:

• If a shadow write fails, log it with full context, bump a divergence counter, and never surface the error to the caller It signals a schema incompatibility the migration plan missed, and it warrants investigation rather than silent retry

Idempotency deserves its own attention. Shadow writes need to be safe to retry; the coordinator should assign each write a deterministic ID so a retry doesn't double-insert on the shadow side. And for providers that process updates asynchronously in the background, through merges or mutations, the shadow provider's visible state may lag even after the write was acknowledged. Any comparison layer built on top of this has to account for that timing window, or it will flag phantom divergences that are really just timing artifacts.

Shaping read traffic: the graduated shift from old provider to new

The shift from old to new provider should look like a ramp rather than a switch. Start by routing a small slice of read queries to the new provider while the old one still handles all reads authoritatively, then widen that slice as confidence builds.

One documented live migration, described in that same dev.to account of a Postgres schema migration, followed a graduated sequence of increasing read-traffic slices moved to the new schema, with the ability to snap traffic back to the old schema the moment something looked wrong.

Three levers control how that routing decision gets made. Percentage-based routing uses a random or hash-based selector to send N% of queries to the new provider, where N is a config value rather than something baked into a deploy. Query-type-based routing sends only read-only, non-critical queries first, reporting jobs, background analytics, while transactional or latency-sensitive reads stay on the authoritative provider longer. User-cohort-based routing sends traffic from internal users or a canary customer segment first, and only lets general production traffic follow once that cohort has confirmed things look right.

None of this is safe to attempt without an instant rollback path. A single config change should be able to route 100% of reads back to the authoritative provider, with no deployment in between. That guarantee is what makes the graduated approach worth trying at all.

While traffic ramps up, watch the latency distribution (p50, p95, p99) on the new provider compared to the old, the error rate on the new provider, and the divergence rate coming out of the comparison layer. Divergence is a signal to investigate, not an automatic verdict against the new provider.

The 50% mark tends to be where things get interesting. At that level, both providers are carrying meaningful load, and issues that never appeared at 5% or 10%, caching quirks, connection-pool exhaustion, hot-partition behavior, often appear for the first time.

Result comparison: detecting divergence before it becomes a user-facing problem

For every read routed to the new provider during the shadow or graduated phase, the coordinator captures both providers' results and compares them. Divergences get logged. They never reach the caller.

What counts as a divergence worth worrying about? Not every mismatch is a bug. Row-count differences can point to a real problem, since the two providers may be sitting on genuinely different data states. Value differences on the same row might be a schema transformation issue, a type coercion quirk, or just a replication lag artifact catching a write mid-flight. Ordering differences can be perfectly fine if the query never had an ORDER BY clause and the application never depended on row order, though they need to be normalized before any comparison runs. And for providers that process changes asynchronously in the background, a result that looks wrong right now might converge correctly after a short delay. So the comparison layer needs to tell "diverged right now" apart from "diverged, and stayed diverged after a grace period."

When a divergence does get logged, it should carry the full query text, both result sets (or hashes of them, if the sets are large), the timestamp and read-routing cohort the query belonged to, and whether the divergence resolved on a retry after some delay.

Divergence rate is a natural signal for controlling migration pace. Tracking it against a threshold over a sustained observation window gives a principled basis for deciding when to allow the next jump in traffic percentage, or the eventual cutover. This same mechanism doubles as a safety net for schema correctness generally: subtle differences in indexing behavior, encoding, or background processing timing between two systems will appear here as divergences long before any of them reach an actual user.

Keeping the state consistent across both providers during the parallel window

The longer the parallel window stays open, the more room there is for small divergences to compound into bigger ones. A missed shadow write turns into a missing row, and a missing row turns into a wrong join result three queries downstream. That's the fundamental tension of running two providers side by side: time is not free.

Async shadow writes always carry some replication lag. What matters is whether that lag is bounded and watched closely. The coordinator needs to expose a lag metric, some measure of how far behind the shadow write queue actually is, and read routing to the new provider should throttle automatically once that lag crosses a defined line.

Schema drift is its own hazard. If the authoritative provider's schema changes mid-migration, a column added, a type changed, the shadow write path has to change in lockstep. Missing that step causes shadow writes to start failing or, worse, silently diverging.

The shadow table approach reduces some of this risk, since it maintains the duplicate through a controlled synchronization mechanism rather than depending on the application's write path directly. That's part of why InfoQ's cases at GitHub, Shopify, and Uber point to it as a way of holding data integrity steady through the migration window.

Race conditions are still possible even with careful design. Two writes to the same record, arriving close together under high concurrency, might commit in different orders on each provider. Writing to the authoritative provider first and propagating second reduces this risk, but it doesn't erase it entirely. Throughout all of this, the authoritative provider has to stay in a state where a full rollback is possible with zero data loss. That means the shadow provider never becomes the sole owner of any write, no matter how far along the migration is.

Operating the proxy layer in production: observability, failure handling, and the cost of the parallel window

The proxy layer is a production system in its own right, and it needs the same treatment any production system gets: health checks, alerting, runbooks written down before something breaks at 2 a.m.

A handful of metrics matter more than the rest. Shadow write queue depth and drain rate. Shadow write error rate, broken down by type, constraint violation versus timeout versus connection failure. Read routing distribution, to confirm what fraction of traffic is actually landing on the new provider versus what the config claims. Result divergence rate, and whether divergences resolve on their own or persist. And latency added by the coordinator itself, since a proxy that becomes the bottleneck has defeated its own purpose.

Failure handling needs a clear default: fail safe. If the coordinator can't reach the shadow provider, it should complete the authoritative write anyway and log the shadow failure separately. It should never block the authoritative path while waiting on the shadow side to respond.

Running two providers at once means paying for two providers at once, compute and storage both, for as long as the parallel window stays open. That's a real cost, not a hidden one, and it's why keeping the window short matters financially as much as technically. The graduated read-traffic approach, gated by divergence-rate thresholds, is the mechanism that keeps that window from dragging on longer than it needs to.

Workloads with unusually high query volume during validation, AI agents probing the new provider repeatedly while testing correctness, deserve a specific note here. Per-query pricing models penalize that validation activity disproportionately, since every probe carries a cost. Compute-hour pricing models, where the meter runs on time rather than per question asked, tend to make that validation window considerably cheaper to sustain.

And logging needs to be complete, not sampled. Every query the coordinator routes, to either provider, should get logged in full. Divergence investigations depend on having the exact query text and timing available after the fact, and partial logs turn a five-minute investigation into a guessing game.

The conditions that make a clean cutover possible

Cutover is a threshold the new provider has to earn its way across, by clearing a defined set of gates rather than by a team simply deciding enough time has passed. It's a threshold the new provider has to earn its way across, by clearing a defined set of gates rather than by a team simply deciding enough time has passed.

Before flipping authority to the new provider, the divergence rate must have stayed below its threshold for a sustained window at a meaningful level of read traffic, and the shadow write error rate must be low and stable, with any remaining errors understood well enough to assess their impact rather than an unexplained constraint violation sitting in the logs. Only once both conditions hold does cutover stop being a gamble and start being the logical next step the data has already justified.