Data Warehouse Insider

Replicating PHI in Real Time When Your Compliance Team Forbids Third-Party Data Storage

Build PHI replication pipelines without vendor storage using log-based CDC and idempotent writes.

· 8 min read
Cover illustration for “Replicating PHI in Real Time When Your Compliance Team Forbids Third-Party Data Storage”
Streaming Pipeline Operations · November 3, 2026 · 8 min read · 1,829 words

Real-time PHI replication can meet a no-third-party-storage mandate when every vendor-managed intermediate layer acts as a passthrough. The architecture needs log-based change data capture, controls that keep PHI out of persistent logs and buffers, and idempotent writes at the destination. A business associate agreement must still be signed before PHI transfer begins, because a vendor that sends PHI has duties even when it stores none.

Why "no third-party data storage" is an architectural instruction, not a veto on real-time replication

A prohibition on third-party PHI storage defines where data may rest. The mandate bars storage in infrastructure outside the covered entity’s control, while replication needs a path from an authorized source to an authorized destination. A vendor-managed store in the middle violates that architectural requirement even if it makes delivery easier. A conduit can process events without writing PHI to persistent storage.

The HIPAA Privacy Rule establishes the business associate requirement at 45 CFR §164.502(e), with a parallel Security Rule requirement at §164.308(b). A vendor that creates, receives, maintains, or transmits ePHI on a covered entity’s behalf must operate under a BAA before transfer begins. A signed agreement does not override a stricter organizational prohibition on vendor storage. What does a pipeline look like when every intermediate layer is a passthrough?

What HIPAA's Security Rule actually governs in a replication pipeline

The HIPAA Security Rule governs the confidentiality, integrity, and availability of ePHI, including data passing through a replication pipeline. The Privacy Rule governs who may access PHI and under what conditions. The Breach Notification Rule governs reporting when exposure occurs. A pipeline can avoid persistent storage and still need access controls, transmission records, and a defined process for handling a breach.

The Security Rule’s safeguards span administrative, physical, and technical controls. Administrative safeguards include risk analysis of systems that store or process ePHI, assigned accountability, workforce training, activity log review, and contingency planning. Physical safeguards concern the data centers, devices, and media where ePHI sits. Technical safeguards cover access controls, audit logging, and encryption. For the architecture described here, unique user IDs, role-based access, MFA, and encryption form the practical control baseline throughout the pipeline.

PHI access, modification, and transmission need tamper-proof records that are available for review, including activity between the endpoints. This requirement creates an important distinction between recording an event’s handling and retaining its PHI payload. The former supports accountability without making the latter necessary. Batch backup requirements, RPO/RTO targets, and disaster recovery planning are outside this streaming-replication discussion, although contingency planning remains part of the broader safeguard framework.

How log-based CDC moves data without storing it

Log-based change data capture reads changes from a database’s transaction log and propagates inserts, updates, and deletes to a destination. It avoids querying live tables to discover each change. The database log supplies the change history, so the replication vendor does not need its own persistent staging store. In a passthrough design, events travel through in-memory or transient buffers before reaching the destination. PHI is not written to vendor-managed persistent storage between the endpoints.

PostgreSQL’s Write-Ahead Log is also its crash-recovery log. CDC reads that log without using production query paths. A replication slot prevents PostgreSQL from recycling WAL segments beyond the consumer’s last-read position, allowing the consumer to reconnect and resume. That recovery mechanism has an operational cost. When a consumer falls behind or goes offline, retained WAL accumulates on the source’s disk, so replication slot lag requires active monitoring to prevent disk exhaustion.

MongoDB Change Streams read changes from the operations log on replica sets or sharded clusters. The deployment configuration is a hard prerequisite. A standalone MongoDB instance has no oplog available for this streaming mechanism and must migrate to a replica set or sharded configuration before CDC is available. This is a source-system requirement. The passthrough design must begin with a source capable of supplying the change stream.

DynamoDB Streams exposes item-level inserts, updates, and deletes with a 24-hour retention window. That window bounds the time available for recovery. A consumer that falls behind beyond it loses the expired events for good, so pipeline uptime must be monitored when PHI is involved. Unlike PostgreSQL’s accumulating WAL, the constraint here is expiring history.

Where PHI can accidentally persist in a pipeline and how to prevent it

The most likely breaks in a passthrough design occur outside its main delivery path. Error queues, dead-letter topics, connector logs, and retry buffers can become stores even when the normal path uses only memory. Debugging is a common entry point for persistence. A connector that prints field values can place plaintext PHI in a vendor-managed log aggregation system with its own retention policy. The engineering review therefore has to cover successful delivery, failure handling, and observability.

Dead-letter queues introduce a similar problem through undelivered events. An event routed there may remain indefinitely, converting an error-handling mechanism into a store of PHI. Durable retry buffers require the same scrutiny. In self-managed Kafka, events persist on disk until consumed, so a vendor-managed disk conflicts with the no-third-party-storage requirement. DDL change records, classed as schema events, also need the same controls when their context reveals PHI structure.

The practical audit starts by mapping every place where an event could rest for more than a moment. Each location must be classified as storage or transit, with its owner and safeguards made explicit. Encryption applies throughout, including at the endpoints. Successful delivery, retries, and dead-letter routing also belong in the transmission audit trail, without turning that trail into a payload archive. Any vendor-managed location holding PHI at rest requires BAA coverage, but coverage alone does not make it acceptable under a no-storage mandate.

What "exactly-once delivery" means in a PHI pipeline and why idempotency is the real guarantee

An unqualified promise of end-to-end exactly-once delivery across heterogeneous systems is not a sound basis for this architecture. The practical standard is at-least-once delivery from the log paired with an idempotent destination. Reprocessing is therefore an expected condition. The destination must apply an event again without creating another logical record. This separates the number of delivery attempts from the resulting state of the data.

MERGE operations keyed on primary keys provide the design pattern described here. Reprocessing an event produces the same destination result as processing it once. That protects more than data quality. A duplicate patient record can also create an audit-trail discrepancy, with repeated writes for one logical change inviting questions about integrity controls. Idempotent writes address the destination outcome, while the audit trail must still account for the transmission attempts.

Adding a clinical code column to a patient table can break a consumer that expects a fixed schema, forcing a restart and creating a gap in delivery. Automatic schema evolution reduces that interruption risk. AWS DMS does not maintain schema history or version tracking in the context described here, making schema-change handling a specific evaluation concern. A managed service must demonstrate both uninterrupted handling of schema changes and idempotent writes, because either capability alone leaves the other failure mode unresolved.

The BAA checklist before any vendor touches PHI in transit

The BAA must define what the replication vendor actually does with PHI. “Transit only” needs to cover the handling of logs, buffers, caches, and failed events alongside the main streaming path. The agreement precedes the first PHI transfer. Its scope should distinguish transient processing from any persistent handling so that the contractual description matches the system engineers have evaluated.

The HIPAA Breach Notification Rule requires notification without unreasonable delay and no later than 60 days after discovery. The BAA should state the vendor’s obligation and timeline explicitly. The 2025 Security Rule proposal, still awaiting finalization as of 2026, leaves that window unchanged and includes adjacent 24-hour and 72-hour operational triggers. Those timings should not be conflated.

A cloud provider or managed Kafka service that touches PHI cannot disappear from the agreement’s scope simply because another vendor operates the connector. The covered entity also needs on-demand access to tamper-proof transmission logs. Encryption terms should specify protection at rest and in transit, including AES-256 and TLS standards, while deletion and return terms should address any buffered or logged PHI when the contract ends or a breach occurs. A genuine passthrough architecture narrows this handling scope, but does not eliminate the need to explain the sub-processor chain.

Real-time PHI replication in practice — clinical dashboards, AI agents, and federated analytics

Clinical dashboards show the operational value of removing batch delay without adding a vendor-managed PHI store. A medication status or lab result that is hours old can become a care coordination liability when the dashboard depends on an overnight pipeline. Sub-minute replication from the EHR’s operational database closes that freshness gap. The intermediate infrastructure remains a delivery mechanism.

Agents trained on or querying stale PHI can produce unreliable outputs, but fresh data does not by itself make those outputs appropriate for clinical use. Governance remains an additional requirement. AI tools accessing PHI need BAA coverage, minimum necessary access, audit trails for queries, and human oversight of outputs influencing PHI-based decisions. Those controls must be designed into the data path before deployment.

Federated analytics extends the same requirement across multiple source databases. Population health and research workflows that bring together change events need streaming coordination to avoid the lag of batch ETL. Freshness alone is not the only benefit. Because the pipeline layer holds no persistent PHI, a security event there does not expose a vendor-managed archive of historical patient data. The architecture limits that layer’s exposure to data in transit while leaving the source and destination subject to their own controls.

How to evaluate managed CDC for a no-third-party PHI storage requirement

A managed replication vendor must show how its passthrough works, with technical proof to back up its claims. Logs, buffers, caches, and error queues all belong in that demonstration. To meet the mandate, vendor-managed handling must be transient and in memory, and any PHI kept in persistent storage must be encrypted and under the customer’s control. The normal path is insufficient evidence on its own. Failure paths show whether the vendor keeps its no-storage commitment during an outage, a malformed event, or a failed delivery.

The contractual evidence must be equally specific. A signed BAA needs a clear handling scope, an identified sub-processor chain, and a concrete answer about audit-log access. Technical evaluation then turns to automatic schema evolution and idempotent destination writes. The vendor should demonstrate how new columns, renamed fields, and type changes are handled without manual restarts, and how delivery that updates existing records or inserts new ones prevents duplicate downstream PHI records after replay. These criteria turn reliability claims into behaviors that a procurement team can compare.

“Show me where PHI sits in your infrastructure between leaving the source and arriving at the destination, and what the BAA covers for each of those locations.” A detailed answer about “managed CDC” identifies the actual locations and handling conditions. A vendor that cannot provide that account has not demonstrated the passthrough requirement.

Sources

  1. HIPAA Data Storage Requirements: Safeguards and Options
  2. HIPAA Compliant Cloud Storage: 2026 Rule Changes You Must Know - Medical ITG

More in Streaming Pipeline Operations