SOC 2 and HIPAA Compliance Requirements for Real-Time Data Pipelines

A clean SOC 2 Type II report tells an auditor that a company's infrastructure is secure. HIPAA adds a separate set of obligations that SOC 2 does not formally require: rules specific to PHI handling, the Business Associate Agreement chain, the minimum-necessary principle, and breach notification to HHS and to affected individuals. A team can hold a spotless SOC 2 report and still send unmasked diagnosis codes or date-of-birth fields into a shared analytics environment, and that is a direct HIPAA violation regardless of what the SOC 2 audit found. The two frameworks overlap a great deal, but the uncovered portion is where HIPAA's PHI-specific rules sit, and that is precisely where CDC pipelines run into trouble. HIPAA also reaches well past hospitals and clinics: any software vendor, cloud provider, or research institution that touches PHI falls under it. This is an engineering problem for a much wider set of companies than the phrase "healthcare compliance" usually suggests. The regulatory floor is rising too. On January 6, 2025, HHS published a notice of proposed rulemaking, "HIPAA Security Rule to Strengthen the Cybersecurity of Electronic Protected Health Information," that would remove the distinction between "required" and "addressable" safeguards and make nearly all of them mandatory. What follows maps each requirement, from both frameworks, to the specific pipeline decisions that satisfy or break it.

CDC internals and compliance exposure at the moment of capture

Every insert, update, and delete in a source database that holds PHI turns into a compliance event the instant change data capture streams it downstream. Understanding how that capture works is the starting point for knowing where controls actually need to live. Log-based CDC reads from a database's internal change log, in PostgreSQL through the Write-Ahead Log accessed by logical decoding, and it turns each row-level change into an event the target system can consume. PostgreSQL's logical replication works at the table level and captures every change type, including TRUNCATE as of PostgreSQL 11, while placing minimal load on the source database. What it does not capture is DDL. Statements like CREATE TABLE, ALTER COLUMN, and DROP never appear in the logical replication stream itself; catching them is a job for the CDC tooling layer, not the database. When that happens, the result is not a configuration bug to patch later. An undetected schema change that routes a new PHI field to an unencrypted destination is an immediate HIPAA violation the moment it occurs.

A second exposure comes from delivery guarantees. That gap is why exactly-once semantics exist: they guarantee each change reaches its destination a single time even across crashes and retries. In a PHI pipeline, if a duplicate unencrypted event lands in an unauthorized destination, that counts as an unauthorized disclosure under HIPAA. Whether that duplicate causes harm also depends on how the destination handles it: idempotent processing, such as setting a status field to "shipped," absorbs a repeated event safely, while non-idempotent processing, such as incrementing a count by one, does not. That distinction is a compliance question as much as a correctness one.

The BAA chain and vendor requirements

HIPAA requires a covered entity to sign a Business Associate Agreement with every Business Associate: any entity that creates, receives, maintains, or transmits PHI on its behalf. A CDC pipeline running on AWS infrastructure without a signed BAA covering every service it touches, object storage, the queue, the compute layer, the replication tool itself, has a broken BAA chain regardless of how well the rest of the architecture is built. That chain extends to the replication vendor directly: a managed CDC service that buffers change events, even temporarily, is itself a Business Associate and needs a signed BAA before any PHI passes through it.

This is where the passthrough principle matters architecturally. Artie's architecture is built as a passthrough: change events move through Artie's infrastructure with only transient buffering in Kafka before delivery to the destination, which keeps the PHI exposure window in the replication layer small and limits the BAA surface to the source and destination systems the customer already controls.

Region selection carries the same weight as the vendor relationship itself. A BAA has to be in place with a vendor in the specific region where PHI is processed, and running compute in a region not covered by that agreement breaks compliance even when the same vendor has signed BAAs elsewhere. For a real-time pipeline, that means the replication compute, any intermediate queue, and the destination warehouse all need to sit inside a region where the BAAs in place actually apply.

Encryption requirements from source to destination

Encrypting PHI at rest and in transit is baseline under both frameworks, but a CDC pipeline creates more at-rest surfaces than most teams count. If the 2025 rulemaking proposal is finalized, that "addressable" flexibility disappears along with the rest.

A traditional ETL pipeline multiplies PHI copies across several points: the source database, a staging layer, an intermediate queue or object store, the warehouse, and often a cache on top of that. For CDC pipelines specifically, the intermediate transport, a Kafka topic, an S3 landing zone, a replication buffer sitting between source and destination, is most often left unaudited. AWS DMS illustrates the pattern well: it typically writes raw change files to S3 before a separate pipeline runs MERGE operations against them, and each hop in that chain is its own PHI-at-rest surface that needs independent encryption and audit coverage.

Key management sits apart from encryption as its own control and needs to be treated that way. Encrypting data does nothing if the keys protecting it are reachable by the wrong people or sitting next to the data they're meant to guard. For a database like DynamoDB, deciding which IAM roles can reach the KMS keys is a separate architectural call from deciding whether the data gets encrypted in the first place, and both decisions need to be made deliberately.

Access controls and the minimum-necessary principle applied to replication

HIPAA's minimum-necessary principle says PHI access has to be limited to what a given purpose actually requires, and in a CDC pipeline that turns directly into decisions about table selection, column filtering, and role-based access at the destination. Replicating a full schema "for convenience" is not a compliant choice once that schema contains PHI; the minimum-necessary principle calls for restricting replication to only the tables and columns the stated downstream use case actually requires.

Column-level filtering has to exist inside the CDC tooling itself, not just at the destination. Multi-factor authentication is required for any human accessing systems that touch PHI, and that includes the replication management plane just as much as the source database itself.

De-identification applied inside the pipeline, rather than left to destination-side controls, is the stronger design for PHI headed into analytical systems. HIPAA's Safe Harbor method requires stripping all 18 identifiers listed in 45 CFR 164.514(b)(2), but most teams underestimate the list, so fields like zip codes or admission dates stay in place and keep the data classified as PHI. Synthetic data is a legally sound substitute for de-identification in analytical and machine learning workloads, though differential privacy applied to real patient data needs additional review whenever human subjects are involved downstream.

Audit logging requirements and pipeline architecture

HIPAA's audit controls requirement means every access to electronic PHI anywhere in the pipeline, not only at the source, needs to produce a log entry that is immutable and precisely timestamped. Whether a pipeline can meet that bar at all comes down to how it's built. The regulation itself calls for audit controls that record and examine activity across every system touching ePHI; NIST SP 800-66 recommends immutable logs, precise timestamps, and unique identifiers as implementation practice, which in practice requires append-only storage that prevents alteration and sequence numbers that can reveal missing events. SOC 2's monitoring requirement goes further than storage: logs are expected to be collected and reviewed actively to catch suspicious activity, not archived and forgotten.

A pipeline that routes PHI through an intermediate Kafka topic or S3 prefix without logging access to that layer has a real gap: the source and the destination might be fully logged while the transit layer in between stays invisible to an auditor. Access logging needs to capture who accessed what and when at every data boundary, down to the kubelet and API server layers if the pipeline runs on Kubernetes. Every log entry needs a precise timestamp, a unique identifier for whoever or whatever accessed the data, and enough context to reconstruct what was accessed and why, and without all three, the log doesn't count as audit-compliant.

Schema evolution events belong in that same audit log. A DDL change that adds a PHI column to a source table, if it goes undetected and unlogged, leaves a hole in the record: ePHI was moving through the pipeline with no trace of when that exposure began. Artie's schema evolution handling, which detects and propagates schema changes automatically, produces a traceable record of exactly when a new column entered the stream, which is what the audit requirement actually asks for: accountability from the moment PHI exposure begins, not after the fact.

Schema evolution as a live compliance control, not a data quality concern

Schema changes in a production database carrying PHI are compliance events, not engineering housekeeping. If a CDC tool cannot detect, log, and respond to DDL changes as they happen, the pipeline is in violation every time the source schema changes. Since DDL statements never appear in PostgreSQL's logical replication stream, the tooling layer has to catch them some other way, typically by polling the information schema or intercepting DDL events directly, and it has to act before downstream consumers ever see the new columns.

There are only two architecturally sound responses to a schema change touching PHI: propagate it to the destination with a logged audit entry, or halt the pipeline and alert the people who run it. Every minute of manual response time is a minute of compliance exposure. Automatic schema evolution belongs on the list of requirements a buyer checks before choosing a CDC tool for any pipeline carrying PHI, a control with the same standing as encryption or access management.

AI and agentic workloads and the compliance stakes for real-time pipelines

Feeding live CDC output into AI agents or machine learning models creates compliance obligations that most teams have not fully worked through, because every PHI record reaching a training run or an inference call is a PHI exposure event that needs to be logged and controlled like any other. Consumer tiers of major LLM products, including OpenAI's ChatGPT (Free, Plus, Team) and Anthropic's Claude (Free, Pro, Max, Team), explicitly prohibit sending PHI without a signed BAA, though both companies offer a BAA path through their APIs and eligible enterprise tiers that can make PHI use permissible under contract. Where generative AI is genuinely required in a HIPAA pipeline, the architecture needs a private LLM, deployed on private cloud or on-premises infrastructure, operating under HIPAA-compliant terms. There is no BAA arrangement that makes public API use of PHI acceptable outside that structure.

AI agents built on candidate profiles, patient records, or inventory states depend on sub-minute CDC latency to function at all, and stale data here is not only a reliability issue. Model inputs and outputs containing PHI cannot be logged for debugging unless the logging system itself is encrypted and HIPAA-compliant, a constraint that has to be designed in from the start rather than bolted on after launch. As of 2025, HIPAA-covered entities also need to account for DOJ and CISA rules restricting the transfer of bulk U.S. sensitive personal data, including de-identified or encrypted health data, to certain foreign countries and entities, which adds a data residency layer on top of everything else for any AI deployment operating internationally.

Architecture of a compliant real-time CDC pipeline, end to end

A pipeline that satisfies SOC 2 and HIPAA at the same time builds passthrough design, exactly-once delivery, schema-aware filtering, immutable audit logging, and BAA coverage into the first design decision. At the transit layer, you need a passthrough design that never persists PHI on vendor infrastructure, which removes the at-rest encryption and BAA burden that persistence would otherwise create, paired with TLS 1.2 or better in transit and no unencrypted intermediate surfaces sitting between source and destination. At the delivery layer, it means exactly-once semantics to keep duplicate PHI events from reaching the destination in the first place, backed by idempotent write patterns that handle any edge cases where at-least-once delivery slips through anyway.

Access control follows the same logic: role-based access control at both the replication management plane and the destination warehouse, multi-factor authentication for every human touchpoint, and column-level access grants at the destination scoped to the minimum necessary for each role. The BAA layer makes all of that enforceable: signed agreements with the replication vendor, the destination warehouse, and every cloud service sitting in the data path, with compute deployed only in regions where those agreements actually apply.

Artie is built to satisfy this architecture as a managed service: a passthrough design that stores no PHI on Artie's own infrastructure, exactly-once delivery, automatic schema evolution with audit trails attached, SOC 2 Type II compliance, and BAA availability, deployable in under an hour. Either path is workable, but every month spent building toward this architecture is a month the pipeline runs without the full set of controls HIPAA and SOC 2 both expect, and that is a compliance cost as much as an engineering one.