De-identifying Clinical Notes Without Destroying Their Value

Most de-identification programs have been built for structured records, which make up only 10-20% of health data. Health insurance claims, lab test results, and prescription dispensing information are typically composed of known, bounded fields. They can be de-identified using statistical models of risk, sometimes even making key decisions solely using the data dictionary. While this forms the basis of most de-identification programs, these approaches are not suitable for clinical notes, visit transcripts, or discharge summaries. This unstructured data sits in a separate pile, too complex to de-identify at scale but too risky to share without processing. As a result, organizations that have spent years building sophisticated privacy programs for structured data are often operating without a coherent strategy for the 80-90% of clinical information that lives in free text.

Recent advancements in named entity recognition and re-identification risk assessment have matured to the point where unstructured de-identification can be done at scale with confidence while preserving the analytical value that made the data worth having in the first place.

In this blog, we will unpack the challenges of unstructured data de-identification and how Datavant’s approach is built to address those challenges without sacrificing the data’s analytical value.  

Why Clinical Free Text Resists Simple Solutions

Structured data de-identification is a well-understood problem in the sense that the entities are known and bounded. A claims file, for example, has defined fields including date of service, diagnosis code, procedure code, and patient identifier. De-identification transforms those fields according to documented rules, and the output is predictable.

Clinical free text is different in every dimension. The same information can appear in dozens of surface forms. A patient’s name might appear as “Mr. Johnson,” “the patient,” “he,” or just “John,” depending on where in the note it falls. Geographic identifiers range from specific hospital names to street addresses to neighborhood references. Dates appear in narrative context (e.g., “three weeks after her procedure,” “since last January”) and de-identifying them correctly requires understanding the clinical sequence. 

The very complexity that makes unstructured data difficult to de-identify is what makes it so rich for analytics. A physician’s note from a cardiology visit contains the narrative of a clinical encounter through documentation of symptom progression, differential diagnoses considered, medication adjustments, and the reasoning behind them. That context, often not captured in structured data fields, makes clinical notes indispensable for pharmacovigilance, real-world evidence (RWE) generation, and (with the appropriate controls) AI/ML model development. Redact it carelessly and you’ve destroyed the asset you set out to protect.

The core technical challenge of de-identifying unstructured data is named entity recognition (NER). This consists of identifying every instance of Personally Identifiable Information (PII) and other potential identifiers in continuous prose across varying clinical writing styles, specialties, and documentation conventions. False negatives or missed entities create compliance exposure. False positives or innocuous information incorrectly treated as an identifier strip context and degrade utility. Organizations that approach this as a checkbox exercise tend to over-redact, replacing names, dates, locations, and any uncertain entity with a black box, resulting in notes that are technically de-identified but analytically useless.

Named entity recognition identifies potential identifiers in clinical free text.

Strong NER can provide accurate identifier detection, but it doesn’t by itself prove the data is de-identified. HIPAA’s Expert Determination standard, established under 45 CFR §164.514(b)(1), sets this bar: a qualified statistical or scientific expert must determine that the risk of identifying an individual is very small and document the methods supporting that conclusion. Applying Expert Determination to unstructured data requires not only NER performance at the entity level, but also a defensible statistical framework for evaluating residual re-identification risk across the entire document corpus, along with the intended use case and safeguards. This combination of entity-level accuracy plus contextual risk assessment ultimately separates a compliant de-identification program from a compliant-looking one.

Remediation Is Not Binary

The most important conceptual shift in unstructured de-identification is to stop treating remediation as a single action (i.e., redact and move on) and start treating it as a library of techniques. This is essential to producing high-utility data that truly has a low risk of re-identification. 

The most common remediations for unstructured data include the following:

Redaction removes identifying information from text. In its place, the name “Jane Smith” may be blacked out or replaced with “NAME”. This removes the identifier but can produce text that feels processed and inauthentic.

Obfuscation or pseudonymization replaces identifying information with new values that are of the same entity type. For example, “Jane Smith” may be replaced with “Alicia Carter”. This allows the data to look and feel like ‘real’ data but without sharing identifiers that trace back to the data subject.

Generalization or truncation reduces precision rather than removing information entirely. An exact date of birth can be generalized to birth year. A five-digit ZIP code can be truncated to a three-digit prefix. This reduces rather than removes the re-identification risk and can improve the utility of the data.

Date shifting moves all dates for a given patient by a consistent random offset. This preserves sequence and intervals while removing the calendar anchor that enables re-identification. For longitudinal research like tracking disease progression, evaluating medication adherence, or building RWE cohorts, the temporal relationship between events matters more than the specific calendar dates. The result supports time-series analysis without exposing when events actually occurred, effectively reducing the re-identification risk.

The right remediation depends on two things: the nature of the entity and the intended use of the data. A name in a clinical note rarely needs to be preserved for downstream research, so redaction or obfuscation is usually appropriate. Dates are different. If you remove them entirely, you lose the temporal structure that makes longitudinal analysis possible. Shift them consistently and you preserve the analytical signal while eliminating the identifying information.

Selecting the right remediation for each entity type and applying it consistently across a document corpus is where most automated solutions fall short. The decision requires understanding both the privacy implications and the analytical requirements of the downstream use case. 

Human Annotation and AI are Both Deployed to  Achieve and Demonstrate De-identification

Performance claims in unstructured de-identification tend to cluster around NER accuracy, reported as precision, recall, and F1 scores on curated benchmark datasets. These metrics matter, but they aren’t sufficient to show whether a de-identification program is defensible in production. Benchmark datasets don’t show how performance holds up on real clinical data (vs curated datasets), and they don’t show how that performance translates into re-identification risk.

AI-based NER systems perform well on common entity types in typical clinical documentation and benchmark training datasets. They perform less reliably on rare entity types, non-standard writing conventions, specialized clinical language, and on text that doesn’t look like curated training data. This means that AI-based NER systems that tout high performance on benchmark data may not have the same performance on your sensitive clinical notes.

This is where a qualified team of human annotators is enabled to validate a statistically meaningful sample of notes, something automated systems cannot do alone. Datavant’s team of privacy analysts doesn’t replace the AI processing, since the volume of clinical notes in any meaningful dataset makes human-only review impractical. Instead, they act as the quality layer on top of it. This annotation allows us to measure the performance of our de-identification process on your data specifically. Systematic errors in detection can be caught early and corrected in the output, and the findings are used to improve the performance of the AI algorithm. This human-in-the-loop approach provides the scale of the AI pipeline alongside the accuracy and confidence of human review.

This matters directly for HIPAA Expert Determination. The §164.514(b)(1) standard requires a qualified expert to document that re-identification risk is very small. Human review lets us produce performance metrics specific to your dataset, which is what makes that determination credible. Our privacy team then combines those performance metrics with the safeguards governing how the data will be shared to estimate the residual risk. This risk estimate is then used to show the data has been de-identified and produce documentation that can withstand scrutiny. Our pipeline allows for scalable processing and defensible de-identification. The human annotation makes the compliance position defensible.

Structured and Unstructured De-identification Must Be Unified

Many use cases require linking de-identified structured and unstructured data together, which is more common than organizations realize. Doing so requires a unified approach across both data types. Suppose your structured data de-identification generalizes dates of birth to year of birth while your unstructured data applies date shifting.The same patient now carries two conflicting dates of birth, with no way to determine which is correct. To retain utility in linked structured and unstructured data, de-identification cannot be applied in isolation.

A unified program maintains consistency across structured and unstructured records, applying the same patient pseudonym, the same date offset, and the same geographic generalization rules across every data element regardless of where it appears. When a released dataset combines claims with clinical notes, every document in that release must be treated as part of the same privacy surface.

This has direct implications for how de-identification infrastructure should be evaluated. Point solutions that handle only structured or only unstructured data create gaps that can lead to reduced data utility or missed privacy risks. A program that manages both, with a shared patient identifier scheme and coordinated remediation logic, eliminates the gap in which re-identification risk goes unnoticed and data utility is compromised.

What a CDO Should Ask When Evaluating Unstructured De-identification Strategies

For data strategy leaders building or sourcing an unstructured de-identification program, benchmark accuracy is only the starting point. Questions that speak to a program’s defensibility matter just as much.

Does the program produce data that is de-identified under HIPAA? The only ways to de-identify HIPAA data are via Safe Harbor or Expert Determination. De-identification with Safe Harbor requires the removal of all instances of 18 kinds of identifiers and of the individual’s relatives, employers, and household members, and the covered entity must have no actual knowledge that the remaining information could be used to identify the individual. There is no wiggle room in the law for removal of, say, only 98% of identifiers; they must all be removed. For any dataset that is too large for manual removal of every identifier, Expert Determination is the pathway that accommodates less-than-perfect detection while still meeting a defined privacy standard. That requires more than documenting which entities were removed. It requires a statistical determination, signed by a qualified expert, that re-identification risk meets the §164.514(b)(1) standard. This is what separates an interesting AI proof-of-concept project from a scalable program with a defensible compliance position.

Is remediation configurable by use case? The same clinical note corpus may need to be de-identified differently for a pharmacovigilance application versus an AI training dataset versus a patient registry. A program applying one fixed strategy to all outputs will either over-protect some use cases or under-protect others.

How is performance measured, and by whom? Precision and recall on NER are necessary but not sufficient. These need to be translated to specific privacy metrics. For instance, can the program characterize, for a given corpus, what residual re-identification risk remains? And does that characterization involve qualified human review, or is it purely algorithmic?

Does the program integrate with your structured data de-identification? Consistent surrogate data values, date shifting, and geographic treatment across record types ensures that a released dataset is coherent, not a collection of independently processed components.

What does the output look like in practice? De-identified notes should be usable by researchers and data scientists without post-processing. If the output requires significant cleanup to restore analytical utility, the remediation decisions upstream were wrong.

The Future of Clinical Note De-identification Is Now

In most health data assets, clinical notes have historically been treated as too difficult to share because tools could not de-identify them at scale with sufficient confidence. That constraint is no longer the limiting factor.

Organizations that operationalize unstructured de-identification now, using a methodology that supports Expert Determination, a remediation library tuned to use case requirements, and integration across structured and unstructured data types, gain access to a signal their competitors will not have as long as those competitors keep treating free text as unshareable. For pharmaceutical companies building pharmacovigilance programs, RWE cohorts, or AI development pipelines, clinical notes are the difference between a dataset that can answer clinical questions and one that cannot.

To learn more about Datavant’s unstructured de-identification capabilities and see a demo, contact us.

By: David Copeland, PhD, Staff Data Scientist on the Privacy Product R&D team, Datavant