Data Governance and AI Security Go Hand in Hand for Healthcare Organizations

The Risks of Shadow Data and Shadow AI for Healthcare

Crume warns that the ease of spinning up AI models using cloud resources has introduced serious risks.

Pilots or experiments often involve copying organizational data into unsecured environments, where sensitive patient details may not be protected by the same safeguards applied to production systems.

Discovering and eliminating shadow data and shadow AI instances must be a top priority for healthcare IT leaders.

One solution is implementing an AI firewall, a control layer that monitors the flow of information into and out of AI systems.

By scanning data for sensitive information such as patient identifiers, health records or financial details, an AI firewall can block dangerous outputs before they leave the organization.

“If we start seeing sensitive information come out as responses from our AI, we immediately redact that information or block it entirely,” Crume says.

Click the banner below to read the new CDW Artificial Intelligence Research Report.

New Attack Surfaces for AI

Protecting data is only one part of the equation. AI models themselves represent a new attack surface. IBM’s framework emphasizes three pillars: secure the data, secure the model and secure the usage.

The novelty of AI models means new categories of attacks are emerging. For example, Crume describes prompt injection as the AI analog to social engineering.

By carefully crafting malicious inputs, attackers can bypass guardrails and make AI systems behave in unintended ways, such as leaking private data.

Models can also be directly tampered with through model infection or data poisoning, in which corrupted inputs compromise results.

These threats shift attention to an area often neglected in traditional cybersecurity: integrity.

“Most of our energy historically has been on confidentiality and availability, and we’ve done very little on integrity,” Crume says. “AI-based attacks tend to lean toward integrity-based attacks.”

He explains that prompt injections, model infections and data poisoning can all be integrity-based attacks. For healthcare, these attacks are especially dangerous, as incorrect outputs could compromise diagnoses, disrupt workflows or even endanger patient lives.

RELATED: Minimum viable data governance enables smarter healthcare.

Balancing Security, Workflow and Governance for Healthcare AI

The challenge for healthcare organizations is weaving these protections into daily operations without slowing down clinicians.

“The best security is the kind that you don’t see, but that’s still effective,” Crume says. “Security should not be an inhibitor but an enabler if done correctly.”

Much like traditional firewalls that screen out malicious packets, AI firewalls can quietly block prompt injections or inappropriate queries while allowing legitimate traffic through.

Although governance and security are often treated as separate concerns, Crume notes that they are increasingly intertwined.

“I think about this as a Venn diagram,” he explains. “You’ve got a governance circle and an AI security circle, and they overlap. You can’t do either in isolation.”

In healthcare, governance often falls under the chief risk officer, focusing on fairness, bias and compliance, while security is overseen by the CISO, concerned with resilience against attack.

Both groups care about reliability and privacy, and both need to collaborate to ensure patient data remains protected.

“There is no privacy without security,” Crume says. “If your system is not secure, you can’t ensure the data is only going to the right people.”

READ MORE: Why is governance key to securing AI?

Building Trust for the Future

Generative AI and AI agents are raising the stakes further. Unlike traditional models, AI agents can operate autonomously, carrying out complex tasks on their own. This power comes with risks.

“Agents are an area that offer us tremendous promise and tremendous risk at the same time,” Crume says. “When they’re exploited, they become great risk amplifiers. What would take a person a day to do, an AI agent can do in a matter of minutes.”

He says that means if it’s doing the wrong thing, it will do the wrong thing fast.

For healthcare organizations, the road to secure AI adoption requires vigilance across data, models and workflows.

Governance and security must work hand in hand, while stakeholders across IT, security and risk management share responsibility for protecting patients.

“We’re all still learning this as we’re doing it, and learning a lot of lessons, sometimes the hard way,” Crume says. “If we get the right tools in place, AI can make healthcare workers’ lives easier, more efficient and more secure.”

Brought to you by: