Accountability in use of clinical AI

Artificial intelligence (AI) is already reshaping Taiwan’s hospital wards. Squeezed by an aging population, nursing shortages and grueling paperwork, major medical centers are rapidly deploying algorithms. National Taiwan University Hospital uses large language models for medical records and International Classification of Diseases (ICD). Chang Gung Memorial Hospital integrates machine learning into disease prediction and triage.

Meanwhile, Taipei Veterans General Hospital has set up dedicated clinical review panels to audit algorithms throughout their lifecycle.

Most hospital AI tools in Taiwan still handle paperwork and triage, but that administrative line is already blurring. Systems are moving directly into diagnostic assistance and treatment pathways. Algorithms might not issue oncology diagnoses on their own, but they already prioritize which people gets urgent CT scans, flag suspicious lesions and suggest chemotherapy regimens.

When an algorithm errs, where does the liability fall? On the attending physician who trusted the screen? The hospital that procured the software? Or the vendor that wrote the black-box code? The danger in clinical AI is not only bad calculations; it is also the diffusion of responsibility.

Healthcare has paid dearly for overestimating software before. The Therac-25 radiation accidents in the 1980s killed and maimed people precisely because human oversight gave way to blind faith in automated code. The failure of IBM Watson for Oncology proved that sophisticated software collapses in real clinics when recommendations depart from clinical context and proven practice.

As legal academic Frank Pasquale wrote in New Laws of Robotics, AI must augment clinical judgement, not supplant it.

Pasquale’s core insight is straightforward: attribution must precede autonomy. In a hospital, an algorithm cannot be cross-examined and a providers cannot hide behind a proprietary model. Legal and moral responsibility must stay tethered to recognizable institutions and clinicians.

Aviation solved a similar dilemma years ago. When the US Federal Aviation Administration (FAA) introduced SMART — an automated tool for rerouting air traffic around severe weather — commercial airlines pushed back hard. Their concern was not whether the software worked, but who was on the hook if an automated route steered an airliner into a storm.

The FAA responded with a deliberate “start, crawl, walk, run” deployment: keeping human controllers firmly at the center of final routing choices until the operational logic was proven.

Clinical medicine needs the same posture.

The newly passed Artificial Intelligence Basic Act (人工智慧基本法) sets down broad principles: human oversight, transparency and accountability. However, broad statutory text does not tell an intensive care unit how to handle a conflicting drug alert.

Because fully autonomous clinical AI is not yet the standard of care in Taiwan, policymakers have a brief window to get the rules right.

Translating rules into clinical reality requires three concrete shifts:

First, tiered, risk-based oversight. Administrative tools for hospital bed tracking should not face the same regulatory friction as diagnostic algorithms that recommend surgery or radiation doses. Regulators must require rigorous clinical validation on local patient populations, transparent training datasets and an unobstructed right for clinicians to question and override recommendations.

Second, unbundled lifecycle liability. Responsibility must be divided by role, not blurred together. Developers must answer for model integrity and dataset bias. Hospitals must take ownership of local deployment and clinician training. Clinicians remain responsible for final treatment decisions within the software’s validated scope.

Third, post-market surveillance: Governance cannot stop at regulatory approval. Algorithms suffer from “data drift” as clinical protocols shift. Hospitals need multidisciplinary committees to track real-world errors, review patient outcomes and audit systems after vendor updates.

The transition toward healthcare AI is not an engineering race. It is a test of institutional credibility. The societies that thrive in the age of algorithmic medicine would not be those with the fastest processors, but those with the discipline to demand human accountability before automation outpaces oversight.

Hanson Wu is a healthcare policy professional with experience in pharmaceutical policy, health security and public affairs. He is a graduate healthcare policy researcher at National Taiwan University.