Software testing in healthcare: risks, regulation and method
Patient records, appointment booking, prescriptions, billing for care: in healthcare software, a defect does not just cost a support ticket. It can affect a patient's care or the confidentiality of their data.
- Identity is the first risk: a document attached to the wrong patient is a critical defect.
- Test data is never real data: fictitious data sets, generated and recreated for every run.
- Hosting is regulated: in France, health data hosted on behalf of a third party requires an HDS-certified host.
- Interoperability is tested on its own: exchanges between systems, formats and rejections.
Oussama Belakhdar, QA architect and founder of AutomationDataCamp, has taken part in testing projects for healthcare software. What follows covers the checks we run first on a patient record system, a booking tool or a medical billing application. Our clients are not named.
What do the digital health figures say?
Figures published by the organisations cited, not results of our own engagements.
Why is healthcare software harder to test than most?
- The same record is read and changed by very different profiles (front desk, physician, nurse, billing, administrator), and each profile has its own scope, sometimes limited to one care unit or one episode of care.
- Copying production is out of the question: health data is sensitive data under the GDPR. Even the acceptance environment needs credible fictitious patients: namesakes, temporary identities, newborns, deceased patients.
- The software sits among other systems (laboratory, imaging, pharmacy, billing, secure messaging), with asynchronous messages and format versions that vary from one hospital to the next.
- Some features have a clinical effect, such as a drug interaction alert or the display of a critical result: a silent regression is invisible on screen but can change a care decision.
Priority scenarios for healthcare software
- Patient identification: creating, searching, merging and unmerging records, namesakes, temporary then qualified identity, a duplicate created through two different entry points.
- Attachment: every report, lab result or image lands in the right record, even when the incoming message carries an old or incomplete identifier.
- Permissions: each profile sees only its scope, emergency "break-glass" access remains possible but is logged, and the access log can actually be used.
- Interoperability: valid message, malformed message, missing mandatory field, recipient unavailable, replay without duplicates, acknowledgement.
- Data life cycle: export at the patient's request, rectification, retention period, archiving.
- Degraded mode: behaviour when a third-party service goes down, then recovery without loss once the link is back.
HDS, GDPR, FHIR, IEC 62304: what regulation requires you to test
Health data hosting (HDS, France). Article L.1111-8 of the French Public Health Code requires certification for anyone hosting personal health data on behalf of a third party; the framework is published by the French Digital Health Agency (ANS). For the QA team this becomes concrete checks: environments holding real data are identified, backups can be restored, administrative access is personal and logged.
GDPR, Article 9. Data concerning health is a special category of personal data. Tests check data minimisation (a screen shows only what the profile needs), access logging and data subject rights: access, rectification, erasure where it applies.
Interoperability: HL7 FHIR and the French CI-SIS. FHIR, published by HL7 International, describes exchanges as resources (Patient, Observation, DocumentReference…) exposed through APIs. In France, the ANS interoperability framework for health information systems (CI-SIS) specifies the profiles to follow. Each exchange is tested as a contract: validation against the profile, error cases, version changes.
National health identity (INS, France). The ANS publishes the INS framework, which governs patient identification in health data exchanges. Identity statuses become test cases: temporary, retrieved, validated, qualified, and the transitions allowed between them.
Software as a medical device: Regulation (EU) 2017/745 and IEC 62304. Software that its manufacturer intends for a medical purpose (diagnosis, monitoring, support for treatment decisions) can be a medical device; Rule 11 of Annex VIII of the regulation then sets its class. IEC 62304 describes the medical device software life cycle and assigns software to safety classes A, B and C: the higher the class, the more detailed the required verification, up to unit testing and requirement-by-requirement traceability.
An example test case: merging two records for the same patient. In HL7 FHIR R4, a merge is expressed through Patient.link: the duplicate record carries a link of type replaced-by (“must no longer be used”) to the surviving record, which carries the reverse link, replaces. A typical test creates the same fictitious patient through two entry points (front desk and an incoming laboratory message), attaches a report to each record, then merges them. Expected results: the duplicate is marked inactive and points to the surviving record; a search on the old identifier no longer returns two active patients; both reports are reachable from the surviving record; the merge appears in the access log. The test is then replayed in reverse, because unmerging a record merged by mistake must return each document to its original patient. These are defects that no single screen reveals: a document attached to the wrong identity looks perfectly normal.
Realistic test data, with no real data at all
In healthcare, the quality of a test campaign often depends on its data set. We build fictitious patients that reproduce the hard cases: two patients born the same day with the same name, a patient who changes her married name, a newborn without a final first name, a patient transferred between two wards, a record merged by mistake and then unmerged.
These data sets are generated by script, versioned with the tests and recreated for every run: no test depends on the state left by the previous one, and no production export travels between teams. For load testing, volumes are synthetic: only the overall volume is modelled on reality.
How do we organise testing for healthcare software?
- Map clinical and confidentiality risks. Each feature is ranked by the impact of a defect on the patient or on medical confidentiality, then by its likelihood; identity and permissions come first.
- Test exchanges at API level. FHIR flows and messages between systems are checked with API tests, faster and more stable than the UI; Playwright covers clinician journeys end to end, k6 covers load, all wired into your CI.
- Trace requirement by requirement. When the software follows a medical device process, each requirement is linked to its tests and results, so the verification evidence is ready for audit.
- AI, with a human review. An assistant speeds up writing scenarios from the specifications. Every generated test is reviewed against our 7-question checklist. See our case study: from a PDF specification to Playwright tests.
Where should you start?
The most useful starting point is often a review of patient identification and permissions, where defects are the most serious.
A QA audit reviews your application: a written assessment, a test strategy adapted to your industry and a costed action plan, in 5 days, for €1,000 excl. VAT as a fixed fee. Delivered remotely, in English or French, for companies in the US, the UK, Canada, Australia, Europe, the Gulf and the Nordics. The first scoping session is free.
See the QA audit at €1,000 excl. VAT
Frequently asked questions about healthcare software testing
Can healthcare software be tested with real data?
No. Tests use fictitious data, generated for the purpose and recreated for every run. Production data has no place in a test environment.
Which tests should be automated first in healthcare software?
High-risk journeys that are repeated in every release: identity and document attachment, role-based access, and exchanges with third-party systems. API tests are often more stable and faster than UI tests for these topics.
Does healthcare software have to follow IEC 62304?
Only if it is a medical device or part of one. IEC 62304 describes the medical device software life cycle; purely administrative software (scheduling, billing) is generally outside its scope. Qualification is decided under Regulation (EU) 2017/745, not by the test team.
How do you test FHIR interoperability?
By treating each exchange as a contract: the resources sent are validated against the expected profile, then error cases are replayed (missing field, unknown code, different version, server unavailable). These tests run at API level in every release.
How much does a QA audit cost at AutomationDataCamp?
The QA audit and diagnosis costs €1,000 excl. VAT as a fixed fee, over 5 days: a written assessment, a test strategy and a costed action plan. The first scoping session is free.
Other industries: E-invoicing · Finance · Blockchain · Logistics · E-commerce · Document management. All industries.
Page signed by Oussama Belakhdar, written with the help of AI. Regulatory references link to the official sources above; they are not legal advice.
Related articles

AI & ML for test automation
What AI concretely changes in test automation.
Read more : AI and ML for test automation
Integrating Automation into CI/CD
Jenkins, GitLab CI, GitHub Actions — practical examples.
Read more : Integrating Automation into CI/CD