Automated OQ Sounds Simple. It Isn’t.

Operational qualification is the part of computer system validation where you prove, with documented evidence, that a system does what its specifications say. Traditionally that means a person working through a Word document step by step — click here, verify this, take a screenshot, sign and date. It is slow and expensive, and it has to be repeated every time the system changes meaningfully.

Automating it looks obvious. Automated scripts can click and check faster than any person, and they never skip a step out of boredom. DnXT maintains automated scripts mapped one to one to its formal OQ documents, run against a qualified environment, producing structured evidence on every run.

The hard part of doing this well is not speed. It is honesty. An automated suite can mislead you in both directions — reporting failures that are not real, and passes that prove nothing — and in a validation context both are expensive. This post describes the discipline we run, and the questions to ask of anyone who offers you “automated validation”.

One Script, One OQ Document

The first rule: every automated script corresponds to exactly one approved OQ document, and its steps mirror that document’s steps. No clever consolidation, no single script that “covers three test cases”.

The reason is auditability. When a QA reviewer or an auditor picks up an OQ document, they must be able to open the matching automation, read it top to bottom, and satisfy themselves that it performs the documented procedure. Once scripts and documents drift into a many-to-many relationship, that traceability dissolves — and with it the evidential value of the whole exercise. Automation that cannot be traced to the approved protocol is a useful engineering test, and worthless as validation.

When the Product Moves, the Scripts Must Follow

Software that is actively improved changes its screens: a button is renamed, a step is added, a workflow is reordered. Every one of those changes can make an automated script fail even though nothing is wrong with the product. Left alone, the suite fills with failures that are really out-of-date scripts.

That situation is dangerous in two ways:

  • Out-of-date failures erode trust. When red results are usually noise, people stop reading them, and one day a real defect scrolls past unnoticed.
  • The tempting “fix” is worse. Under schedule pressure, the easy move is to loosen the check — wait longer, accept vaguer text, verify less. Do that enough times and you have a suite that passes 100% while checking almost nothing. It looks like validation evidence. It is theatre.

The Honest-100% Rule

Every failure is triaged into one of two outcomes, with no third option:

  • The product changed legitimately. The script is updated to check the new, correct behaviour — and the matching OQ document is updated through change control, because the documented procedure changed too.
  • The product is wrong. A defect is raised, and the script stays red until the product is fixed. The failing evidence is the point.

What is never allowed: weakening a check so that a red result turns green. No broadened text matches, no removed verification steps, no “temporary” skips that quietly become permanent. A pass must mean the documented behaviour was verified.

We call the goal the honest 100%: a suite where every script passes because the product actually does what the OQ documents say — and where the route there is visible in the record as change-controlled document updates and closed defects. That history is itself evidence of a working quality system.

What an Evidence Run Looks Like

A qualification run executes against the qualified environment configuration, not a developer’s sandbox. Each script records step-level results, timestamps and screenshots at every verification point, assembled into an evidence package a reviewer can read beside the OQ document.

Because runs are inexpensive, qualification stops being a once-a-year event. The full suite can be re-run after any significant change, with fresh evidence attached to the change record. That reverses the usual trade-off in regulated software, where expensive requalification pushes teams into rare, large releases. Inexpensive requalification makes smaller, safer, more frequent releases possible — with current evidence every time. Our wider security and compliance approach depends on exactly this.

Where People Stay Accountable

Automation verifies specified behaviour; it does not exercise judgement. People still approve the protocol, decide how each failure is triaged, own the change-control record when procedures change, and review the evidence package before it is signed. Automation replaces the mechanical clicking and screenshotting — not the accountability. Every package still carries a human reviewer’s signature, backed by the platform’s audit trail.

Questions to Ask About Anyone’s “Automated Validation”

  • Is there a one-to-one mapping between automated scripts and approved OQ documents? Can you see it?
  • When a script fails, what is the documented triage path — and can they show you an example of each outcome?
  • What stops checks from being quietly weakened over time?
  • Was the evidence produced on a qualified environment configuration, or on a developer build?

A team that answers these crisply is doing validation. One that answers with a pass rate and no history is doing decoration.

The Takeaway

Automated OQ is worth it. But its value comes entirely from discipline: one-to-one traceability, two-outcome triage, and checks that are never watered down. A 100% pass rate is only meaningful if it was earned honestly.

If you want to see what audit-ready automated qualification evidence looks like — or you want a vendor whose validation story holds up under questioning — talk to a regulatory expert.