NHS Software Engineering Quality: A Practical Guide for Digital Health Teams

Engineering high-quality software for the NHS requires more than selecting a modern technology stack, introducing automated tests or putting a continuous integration pipeline in place. Quality has to be designed into the way a digital service is built, changed, deployed and operated.

For digital health teams, the practical challenge is to turn broad expectations around security, reliability, maintainability, accessibility, interoperability and clinical safety into engineering controls that work every day. Those controls need to operate at the speed of delivery rather than becoming a separate assurance exercise immediately before a release.

The NHS Software Engineering Quality Framework provides a useful starting point for this approach. Its most important practical idea is that quality should be built into engineering rather than inspected into software after development. Teams can apply that principle by defining the characteristics their service must exhibit, translating them into measurable controls, automating those controls wherever appropriate and continuously using production evidence to improve the system.

The result is not simply better code. It is a delivery environment in which teams can make changes with a clearer understanding of risk, detect problems earlier and operate services with greater confidence.

1. Define what engineering quality means for the service

A service can contain well-written code and still be difficult to operate. It can have comprehensive automated tests and still fail under realistic load. It can meet its functional requirements while introducing security vulnerabilities, accessibility barriers or unsafe clinical workflows.

Teams therefore need to define engineering quality across several dimensions before deciding which tools or controls to introduce.

A practical quality model for an NHS digital service will normally consider functionality, maintainability, security, resilience, performance, operability, accessibility, interoperability and, where applicable, clinical safety. The relative importance of each characteristic will depend on the service.

A public information service and an application involved in a clinical workflow should not have identical assurance profiles. Similarly, an API handling a small number of administrative transactions has a different risk profile from an integration sitting in the path of thousands of clinical messages.

Start by documenting the quality attributes that matter to the service and expressing them in measurable terms.

Instead of stating that the platform must be “reliable”, define the service behaviour expected during component failures. Instead of saying that an API must be “fast”, define acceptable response times at representative and peak transaction volumes. Instead of saying that deployment must be “safe”, define the checks that must pass, how deployment is observed and how a failed change can be reversed.

For each important quality attribute, establish four things: the expected behaviour, the evidence used to demonstrate it, the threshold at which the expectation is considered met and the action taken when it is not.

A simple working model might include:

• Functionality: unit, integration, contract and end-to-end tests demonstrate that the service behaves as expected.
• Security: code, dependencies, infrastructure and deployment artefacts are continuously checked for known weaknesses and configuration errors.
• Maintainability: changes can be understood, tested and deployed without disproportionately increasing risk or delivery time.
• Resilience: expected infrastructure, network and dependency failures are tested and handled predictably.
• Performance: response times, throughput and resource consumption remain within agreed limits under representative load.
• Accessibility: relevant interfaces are continuously tested and periodically reviewed using appropriate manual techniques.
• Interoperability: interfaces conform to their contracts and remain compatible with known consumers and dependencies.
• Operability: logs, metrics, traces, dashboards and alerts provide enough information for the team to understand the behaviour of the service.
• Clinical safety: identified technical and workflow hazards are connected to engineering controls, tests, operational procedures and evidence where relevant.

This exercise should happen at service level rather than organisation level alone. Organisation-wide engineering standards are useful, but quality controls become meaningful only when a team understands how they apply to a particular system.

Risk should also determine the depth of the controls. Applying every possible test to every component creates slow pipelines and noisy assurance. Applying only a generic minimum creates blind spots. Teams should instead ask what can realistically fail, what the consequences would be, how the failure could be detected and which engineering control provides the earliest useful feedback.

The output should be a living quality profile rather than a one-off document. Review it when the architecture changes, when new integrations are introduced, when usage increases, when incidents reveal previously unknown risks and when the service moves into a different stage of its lifecycle.

2. Turn quality expectations into automated engineering controls

Once quality attributes have been defined, the next step is to move them into the engineering workflow.

A useful principle is that a quality requirement should become an executable control wherever it reasonably can.

If secrets must not be committed to source control, scan for them automatically. If an API contract must remain compatible, execute contract tests on every relevant change. If high-severity dependency vulnerabilities are unacceptable, make that condition part of the build. If infrastructure must satisfy security policies, validate infrastructure as code before deployment.

The objective is to shorten the distance between introducing a problem and receiving feedback about it.

A developer discovering an issue locally or minutes after opening a pull request is operating in a very different delivery environment from a team discovering the same issue during a pre-production assurance exercise several weeks later.

A mature pipeline therefore consists of layers of increasingly expensive assurance.

Fast checks should run early. Formatting, linting, static analysis, unit tests, secret detection and basic security checks can normally provide feedback quickly. Integration and contract tests can follow once an artefact has been built. More expensive performance, resilience, security and end-to-end testing can run at appropriate points later in the pipeline or on scheduled environments.

The important distinction is between running a check and enforcing a control.

A dashboard showing a vulnerability does not prevent vulnerable software from progressing. A code-quality report nobody reviews does not protect maintainability. A test suite that is routinely ignored because it produces false positives is not an effective quality gate.

For each automated check, teams should therefore define the conditions under which the pipeline passes or fails. Exceptions should be explicit, time-limited and owned rather than silently accepted.

Consider dependency vulnerabilities. A useful implementation does more than produce a list of CVEs. It identifies the affected direct or transitive dependency, determines the severity according to an agreed policy, fails delivery when the policy threshold is exceeded and records any accepted exception with an owner, rationale and review date.

The same pattern can be applied to code-quality findings, test failures, accessibility defects and infrastructure-policy violations.

Teams should also pay attention to pipeline design itself. A technically comprehensive pipeline that takes several hours to provide basic feedback will encourage developers to work around it. Tests should be positioned according to their cost and value, with the quickest and most deterministic controls running first.

Flaky tests deserve particular attention. A test that sometimes fails without a product defect steadily destroys confidence in the entire quality system. Teams begin rerunning failed pipelines automatically, genuine failures become easier to dismiss and delivery slows down. Treat recurring test instability as engineering debt and give it the same ownership expected for production defects.

Code review should operate alongside automation rather than duplicating it. Machines are better suited to repeatedly enforcing formatting, known vulnerability rules, dependency policies and deterministic tests. Human review is better used for design quality, unintended behaviour, domain understanding, security reasoning, maintainability and whether the change solves the intended problem cleanly.

The repository should then become a significant part of the service’s engineering evidence. Application code, tests, infrastructure definitions, API specifications, deployment configuration, observability configuration and important technical decisions should be version-controlled wherever practical.

That creates traceability between a change and the evidence used to approve and operate it.

3. Engineer testing, interoperability and clinical safety into delivery

Testing NHS software should begin with system design rather than with a test phase.

Systems that are difficult to test usually become expensive to change. If components are tightly coupled, interfaces are ambiguous, production-like data is difficult to generate or environments behave differently, teams spend increasing amounts of time constructing test conditions instead of evaluating behaviour.

Design for testability from the beginning.

Keep component boundaries understandable. Make dependencies explicit. Provide mechanisms for substituting external services during testing. Build repeatable test-data generation. Keep test execution consistent between developer machines and CI environments. Where possible, ensure the same artefact progresses between environments instead of being rebuilt differently for each stage.

Use different forms of testing to answer different questions.

Unit tests should provide rapid feedback about local behaviour. Integration tests should demonstrate that components and important infrastructure dependencies work together. Contract tests should protect interfaces between independently changing systems. End-to-end tests should exercise important user or business journeys without becoming the primary mechanism for testing every permutation.

Performance tests should establish how the system behaves under realistic and exceptional load. Resilience tests should exercise controlled failure. Security testing should combine automated analysis with techniques capable of identifying issues that static rules cannot detect.

Testing should continue after deployment.

Health checks, synthetic journeys, application metrics, logs, traces and user-behaviour telemetry can identify conditions that test environments cannot reproduce. Feature flags, progressive delivery and controlled rollout strategies can reduce the number of users exposed to an unexpected behaviour while still generating production evidence.

For NHS integrations, teams need to extend this approach beyond the software they directly control.

An interface can be syntactically valid and still fail operationally. Healthcare integrations encounter partial data, delayed responses, inconsistent implementations, duplicate messages, unexpected terminology, network interruptions and downstream systems that change independently.

FHIR conformance testing, for example, should not become the entirety of FHIR integration testing. Teams also need to examine business rules, profiles and extensions, terminology, pagination, search behaviour, authentication, error handling, retries, idempotency and the assumptions made by consuming systems.

The same principle applies to other NHS APIs and healthcare messaging standards. Test the contract, but also test the behaviour around the contract.

Where external dependencies are critical, explicitly model their failure modes. Determine what happens if a request times out, if authentication cannot be completed, if an upstream service becomes unavailable or if a downstream system acknowledges a message but fails to process it correctly.

Make ownership of failed transactions visible. A resilient integration should make it possible to determine what happened to a transaction without reconstructing events manually across multiple disconnected systems.

Clinical safety should be incorporated into this delivery model rather than operating as an isolated documentation stream.

When the service is within scope of clinical risk management, engineers, clinicians, product specialists and clinical safety professionals should collaborate early enough for hazards to influence architecture and implementation.

A hazard should be capable of producing concrete engineering consequences.

If incorrect matching of patient records could create a hazardous situation, the mitigation may include identity checks, constraints, user-interface behaviour, automated tests, operational monitoring and exception handling. Those controls should be traceable to the hazard rather than existing independently in separate documents.

Similarly, when an integration can deliver incomplete, delayed or duplicated clinical information, those behaviours should appear explicitly in both the hazard analysis and the technical test strategy.

This creates an important connection between engineering evidence and clinical safety evidence. The clinical safety case describes how risks are managed, while the engineering environment produces repeatable evidence that many of the associated controls continue to operate as intended.

Teams should treat changes in clinical workflow, integration behaviour or system boundaries as triggers for reviewing this relationship. A small code change can alter a risk control; a new dependency can introduce a new failure mode; a change in how information is displayed can affect how a clinician interprets it.

The delivery process should make those relationships visible before release.

4. Build production reliability and operability into the service

The transition into production should not mark the end of engineering quality assurance. It provides access to a new set of evidence.

Before operating a service, teams should define what healthy behaviour looks like. That includes technical measures such as availability, latency, error rate, throughput and resource utilisation, but it should also include signals closer to the service outcome.

An API returning HTTP 200 responses may technically appear healthy while users are receiving incomplete information. A message-processing service may remain online while its queue steadily increases. A clinical workflow can continue accepting transactions while a downstream dependency prevents those transactions from completing.

Monitoring should therefore start with service behaviour and work backwards towards infrastructure.

For each critical journey, determine which signals demonstrate successful operation, which signals indicate degradation and which conditions require human intervention.

Alerting should then focus on actionable conditions. Large quantities of low-value alerts create operational noise and eventually train teams to ignore notifications. Alerts should identify a condition that matters, provide enough context for someone to investigate it and connect to an appropriate response procedure.

Logs should contain sufficient contextual information to reconstruct important events while respecting information-governance and security constraints. Distributed services should use correlation identifiers or equivalent mechanisms so a request can be followed across component boundaries.

Metrics should expose trends rather than only failures. Increasing latency, queue depth, retry volume or dependency errors can indicate a developing problem long before a complete outage occurs.

Distributed tracing becomes valuable where a transaction crosses several components or external dependencies and the point at which time is being spent would otherwise be difficult to determine.

Operational documentation should be built from real service scenarios.

Runbooks should cover conditions the team is actually likely to encounter: failed deployments, unavailable dependencies, certificate expiry, queue backlogs, data-processing failures, capacity constraints and security incidents. Validate the important procedures periodically rather than discovering during an incident that documentation has become obsolete.

Resilience should also be tested deliberately.

Teams can simulate loss of application instances, unavailable dependencies, network latency, database failover or sudden increases in traffic. The objective is not to break systems indiscriminately. It is to establish whether the system behaves as designed, whether monitoring detects the condition and whether the team can recover it using known procedures.

Release engineering should support the same objective.

Automate deployments and make them repeatable. Minimise differences between environments. Avoid manual production configuration that cannot be reconstructed. Use immutable or declaratively managed infrastructure where appropriate. Make deployment state visible.

Prepare a recovery strategy before a release rather than during a failed one.

The appropriate mechanism might be rollback, roll-forward, disabling a feature, shifting traffic or restoring data. The mechanism is less important than ensuring the team has considered the failure mode and can execute the recovery reliably.

Measure releases in production as well. Observe application behaviour immediately after changes, compare key indicators with previous baselines and define conditions that should stop or reverse a deployment.

That creates a closed engineering loop:

design → build → test → deploy → observe → learn → improve.

Production stops being a destination and becomes part of the quality system.

5. Create a continuous engineering quality improvement cycle

Engineering quality becomes sustainable when teams can see their current condition and systematically improve it.

Begin with a periodic engineering quality review. The objective is not to award the team a maturity score or compare one delivery team with another. Use the review to expose differences in understanding and identify the constraints that are making safe delivery harder than it needs to be.

Review architecture, code health, testing, security, delivery, interoperability, operational readiness, observability, team capability and technical debt. Ask engineers to provide evidence rather than relying entirely on perception.

That evidence can come from repositories, delivery pipelines, vulnerability scanners, monitoring platforms, incident records, architecture decisions and engineering metrics.

A useful engineering dashboard should combine several categories of evidence rather than collapsing quality into one number. Track delivery behaviour, reliability, security posture, maintainability and improvement work separately.

For example, a team might monitor deployment frequency and change lead time alongside failed deployments, recovery time, build reliability, outstanding high-severity vulnerabilities, dependency freshness, test stability and technical-debt trends.

The measurements should lead to decisions. If the dashboard is only displayed at a governance meeting, it will become reporting overhead.

When a metric deteriorates, investigate the underlying constraint. An increase in lead time might come from oversized changes, slow test suites, unstable environments or manual approval steps. A high failure rate might point towards poor integration testing, weak deployment controls or architectural coupling. Increasing recovery time might indicate inadequate observability or ownership.

Technical debt should be managed in the same way.

Avoid maintaining a large undifferentiated list labelled “tech debt”. Record the specific engineering condition, the consequence it creates and the circumstances under which that consequence becomes significant.

A useful technical-debt record might state that a component relies on an unsupported library, identify the systems affected, describe the security or delivery constraint it creates, estimate the work needed to resolve it and record an accountable owner.

Prioritise debt according to its impact on change, reliability, security and operational risk rather than according to developer preference alone.

At programme scale, standardisation can reduce the amount of quality work every individual team needs to invent.

Rather than telling each team to establish good CI/CD, provide reusable pipeline components. Rather than requiring each service to design secret scanning independently, provide a standard implementation. Rather than documenting cloud-security rules in guidance, encode them as automated policy. Rather than asking every team to create observability conventions from scratch, provide a standard telemetry pattern.

This is where engineering standards can evolve into engineering platforms or “golden paths”. Teams retain the ability to make different choices when their context demands them, but the easiest route to production already contains the organisation’s expected controls.

A practical improvement cycle can then remain relatively simple:

• Establish the current engineering condition using observable evidence.
• Identify the constraint creating the greatest risk or friction.
• Define a specific target condition.
• Make one or two measurable changes.
• Observe whether those changes improved the system.
• Incorporate successful approaches into reusable standards, tooling or platform capabilities.
• Repeat.

The important characteristic is movement rather than theoretical maturity.

A service does not become high quality because it passes a one-off assessment. Software, dependencies, infrastructure, threats, integrations and user needs continue to change. The quality system therefore needs to change with them.

For NHS digital teams, the most effective engineering quality model is one in which assurance becomes an outcome of normal delivery activity. Tests continuously produce evidence. Pipelines enforce agreed controls. architecture decisions are recorded as they are made. Clinical risks connect to implemented mitigations. Production telemetry validates system behaviour. Technical debt is visible and actively managed. Incidents feed improvements back into design.

NHS software engineering quality checklist

• Check NHS DTAC requirements: confirm whether the Digital Technology Assessment Criteria apply and maintain the evidence needed for NHS digital health assurance.
• Create a Software Bill of Materials: generate an SBOM for each releasable software artefact so components and software supply chain risks can be identified and managed.
• Plan for open source and reuse: publish new NHS source code under an appropriate open licence where possible, documenting clear reasons for any exceptions.
• Assess AI-specific engineering risks: where artificial intelligence is used, evaluate its impact on inclusion, user experience and the reliability of information or decisions.