Rhapsody Trust Integration Engine FIFO Explained: How Message Ordering Works Across Routes

FIFO in Rhapsody is often treated as though it were a simple switch: enable first-in, first-out processing and messages will move through the integration engine one at a time in the order they arrived.

That is not how Rhapsody’s route execution model works.

A Rhapsody engine is multi-threaded. By default, ten worker threads are available for route processing, and a route execution thread can work on a message independently of the threads processing other messages. A later message can therefore complete an expensive filter before an earlier message, take a different conditional path, or spend less time waiting for an external dependency. Depending on the FIFO mode configured on the route, Rhapsody may allow that message to leave immediately, hold it until preceding messages have completed, preserve ordering created by a de-batch operation, or prevent concurrent processing altogether.

For an NHS Trust integration estate, those behaviours need to be understood precisely. Interfaces carrying ADT events, order updates, results, document notifications or state-changing messages may depend on sequence in ways that are not obvious from the route diagram. The problem becomes more complicated when a flow spans several routes, messages are split, a collector is involved, or an error is reprocessed.

FIFO should therefore be designed around the ordering requirement of the message flow, rather than enabled indiscriminately as a precaution.

Processing order and delivery order are not the same thing

The starting point is Rhapsody’s threading model.

Rhapsody normally creates ten route worker threads. A worker takes the next available route operation, processes one message through a filter, completes that processing transaction and becomes available for another operation. Threads are shared across the engine rather than permanently assigned to particular routes.

This means several messages from the same route can be in flight simultaneously.

Imagine three messages entering a route: M1 → M2 → M3

The route contains validation, a lookup and a transformation. M1 happens to require a relatively slow branch of the transformation. M2 and M3 are simpler.

Internally, processing could develop more like this:

M1 ──────────────── processing ──────────> complete

M2 ───── processing ───> complete

M3 ────── processing ────> complete

M2 may finish before M1 despite entering the route later.

FIFO configuration determines what Rhapsody is allowed to do about that difference.

This is why it is useful to separate two concepts:

Processing order describes the order in which messages actually execute through filters.

Output order describes the order in which messages are allowed to leave the route.

Most Rhapsody FIFO modes constrain the second without completely serialising the first. That allows the engine to retain parallelism while still preserving an externally visible sequence.

Rhapsody provides four route FIFO modes.

Disable FIFO Enforcement places no restriction on the order in which messages leave the route. Rhapsody will attempt to process messages in order, but there is no guarantee. If M2 completes before M1, M2 can leave first.

Enforce FIFO By Route Input Order is the default. Messages must leave the route in the same order they entered it. Internally, they may still be processed concurrently and may complete at different times. Rhapsody holds later messages as necessary so that the route’s output sequence matches its input sequence.

Enforce FIFO By Absolute Message Order (De-batching) extends the ordering requirement to messages created through splitting or de-batching. It preserves the order in which those child messages were produced as well as the order in which messages entered the route.

Enforce FIFO Within Route (Single-Threaded) provides the strongest constraint. Only one thread processes messages on the route at a time. Messages therefore enter, execute and leave sequentially.

The difference between the latter three modes is significant. A route does not need to be single-threaded merely because its destination expects messages in arrival order. Default route-input FIFO can preserve the outgoing sequence while retaining concurrent processing.

Consider an interface receiving a high volume of independent laboratory result messages. If the only requirement is for the messages to leave the route in the same order they arrived, forcing the complete route to single-threaded execution may unnecessarily restrict throughput. Route-input FIFO can provide the required delivery behaviour without giving up parallel execution.

There are cases where processing itself must be sequential, however. If the processing performed for M2 depends on state that M1 is expected to change first, preserving only output order is insufficient. M2 might be transformed before M1 has completed its state-changing operation even though Rhapsody later releases the two messages in the correct order. That is the sort of scenario where single-threaded route processing becomes appropriate.

The engineering question is therefore not simply, “Do these messages need to arrive in order?” It is whether they need to execute in order or merely leave the route in order.

Splitting and de-batching change what “first” means

Message splitting makes FIFO more subtle because one input message can become several output messages.

Suppose a source sends: A → B → C

During processing, message B is de-batched into: B1 → B2 → B3 → B4

There are now two levels of sequence to consider. The engine needs to understand the relationship between A, B and C, but there is also an ordering relationship between B1, B2, B3 and B4.

With Enforce FIFO By Route Input Order, Rhapsody guarantees ordering based on the messages entering the route. Once a message has been split, its resulting messages are treated equally from that point. That mode is therefore not intended to provide absolute ordering between every child message generated by a split or de-batch operation.

If the downstream system requires the de-batched messages to remain in the order in which they were created, Enforce FIFO By Absolute Message Order (De-batching) is the more appropriate mode.

The resulting sequence can then be maintained as: A → B1 → B2 → B3 → B4 → C

rather than merely preserving the broader relationship between route inputs.

This can matter where a batch represents a sequence of dependent transactions. If ten messages have been deliberately packaged in a known order, accepting the batch correctly but processing its contents in an arbitrary order may change the semantics of the exchange.

Single-threaded FIFO also preserves split and de-batched ordering, but carries the additional constraint that only one thread is allowed to process messages on the route. Absolute-message-order FIFO should therefore be considered separately from single-threading. The requirement to preserve de-batch sequence does not automatically mean that every operation on the route must run serially.

Message collectors add another ordering boundary.

A collector can hold messages arriving from one or more sources until the configured collection criteria have been met. Importantly, messages sent to a collector can prevent those messages from leaving the route without preventing other messages from being processed on that route.

That behaviour is easy to misinterpret in a route that has been designed around FIFO assumptions. One set of messages can remain held in a collection while subsequent route work continues.

Collectors can also introduce ordering behaviour of their own. An Input Collation collector, for example, has a Collection Order property that defines both the inputs required before processing continues and the order in which the collected messages are processed once that condition has been met.

A Message Set collector operates differently. It typically receives messages generated by a split operation, with messages carrying a set identifier and a position within that set. Processing is delayed until all members have been received.

There is another important boundary after collection. The collected messages are supplied to the attached filter as a batch. If that filter combines them into one output, ordering of the collection can be represented within that resulting message. If the filter instead emits several messages, Rhapsody does not guarantee the order of those multiple output messages.

A design that relies on the sequence of outputs from a collector should therefore be tested explicitly. It is not enough to see that the input route uses FIFO and assume that every subsequent fan-out will inherit the same ordering semantics.

Preserving FIFO across multiple Rhapsody routes

Large Trust integration configurations are rarely implemented as one route per interface from socket to socket. Processing is often decomposed into several routes so that validation, transformation, publication and destination-specific processing can be managed independently.

Once a message crosses a route boundary, route-level FIFO needs to be considered again.

Rhapsody supports directly chaining routes using cross-route connectors, but its own guidance is particularly relevant where strict ordering must be preserved: when two routes use Enforce FIFO Within Route (Single-Threaded), communication points such as Dynamic Routers should be used between them rather than cross-route connectors.

The reason is error handling. In some cross-route error scenarios, ordering cannot be maintained when routes are connected directly with connectors.

A safer ordered architecture might therefore look like: Source → Route A [single-threaded] → Output Dynamic Router > Input Dynamic Router → Route B [single-threaded] → Destination.

A Dynamic Router communication point provides a route boundary without requiring a conventional external transport. An output Dynamic Router can send to one or more input Dynamic Routers using configured or dynamically selected destinations. The message is placed directly into the receiving input communication point queue without being persisted again purely because of the internal hand-off.

On the receiving side, the input Dynamic Router places messages onto the associated route queue. Its running state behaves differently from a conventional input communication point: stopping the input Dynamic Router itself does not stop messages from being added to its receiving route. If flow needs to be halted, the output Dynamic Router or the receiving route must be stopped.

That behaviour is operationally relevant to ordered processing. Stopping a receiving route can cause messages to accumulate on its processing queue. When the route is restarted, the queued messages resume processing. Anyone managing an incident needs to understand where the ordered backlog currently exists before starting, stopping or deleting anything.

Dynamic Routers also support modular route design. A publishing route can send to multiple subscribers without having to be modified every time a new consuming route is introduced. That can be particularly useful in a Trust estate where a core feed is reused by numerous downstream systems.

Ordering must still be considered independently for each receiving path. If one Dynamic Router target feeds Route B and another feeds Route C, FIFO on Route B governs Route B’s sequence; it does not create a global transaction order across two independently executing destination routes.

Route priority is another mechanism that is sometimes confused with FIFO. It solves a different problem.

Rhapsody routes can be assigned Low, Below Normal, Normal, Above Normal and High Realtime priorities. A High Realtime route receives processing preference over non-realtime routes when work is waiting. If multiple realtime routes have waiting messages, scheduling attempts to prevent one realtime route from starving another; realtime processing can, however, starve lower-priority routes.

Priority determines which route gets processing capacity. FIFO determines how messages are ordered within a route. Raising the priority of an ordered route does not strengthen its FIFO semantics, and changing FIFO does not reserve worker threads for it.

The two settings should therefore be reviewed independently.

Error handling is where FIFO designs most often break

The Error Queue creates a hard boundary in Rhapsody’s normal ordering model.

When a message is sent from a route to the Error Queue, it is removed from that route. Its previous FIFO position is no longer retained.

If the failed message is later reprocessed, Rhapsody adds it to the end of the route processing queue and treats it as though it has just arrived.

Take this sequence: M101 → M102 → M103 → M104

If M102 fails and is sent to the Error Queue, the remaining messages may continue: M101 → M103 → M104

An operator later fixes the cause and reprocesses M102. It is now effectively a new route arrival: M101 → M103 → M104 → M102

Configuring ordinary FIFO does not repair that sequence. The original position of M102 has already been lost.

For interfaces where this is unacceptable, Rhapsody supports a deliberate operational pattern. Configure the route as single-threaded and enable Stop the route when a message is sent to the Error Queue. If a message fails, the route stops when that message is placed in the Error Queue. The fault is resolved, the message is reprocessed onto the route, and the route is then restarted. Because subsequent messages have not continued through the stopped route, the reprocessed failure becomes the next message to execute.

In simplified form: M101 → M102 [fails] → ROUTE STOPS

Queue remains: M103 → M104 → M105 …

After remediation: reprocess M102 → restart route

Processing continues: M102 → M103 → M104 → M105

This is substantially different from simply enabling FIFO and monitoring the Error Queue.

It also creates operational obligations. An ordered interface configured to stop on error needs alerting and support procedures capable of responding to a stopped route. A transient malformed message can intentionally halt all following traffic because that is the mechanism preserving sequence. For an interface carrying high volumes, the resulting queue can grow quickly.

The decision therefore belongs in the interface design, not in generic engine configuration.

Rhapsody documents a different workaround where the route is not single-threaded: enable stop-on-error, create a duplicate route to process the failed message, then restart the original route to continue processing the messages that remain queued. This is a more specialised recovery pattern and should be understood and rehearsed before relying on it operationally.

Manual manipulation of queues deserves the same caution. Once engineers start downloading, deleting, redirecting or reprocessing individual messages during an incident, the sequence seen by the destination may no longer correspond to the source sequence. A support runbook for an ordering-sensitive interface should state explicitly what operators are allowed to do with queued and errored messages.

Designing ordered interfaces without unnecessarily serialising the engine

The safest way to implement FIFO is to begin with the actual ordering contract of the interface.

For many feeds, strict processing order is unnecessary. Individual observations, documents or result messages may be independent even though preserving input order at delivery is desirable. The default Enforce FIFO By Route Input Order can retain that sequence while allowing Rhapsody to process messages concurrently.

Use Absolute Message Order when splitting or de-batching introduces child messages whose generated sequence must remain intact.

Use Single-Threaded FIFO when the processing of one message genuinely depends on the completion of an earlier message, or where the complete route has to operate serially to satisfy the integration contract.

Before implementing an ordered multi-route flow, document at least these behaviours:

• the point at which the required ordering begins and ends;
• whether ordering applies only to delivery or to execution inside Rhapsody;
• whether any route splits, de-batches, collects or produces multiple messages;
• how route boundaries are crossed and whether Dynamic Routers are required;
• what happens when a message fails and enters the Error Queue;
• whether later messages may continue after a failure;
• how support staff should reprocess an errored message without corrupting sequence.

Filter concurrency should be reviewed as part of the same design. Setting a filter’s maximum concurrency to a non-zero value limits how many messages can execute through that filter simultaneously. A concurrency value of one can drastically reduce throughput and effectively serialise processing through the affected part of the route.

There are legitimate uses. Database updates may need constrained concurrency to reduce deadlock contention. A resource-intensive Mapper may need a cap to prevent excessive simultaneous memory consumption. In-route orchestration may require one operation to complete before the next begins.

Concurrency controls should not, however, be used as an improvised replacement for a clearly defined FIFO design. Route FIFO and filter concurrency solve related but different problems.

Testing also needs to use messages capable of exposing ordering faults. Sending ten identical messages through an interface and checking that all ten arrived says very little. Use an explicit sequence number or another visible identifier and test under enough concurrency for messages to take different processing paths or processing times.

For example:

TEST-0001

TEST-0002

TEST-0003

...

TEST-1000

Introduce deliberately uneven processing where appropriate, exercise split/de-batch behaviour, and verify the sequence at the destination rather than simply inspecting the input queue.

Failure tests are even more important. Force TEST-0042 into the Error Queue and observe what happens to TEST-0043 onwards. If the intended design says sequence must survive a failure, the test should demonstrate that behaviour rather than assuming the FIFO setting will provide it.

For flows crossing multiple routes, test each boundary. Stop the receiving route and allow messages to build. Restart it and verify sequence. Generate an error in the upstream route, then in the downstream route. Exercise the support procedure for reprocessing the failed message.

An integration can pass perfectly during normal operation and still have incorrect FIFO behaviour during the exact failure scenario that matters most.

That is the key technical point in Rhapsody’s FIFO implementation: ordering is a property of a route and its processing mode, not a universal guarantee attached to a message for its entire journey through the engine.

The route may preserve input order while processing messages concurrently. A de-batch operation can create another ordering requirement. A collector can hold one set of messages while later work continues. A route boundary introduces another queue and another FIFO configuration. A cross-route connector may not preserve ordering during certain error scenarios. The Error Queue removes a message from its original sequence entirely.

A well-designed Trust integration flow makes each of those boundaries explicit.

The objective is not to make every Rhapsody route single-threaded. Doing so sacrifices one of the main advantages of the engine’s multi-threaded execution model and can reduce throughput dramatically on busy estates. The objective is to apply the minimum ordering constraint required by the clinical or operational behaviour of the interface, then design the route boundaries, error handling and support procedures so that the same guarantee still holds when the integration is under load or something fails.

That is when FIFO becomes an intentional part of the integration architecture rather than a checkbox that happens to be enabled.

Rhapsody FIFO FAQs

How can I check which FIFO mode a Rhapsody route is using?
The configured Rhapsody FIFO mode can be viewed from the route properties and is also exposed in the Management Console route details. For larger integration estates, the Object Browser is useful because FIFO Mode can be displayed as a route property, making it easier to audit message-ordering configuration across multiple interfaces without opening every route individually.

Are Rhapsody FIFO settings preserved when exporting and importing an RLC?
Yes. A route’s FIFO mode is included in the standard Rhapsody configuration when an RLC file is saved and loaded. If an imported route does not contain a FIFO mode, Rhapsody applies the default FIFO setting. This is worth checking when migrating older Rhapsody configurations or moving routes between development, test and production environments.

Can you change the FIFO mode on multiple Rhapsody routes at the same time?
Yes. Rhapsody IDE’s Object Browser supports bulk changes to route properties, including FIFO Mode. Multiple routes can be selected and updated together. In a Trust integration environment, bulk changes should still be reviewed route by route before deployment because identical FIFO settings can have very different effects depending on message volume, splitting, external dependencies and the ordering requirements of each interface.

Do Rhapsody communication points have their own message-ordering behaviour?
Yes. Route FIFO controls ordering within a route, but some communication point modes impose additional sequencing. In In→Out mode, Rhapsody receives a message and will not accept another until the corresponding response has been sent or the response timeout expires. Out→In works in the opposite direction: Rhapsody sends a message and waits for its response before sending the next. These request-response behaviours should be considered separately from the route’s FIFO configuration.

How can I inspect messages waiting on a stopped Rhapsody FIFO route?
The Management Console’s route Processing Queue shows messages waiting on a stopped route. With the appropriate permissions, operators can download individual messages or the complete queue for investigation. This is useful when diagnosing an ordering-sensitive interface because the queue provides visibility of the messages that have not yet been processed, allowing the sequence to be reviewed before the route is restarted or any queue administration is performed.