Written by Technical Team | Last updated 16.10.2026 | 17 minute read
Performance problems in Rhapsody rarely present as a single obvious fault. A busy integration engine can continue accepting messages while processing slows elsewhere, queues form behind an expensive route, or route execution threads spend most of their time waiting for an external resource. CPU may be high, but it may also remain relatively quiet while throughput deteriorates.
The useful starting point is therefore not server utilisation alone. Rhapsody provides performance statistics that measure how processing time is distributed across routes and filters, allowing you to identify which parts of the configuration are consuming the greatest proportion of message-processing time. Used alongside queue depth, route execution thread state, message latency and operating-system metrics, these statistics give a much more precise view of where an integration bottleneck actually sits.
Rhapsody’s performance-statistics collection is deliberately switched on for a defined period rather than running continuously. Collection introduces its own processing overhead, so it is intended primarily for controlled performance investigation and is recommended for use in a non-production environment. That also affects how you should use the results: collect data against a workload that resembles the problem you are trying to diagnose, rather than enabling statistics against an arbitrary test flow and assuming the result represents production behaviour.
Performance statistics are available in the Management Console under Reports > Performance Statistics. Collection must be started manually. You select a collection duration, start the collection, and Rhapsody stops automatically when that period expires. Collection can also be stopped manually and the results discarded if the sample is no longer useful.
The first view to examine is route performance.
Rhapsody compares the processing time consumed by currently active routes across the engine. Routes accounting for more than five per cent of total message-processing time are shown separately in the performance chart, with lower contributors grouped together. The associated table presents the routes consuming the most time, with the top ten shown by default.
The percentage itself is useful, but it is not enough to decide that a route is badly designed. An integration processing a very high proportion of the engine’s messages can legitimately account for a large proportion of processing time. What you need to establish is whether the processing cost is reasonable for the workload.
Consider two routes during the same test period.
One processes 500,000 simple ADT messages and consumes 35 per cent of total route processing time. Another processes 4,000 messages and consumes 20 per cent. The first route may simply be carrying most of the workload. The second is much more interesting because relatively few messages are consuming a disproportionate amount of processing capacity.
That is why performance investigation should move from the route-level view into the filter statistics rather than stopping at the headline chart.
Queue behaviour also gives useful context. A route has a processing queue, while communication points have inbound and/or outbound queues depending on their direction. Under normal operation, queuing should generally be transient. Persistent queue growth means work is arriving faster than it is leaving that part of the flow.
A large route queue can have several causes that look similar from the outside:
A high-cost route identified by performance statistics, accompanied by a steadily growing processing queue, is a much stronger indication of an internal processing bottleneck than a high route percentage on its own.
Conversely, a route may appear healthy while its output communication point develops a backlog because the receiving system is responding slowly. That is not the same problem, and optimising filters in the route will not fix it.
The filter statistics are usually where a broad performance problem becomes an actionable engineering problem.
Rhapsody reports the filters taking the greatest amount of message-processing time. As with routes, filters responsible for more than five per cent of total processing time are displayed individually, while the remainder are grouped together. The table shows the top ten filters by default and includes the filter name, parent route, filter type, percentage of total processing time, total processing time, message count and mean processing time.
Those fields should be considered together.
Percentage of total time tells you how dominant the filter is across the workload captured during the collection period.
Total time shows the cumulative processing cost.
Messages tells you the volume that passed through the filter.
Mean gives you the average processing time per message for that filter on the route.
The relationship between total and mean time is particularly useful. A filter can appear expensive for two very different reasons.
A high-cost Rhapsody filter is not automatically a badly performing filter. When analysing Rhapsody performance statistics, compare total processing time with message count and mean processing time. A filter may consume a large share of engine time because it handles most of the traffic, while a low-volume filter with a high mean processing time can indicate a more specific Rhapsody route performance bottleneck.
A filter with a low mean execution time may still account for a large percentage of total processing simply because virtually every message passes through it. The optimisation opportunity is then about shaving a small amount of cost from a very hot path. Even a modest reduction can be worthwhile if the filter executes millions of times.
A filter with a high mean execution time but relatively few messages is a different class of problem. It suggests that the work performed for each message is expensive. That may point towards complex message manipulation, repeated parsing, encryption, database access or another heavyweight operation.
Rhapsody explicitly identifies message complexity, large messages and compute-intensive transformations such as encryption as possible causes of CPU-bound processing. Message parsing is another important cost. Parsing a message is expensive, and repeated parsing becomes particularly wasteful when several definitions are applied unnecessarily to the same message.
When an expensive route contains a chain of transformations, validation and manipulation filters, it is worth reviewing whether each stage needs to exist independently. The Rhapsody guidance recommends minimising processing through the most time-consuming filters and considering whether multiple operations can sensibly be combined.
That does not mean collapsing everything into one large script. Maintainability still matters. The objective is to remove avoidable work: repeated parsing, duplicated transformations, redundant property extraction or filters whose output could have been produced during an earlier processing stage.
The collected statistics can be exported to CSV. For any serious performance exercise, keeping these exports is useful because it allows you to compare runs after making configuration changes rather than relying on impressions from the Management Console.
A practical optimisation cycle is straightforward: establish a representative workload, collect a baseline, identify the dominant route and filters, make one controlled change, repeat the same workload, and compare the result. Changing five things at once may improve performance, but you will not know which change produced the improvement or whether another introduced a regression.
One of the more deceptive Rhapsody performance problems occurs when there is a large backlog but CPU utilisation is not especially high.
The engine uses route execution threads to process messages on routes. By default there are ten route execution threads, meaning that only ten messages can be actively processed by routes at the same time. Each thread works on one message, and threads are shared across routes rather than being permanently assigned to a particular route.
A thread takes the next eligible message from the available routes, with route priorities influencing scheduling. It is therefore possible for multiple route execution threads to be processing messages from one route simultaneously.
This model becomes important when filters access external resources.
Take a database lookup filter. While the filter waits for the database to respond, the route execution thread handling that message remains occupied. The processor may have very little local work to do during that period, so CPU usage can remain low. However, the thread is unavailable for other route work.
If several messages enter the same slow database lookup and all ten default route execution threads become occupied waiting for responses, route throughput can fall dramatically even though the server itself appears underused.
The symptom is usually a queue growing somewhere behind the blocked work rather than high CPU.
Rhapsody’s Threads view helps confirm this kind of problem. Route execution threads expose their current state, the route they are processing, the filter they are processing and the length of time spent in that state. Relevant states include pulling from an input queue, delivering, property extraction, filtering a message, routing and waiting for work.
When investigating a suspected filter bottleneck, the useful question is whether route execution threads repeatedly spend long periods inside the same filter. Performance statistics tell you which filters consume time in aggregate; the thread view lets you see what the engine is doing at a particular moment.
For database and other external lookups, optimisation should generally start with the dependency itself. Check query execution time directly against the database. Review network latency. Examine indexes and query design. Determine whether the lookup actually needs to be synchronous inside a route.
Rhapsody specifically suggests moving external lookups to communication points where possible. Communication points use their own threads, so waiting for an external response does not occupy the shared pool of route execution threads in the same way.
Simply increasing the number of route execution threads can sometimes improve throughput, but it should not be treated as the first response. If ten threads are blocked on an inefficient database query, increasing the pool may simply create twenty simultaneous inefficient queries. That can move the bottleneck into the database and make contention worse.
Filter concurrency controls can also be relevant. Rhapsody allows maximum concurrency to be limited for filters where unrestricted simultaneous processing would be undesirable. A complex Mapper operation, for example, can be restricted to a small number of concurrent executions so that it does not monopolise memory under load. Database operations can similarly be constrained where concurrent transactions are causing deadlock contention.
There is a trade-off. Once a filter reaches its maximum configured concurrency, route execution threads may wait for access to it. Rhapsody applies a timeout to this wait, with a default maximum waiting period of 30 seconds. A concurrency limit can protect an external resource or the engine, but an unnecessarily restrictive limit can itself become the reason a route queues.
Performance statistics show where Rhapsody spends processing time. They do not by themselves tell you whether the underlying limitation is CPU, disk, memory, an external dependency or configuration behaviour.
For that, you need to correlate several views.
Message latency measures the time a message spends inside the Rhapsody engine between a selected input and output communication point. Reports include message count, mean latency, standard deviation, minimum and maximum processing times. They also allow the slowest message for a time division to be opened in Message View, which is useful when the average looks acceptable but individual messages are taking unusually long.
That makes latency a useful validation measure after filter-level analysis.
Suppose performance statistics identify a mapper as the most expensive filter. If message latency rises at the same time, the route queue grows and CPU is saturated, the evidence points towards an internally CPU-intensive processing path.
If latency rises and queues grow while CPU and disk remain quiet, inspect route threads for blocked filters and test any external dependencies.
If Rhapsody processing statistics do not reveal an expensive filter but an outbound queue continues growing, investigate the communication point and receiving system instead. Communication points have their own processing threads. A slow Out->In communication point, for example, can accumulate messages while waiting for synchronous responses without consuming the route execution thread pool in the same way a filter query does.
Disk also needs separate attention. Rhapsody depends heavily on datastore I/O for messages and other persisted information. High disk queue length can constrain performance even when filter logic itself is efficient. Possible causes include slow storage, a slow path to shared storage, interference from other applications and excessive use of indexed message properties.
The important operational habit is not to treat any single graph as the diagnosis.
A useful investigation normally combines:
The sequence matters. Start close to the message-processing path, then work outwards. If performance statistics show one route consuming most of the engine’s time, inspect its filters before tuning VM memory or storage. If every route is slowing simultaneously and disk queues are high, changing one mapper is unlikely to produce a meaningful improvement.
Route and filter performance statistics become more useful when they are interpreted alongside the operating limits of the Rhapsody engine. Some performance problems will not appear as an obviously expensive filter: storage latency, JVM heap behaviour, FIFO constraints and communication point architecture can all restrict throughput elsewhere in the message flow.
The following figures and behaviours provide useful reference points when investigating Rhapsody performance. They should be treated as diagnostic indicators rather than universal production thresholds, as the appropriate limits will depend on message volumes, message size, configuration complexity and the underlying infrastructure.
| Performance indicator | What to look for | What it can indicate | Engineering response |
|---|---|---|---|
| Disk queue | An Average Disk Queue Length above approximately 2 per physical disk for sustained periods | Rhapsody may be waiting on datastore I/O rather than spending time inside an individual route or filter. Slow storage, storage controllers, network-attached storage, indexed properties or competing applications can contribute. | Review datastore storage latency, indexed message properties and other disk-intensive processes. Where appropriate, separate the Rhapsody datastore from competing workloads or distribute archive I/O across suitable storage. |
| JVM heap usage | Heap usage should normally fluctuate rather than remain close to its maximum. Rhapsody guidance identifies approximately 40–80% as a useful operating range, with sustained usage above 80% requiring investigation. | Large messages, bursts of traffic, full-text searches or configuration changes may be increasing memory demand. Persistent high utilisation can lead to more frequent or longer garbage-collection activity. | Identify what is retaining or allocating memory before simply increasing the heap. Check message size, traffic bursts and configuration changes, and correlate memory graphs with GC behaviour and message-processing latency. |
| Garbage collection pauses | Long pauses or periods where processing temporarily stops despite sufficient CPU and infrastructure capacity | An oversized JVM heap can result in less frequent but much larger full garbage collections. Full GC pauses application threads while memory is reclaimed. | Review heap sizing and GC behaviour rather than assuming that allocating more memory will improve performance. A larger heap can increase the duration of a full collection. |
| Out→In communication point queue | Messages accumulating on an Out→In communication point while route processing remains relatively healthy | The external endpoint may be responding slowly. Communication points have their own threads, so this problem does not necessarily exhaust the shared route execution threads. | Measure the response time of the external system and confirm whether synchronous Out→In processing is genuinely required. Changing route execution thread settings will not normally address the underlying dependency. |
| FIFO-related delay | Later messages appear ready to leave a route but remain delayed behind an earlier, slower message | FIFO enforcement can create head-of-line blocking where messages on the same route have significantly different processing times. | Review whether the required message ordering justifies the configured FIFO mode. Where appropriate, separate different processing paths into multiple routes rather than allowing one slow message class to delay unrelated traffic. |
| Configuration size | Increasingly slow Management Console pages or Rhapsody IDE check-ins, sometimes before message throughput noticeably deteriorates | The engine configuration itself may be approaching a practical scale limit. Rhapsody documentation gives approximately 2,000 communication points as a recommended maximum based on TCP Client and TCP Server testing, with a lower practical limit for more resource-intensive components. | Review overall engine capacity rather than examining only individual routes. Resource-heavy web services, Mapper filters and other complex components can reduce the practical size of a single-engine configuration. |
| Communication point throughput limit | A particular communication point remains the bottleneck even though routes, CPU and JVM resources have available capacity | The protocol, communication point configuration or underlying system may have reached its own throughput limit. Directory-based interfaces, for example, can ultimately be constrained by disk access. | Determine whether the workload can be distributed across multiple communication points or, for larger workloads, across additional load-balanced Rhapsody servers. |
| Lookup table failure growth | A very large number of accumulated lookup failures or excessive memory use when opening lookup monitoring views | Rhapsody retains unique lookup failures by default. If the failure list is allowed to grow unchecked, it can itself consume resources and affect Management Console or IDE behaviour. | Monitor lookup failures as an operational item and correct missing mappings rather than allowing the failure list to grow indefinitely. |
The most useful Rhapsody performance work is repeatable rather than reactive.
Maintain a known workload that exercises representative message types, sizes and volumes. A test consisting only of a few simple HL7 messages may tell you whether a route works, but not how it behaves when thousands of messages arrive after an upstream system reconnects or when a large ORU message triggers several expensive transformations.
Test environments should be sufficiently similar to production to make the results meaningful. That includes the configuration, external dependencies and hardware characteristics where performance is under investigation. A route that is fast against a local test database may behave very differently against a remote production database with a higher network round trip.
When collecting statistics, choose a period long enough to include the behaviour you care about. A five-minute sample during quiet traffic can hide expensive routes that only activate during batch processing. Equally, an excessively broad sample can dilute a short but severe performance event.
Record the conditions of each run: configuration version, test data, message volume, collection duration and any relevant external-system behaviour. Export route and filter statistics before making changes. Otherwise, performance tuning quickly turns into anecdotal comparison.
Once an expensive filter has been identified, inspect its design rather than immediately increasing infrastructure. Check whether the message is being parsed repeatedly, whether a transformation is doing more work than necessary, whether the route performs an external query for every message, whether filters can be simplified, and whether concurrency is appropriate for the workload.
Then rerun the same test.
A successful optimisation is not simply a lower mean execution time for one filter. Check that overall route throughput has improved, queues remain controlled, message latency has fallen or stayed within the required range, and the change has not shifted the bottleneck somewhere else. Rhapsody performance issues commonly manifest downstream from their actual cause, so moving a backlog from one route to another is not an improvement.
The same discipline applies in ongoing support. Establish what normal queue depth, latency and host utilisation look like for the environment. Configure notifications around the behaviour that represents a genuine problem for that estate rather than relying blindly on default thresholds. Rhapsody supports alerts for conditions such as large queues, slow message processing, high CPU utilisation, low JVM memory and low disk space, but useful thresholds depend on the way each integration is designed and operated.
Trust integration engine monitoring is most effective when it answers a precise engineering question. Which route is consuming the engine’s processing time? Which filter inside it accounts for that cost? Is the filter doing local computation or waiting on an external system? Are route execution threads being exhausted? Is a queue forming because processing cannot keep up, or because a downstream endpoint is unavailable?
Rhapsody exposes enough information to answer those questions without treating the engine as a black box. The job is to join the measurements together: route and filter statistics to locate expensive processing, thread state to understand what execution is waiting for, queues to show where throughput is falling behind, latency to measure the effect on messages, and host metrics to distinguish application behaviour from infrastructure constraints. Once those signals are read together, performance tuning becomes a controlled exercise in removing specific bottlenecks rather than adding capacity and hoping the symptoms disappear.
Is your team looking for help with Rhapsody Integration Environment Support and Monitoring? Click the button below.
Get in touch