Introduction

This is part of my HL7 article series. Before we get started on this tutorial, have a quick look at my earlier article titled “A Very Short Introduction to the HL7 2.x Standard”. My article on “HL7 Programming using HAPI - Message Validation” provided an overview of why technical and semantic interoperability within healthcare systems is needed, and the various challenges that exist in this space. In that tutorial, we also looked at various approaches that are utilized in HL7 2.x systems to address these challenges and then looked at features available within the HAPI framework that enable message validation. This included "basic" as well as "custom" message validation functionality*. In this tutorial, we will dive a into a more advanced topic around message validation called Conformance Profiles (also known as Message Profiles in HL7 2.x) and see how this is supported in the HAPI framework.

Let me mention something quickly before we get started. The area that I am exploring in this article is an extremely complex one, and the HL7 group and other related organizations such as NIST, IHE, etc. have published hundreds if not thousands of documents, tutorials, whitepapers, etc. some of which are available freely and some may be for members only to help you get started. These documents should be what you and your team should ultimately rely on during for any implementation. However, my goal in this short tutorial is to simply highlight and help bring attention to this important area since interoperability is often the ultimate goal of any message exchange-based workflow. Armed with some fundamentals provided in this article, you should be able to explore these topics more deeply on your own. As you explore these topics further on your own, you may run into documents that were published a long time ago and may not have been updated more recently. This is very common. The HL7 standard (2.x version) has been around for a long time, and so, several critical tools, documents and other links on this subject (some of which I cover in this article) are extremely old and have not been updated in a long time. Like many other large standards organizations, things do take a long time to move forward within the HL7 standard. This is because any activity almost always requires consensus/approval from many members, groups, committees and other partner organizations to move forward. A lot of work in this area is ultimately achieved through volunteer effort. And so, if you are frustrated about anything, perhaps you can contribute to any efforts in that area. When in doubt, always contact your regional affiliate, or its main site here directly for more information. I am not an expert in this area and may not be able to answer every question and so, please be ready to research or reach out to other experts in the field on this or any other topic that I have covered in this article series. Let us get started.

Tools for Tutorial

• JDK 1.4 SDK or higher
• Eclipse or any other Java IDE (or even a text editor)
• Download HAPI compiled library from the HAPI project website
• View HAPI source code on their GitHub site here
• Download HAPI Test Panel (optional) from here
• XSD schema file for HL7 2.x the Conformance Profile
• Message Workbench Tool and related information can be found here
• Message Maker Testing Tool and related information can be found here
• HAPI documentation on the conformance-related tools and testing can be found here
• You can also find all the code demonstrated in the tutorial on GitHub here

Need for Conformance and Validation in HL7

In my previous article in my HL7 article series, you will recall that I had described that the information optionality that exists within the data model of HL7 2.x standard is seen both as a strength and a weakness at the same time. The optionality enables flexibility in design as it sometimes offers multiple ways to enable the same information exchange. However, a major problem (and a weakness) this also created was in technical and semantic interoperability where parties exchanging data did not have a concrete way of claiming strict adherence towards a specific use case scenario. Technical interoperability refers to "syntactical" characteristic of the message exchange, and semantic interoperability, which is the highly sought-after but sometimes elusive goal in any message exchange, is the enablement of the correct "understanding" of the exchanged information without any ambiguity. These two characteristics are extremely important since ultimately the success/quality of any technical implementation (healthcare-related or otherwise) is dependent on how well the initial requirements can be conveyed between the parties involved and later can be independently verified either before or after the technical implementation. We will look at some of the terminology related to this area in some more in detail before diving into some code examples.

“No matter what people tell you, words and ideas can change the world.” ~ Robin Williams

Many standards utilize terminology such as "conformance", "compliance", "verification" and "validation" (some of these words are often used interchangeably even though there are subtle differences in their meanings). ISO, a large standards organization, for instance defines conformance as "the fulfillment of a product, process, or service of specified requirements". The act of "conforming" to a specification usually means that a given implementation adheres to a precise specification and the interpretation of the requirements. This process itself is usually enabled by another important process known as "conformance verification" which allows for meaningful assertions on that technical implementation. Such mechanisms and constructs are extremely important if we attempt to discuss "plug-n-play" for instance which would be almost impossible since any HL7 data could be transmitted in many ways with many different message definitions. It is not uncommon for many healthcare implementations to go live after a lengthy effort and still require "tweaks" after as each side cannot be sure what to expect from one another during the entire message processing workflow. To address these and many other challenges in this area, the HL7 2.x group came up with a new mechanism called "message profiles" by which these problems can be mitigated/remedied. We will look at how this works next.

HL7 Message Profiles

A message profile in the HL7 2.x standard is a precise and unambiguous specification of a HL7 message that has been "constrained" to be used towards a specific set of requirements. The word "constraint" refers to the idea of reducing the optionality within the definition of a specific message so there can be no confusion as to the mandatory information and behavior that can expected from one another by each party involved in the message exchange. However, there is more to message profiles than simply reducing optionality. The definition defines things such as the static structure of the messages, the content of the HL7 message carried by these structures, and also any "dynamic interaction characteristics" of the message flow which explain how and when the messages are communicated from the sending application to one or more receiving applications.

The overall information communicated between the various parties involved when specifying conformance or message profile includes information such as:

• Use case information that describes what the goal of the message exchange is and who the actors are in this message exchange
• Interaction information that describes the collaboration between the actors (parties) involved
• Authorship and organizational information of the message profile
• Role or perspective from which this information is written (Sender or Receiver for instance)
• Version of the HL7 message as well as the message trigger type and event
• Structural information of the HL7 message such as optionality, cardinality, length and content that is permitted within it
• A set of user-defined table values to be used within the fields used in the message segments
• Information whether a specific HL7 message requires an accept acknowledgment or an application acknowledgment
• A unique identifier to help to distinguish profile definitions from others (HL7 group may assign one if this is empty when submitting to a global registry)

All this information is encoded into an XML file with a precisely defined XSD schema declaration (see screenshot below showing the XSD for this file) enabling easy validation through any XML processor to ensure that the encoded information is acceptable for use by other tools, applications and end users. Several tools and approaches are also used in conjunction with this XML file to create, exchange, test and validate any message interface. We will look at how they all work together next.

Messaging Workbench

Although the XML-based message profile file can in theory be generated by hand, it can be a tedious activity. Therefore members, vendors and various implementors have came up with various tools and approaches over the years to automate the generation of this file. One such tool is the Messaging Workbench. This is a free Windows desktop tool that enables the development of the message profile and was developed out of a healthcare initiative undertaken by the Department of Veteran Affairs in the United States. The message profile to me is really a consolidation of three sub-profiles (message, segment and field levels), and its helps reduce any ambiguity as far as structure, content and any message acknowledgement behaviour pertaining to the message. A screen shot of the tool is shown below highlighting some areas that will be worth noting for now. Commercial tools also happen to exist that help automate the production of these message conformance profiles, but I cannot vouch for the accuracy and/or stability of such tools on the market as I have not used them myself.

There are a lot of convenient features in the Messaging Workbench tool including the following:

• Start with and/or easily customize pre-built "starter" message profiles
• Ability to compare the differences between two message profiles on an element by element basis
• Ability to capture and reverse engineer message profiles from sample HL7 messages
• Generate various reports highlighting many message profile characteristics
• Export the message profile definition in PDF format reports
• Register the message profile in the local repository
• Register the message profile with HL7
• Browse and download message profiles from the HL7 registry

You can find a link to the download page as well as usage instructions for the Message Workbench here. Installing the application should be straight forward as there is an installer provided (use the "setup.exe" to launch the installation) and the application should install normally like any other Windows programs. The program comes with some helpful documentation, but you may find this a bit overwhelming if you are just getting started with it. Read the documentation before getting started or contact your HL7 regional affiliate, or the HL7 main site here directly if you have any questions.

I created a quick messaging workbench project and generated a sample message profile using the workbench and have included the files in my GitHub page for your reference. As you can see from the screenshots shown below, the workbench project enables you to define the structure and content that is permitted for a message processing workflow, the role that each actor plays in the messaging workflow as well as the message acknowledgement behavior that is expected for the transmitted messages. For our example, I am specifying an ADT A01 message with four message segments and have arbitrarily defined various rules such as cardinality, optionality, length, code set rules, etc. for the segments as well as the fields within these segments. To enable testing within the messaging workbench itself, I have also used a sample ADT A01 message (also provided in my GitHub page) which intentionality does not conform to many of the rules such as length as well as permitted data within code-set enabled fields.

A nice feature of the Messaging Workbench is that you can easily test how well a HL7 message matches up to your message profile. Screenshot shown below shows me testing a sample ADT A01 message against the message profile under development. A list of validation errors and warnings generated are shown in the screenshot below for your reference.

Once you have completed specifying the message profile characteristics within the messaging workbench, you should be able to generate the message profile XML file using the export options provided under the "File" menu of the application. For our example, I used the "Export HL7 Conformance Profile" menu to generate the XML file that I will be using later for our HAPI-related examples below to demonstrate how application validation can be enabled using this message profile.

Message Maker

Once you have the message profile (see screenshot of the XML file above), you can either upload it to a global registry or distribute it to a partner directly who can proceed to use the information in several ways. One interesting way to use the message profile is to help generate test HL7 messages using a tool called "Message Maker". This tool uses the message profile definition information contained within the XML file to create a test suite of both valid and invalid HL7 messages to enable the implementors to test for both good as well as bad scenarios when implementing this interface specification. The test messages generated can be output in multiple formats including ER7 and XML which can be used for any development and testing related efforts of the parties working towards the implementation of this message interface. Screenshot of the message maker tool is shown below for reference. You can find a link to the download page as well as usage instructions for the Message Maker here.

Using Message Conformance Profiles with HAPI

Let us now look at the support offered by the HAPI HL7 library in the area of message validation using conformance or message profiles. HAPI offers two different mechanisms around message validation using message conformance profiles. They are namely:

• Through generation of strongly typed classes against this message profile XML file using a Maven plugin
• By using a runtime validation enabled by the RuntimeProfile and the DefaultValidator classes in the HAPI HL7 library

I am only going to illustrate the runtime validation mechanism here and I will let you explore the generated class approach by yourself. Please note that there is a help page containing useful documentation on using the various tools HAPI provides towards conformance-related testing including generating strongly typed classes that I mention above. You can find that information here.

In the code example shown below, I take the same message profile that I previously generated and perform message validation against a fictional HL7 ADT A01 message. This HL7 message is not valid according to the rules specified in the message profile, and we should be able to use the HAPI framework to generate the list of warnings and errors that arise as a result. HAPI makes the process of validation against message profiles easy by permitting us to use the same message validation mechanism that we had previously explored in our tutorial on basic as well as custom message validation. We use a special type of parser called ProfileParser which helps parse the message profile XML file into a RuntimeProfile object. We also instantiate a pipe parser, parse the test HL7 message that we are wanting to validate and pass it on to the DefaultValidator class which helps validate the message object against the definition contained in the profile object. Any errors and warnings are returned as an array of instances of the familiar HL7Exception class. I then print these errors to the program console for display.

    package com.saravanansubramanian.hapihl7tutorial.conformanceprofile;

    import ca.uhn.hl7v2.HL7Exception;
    import ca.uhn.hl7v2.conf.check.DefaultValidator;
    import ca.uhn.hl7v2.conf.parser.ProfileParser;
    import ca.uhn.hl7v2.conf.spec.RuntimeProfile;
    import ca.uhn.hl7v2.model.v23.message.ADT_A01;
    import ca.uhn.hl7v2.parser.PipeParser;
    import java.nio.file.*;

    public class HapiConformanceProfileBasicExampleOfValidationFailure {

      public static void main(String[] args) throws Exception {

        try {

          // Load the conformance profile
          ProfileParser ourProfileParser = new ProfileParser(false);
          RuntimeProfile ourConformanceProfile = ourProfileParser.
              parseClasspath("com/saravanansubramanian/hapihl7tutorial/"
                  + "Saravanan Adv Testing Example profile - ADT_A01.xml");

          //Read a test non-conformant HL7 from file. See my GitHub page for this file
          String message = readHl7FileDataAsString("C:\\HL7TestInputFiles\\FileWithNonConformingAdtA01Message.txt");

          //parse the HL7 message from the file data
          ADT_A01 msg = (ADT_A01) (new PipeParser()).parse(message);

          // Validate the HL7 message using the sample HL7 conformance profile I have provided. See my GitHub page for the XML file
          HL7Exception[] errors = new DefaultValidator().validate(msg, ourConformanceProfile.getMessage());

          // Display all the validation errors that are generated.
          System.out.println("The following validation errors were found during message validation:");

          for (HL7Exception hl7Exception : errors) {
            System.out.println(hl7Exception);
          }


        } catch (Exception e) {
          //In real-life, do something about this exception
                e.printStackTrace();
        }

      }

      public static String readHl7FileDataAsString(String fileName) throws Exception
        {
          String data = new String(Files.readAllBytes(Paths.get(fileName)));
          return data;
        }

    }

The console output from running our program is shown below. As you can see, the validation process checks the sample HL7 message against the message profile generated earlier using the Messaging Workbench tool. Various validation errors and warnings are shown below including length as well as message content mismatch related errors. You will recall seeing these validation messages when testing the message profile against the sample HL7 message in the messaging workbench tool as well.

The following validation errors were found during message validation:
ca.uhn.hl7v2.conf.check.ProfileNotFollowedException: Message structure null doesn't match profile type of
ca.uhn.hl7v2.conf.check.ProfileNotFollowedException: The type namespace ID has length 19 which exceeds max of 3 at MSH-3
ca.uhn.hl7v2.conf.check.ProfileNotFollowedException: The type namespace ID has length 16 which exceeds max of 3 at MSH-4
ca.uhn.hl7v2.conf.check.ProfileNotFollowedException: The type namespace ID has length 21 which exceeds max of 3 at MSH-5
ca.uhn.hl7v2.conf.check.ProfileNotFollowedException: The type namespace ID has length 18 which exceeds max of 3 at MSH-6
ca.uhn.hl7v2.conf.check.ProfileNotHL7CompliantException: HL7 datatype TSComponentOne doesn't match profile datatype NM at MSH-7
ca.uhn.hl7v2.conf.check.ProfileNotFollowedException: The type Message Control ID has length 21 which exceeds max of 20 at MSH-10
ca.uhn.hl7v2.conf.check.ProfileNotHL7CompliantException: HL7 datatype TSComponentOne doesn't match profile datatype NM at EVN-2
ca.uhn.hl7v2.conf.check.ProfileNotFollowedException: The type ID has length 6 which exceeds max of 3 at PID-3
ca.uhn.hl7v2.conf.check.ProfileNotFollowedException: The type family name has length 5 which exceeds max of 3 at PID-5
ca.uhn.hl7v2.conf.check.ProfileNotFollowedException: The type given name has length 6 which exceeds max of 3 at PID-5
ca.uhn.hl7v2.conf.check.ProfileNotHL7CompliantException: HL7 datatype TSComponentOne doesn't match profile datatype NM at PID-7
ca.uhn.hl7v2.conf.check.ProfileNotFollowedException: The type street address has length 12 which exceeds max of 3 at PID-11
ca.uhn.hl7v2.conf.check.ProfileNotFollowedException: The type city has length 16 which exceeds max of 3 at PID-11
ca.uhn.hl7v2.conf.check.ProfileNotFollowedException: The type zip or postal code has length 5 which exceeds max of 3 at PID-11
ca.uhn.hl7v2.conf.check.ProfileNotHL7CompliantException: HL7 datatype XTN doesn't match profile datatype TN at PID-13
ca.uhn.hl7v2.conf.check.ProfileNotFollowedException: The type ID has length 4 which exceeds max of 3 at PID-18

Validation and Testing Tools from IHE

In addition to the tools and techniques listed above, there is another set of testing and validation tools available from a group known as "Integrating the Healthcare Enterprise (IHE)". The primary goal of this group is to enable any coordination as well as standardization efforts especially from an implemention standpoint using standards such as HL7, DICOM, etc. This group creates as well as publishes numerous frameworks, handbooks, case studies and other resources to help make the implementation work of healthcare systems easier. For example, this group publishes something known as "IHE Profiles" which helps customers and vendors involved in both the buying and selling of healthcare systems by enabling an unambiguous way of verifying or claiming adherence to specific "use case profiles" that were previously validated/vetted by numerous testing groups representing many diverse areas within healthcare such as cardiology, pathology, laboratory medicine, etc. Through the use of the validation tools also offered by IHE, vendors and customers can test their systems against the many use cases described by these profiles. For example, the Anatomic Pathology Workflow is one such example of an IHE profile which helps test and validate conformance around aspects such as ordering, reporting, imaging that are all necessary to support any healthcare information flows pertaining to surgical pathology, clinical autopsy and cytopathology within a clinical setting. I plan on writing a separate tutorial on IHE and IHE profiles in the near future, so I will simply direct your attention to their main website for now.

Conclusion

This short tutorial covered the concept of conformance (or "message profiles" as they are known in the HL7 2.x standard) and how they are utilized during message validation process. Message profiles help precisely define the structural (static) and behavioral (dynamic) constraints of a message interface. The use of message profiles promotes interoperability by providing message exchange partners a common format for documenting, communicating, developing and testing message interfaces that conform to strict HL7 interface specifications which help enable interoperability. We reviewed several tools in this tutorial that are available from HL7 and organizations such as NIST to assist us in these efforts. These included the Message Workbench Tool, the Message Maker tool, the OID Registry and the HAPI library classes that enable the creation, exchange and validation of HL7 message interfaces. That brings us to the end of another tutorial on the HAPI framework. Like I mentioned earlier, there is a lot more to this subject, and I hope you will be able to explore this area a lot more easily armed with the information provided here. This is sadly also the end of the HAPI-related HL7 programming tutorials. However, there is a lot more to come as far as my overall HL7 series is concerned since I intend to cover many more topics that should be of interest to software developers. We may still explore HAPI in the future when I cover an exciting new initiative called FHIR (stands for "Fast Healthcare Interoperability Resources"). HAPI is used as a foundation in the FHIR reference implementation, and you will find the exposure to HAPI extremely useful when we cover that topic. However, in the next tutorial in my HL7 article series, I will be moving on to explore HL7 programming using the Microsoft .NET framework and the C# programming language. I will begin by exploring HL7 programming purely using .NET and C# programming language and introduce you to the core HL7 2.x concepts including network programming using sockets and the MLLP protocol. This should be very similar to my earlier article titled "HL7 programming using Java". Later, I will cover a nifty framework called "NHapi" that was spun off from the HAPI Java framework that we have looked at in this series so far. See you then!

* - Sometimes, I use a new term that I came up with myself as there was either no formal definition of a concept in the official documentation, or because I feel that the concept is better explained using this new terminology. When in doubt, always consult the official HAPI and HL7 documentation for final reference.