One of the most important steps for successful interoperability is thorough documentation of the implemented interfaces. Unfortunately, most vendors do not follow this practice. When vendors do document the implemented interface, they often include nothing more than an extract of the original standard. Profiles can be, and should be, used to facilitate the comprehensive documentation of interfaces.
If an interface fails, an accurate understanding of the expected system behavior is essential for resolving the problem, which is why the interface documentation a vendor provides must include more information than just a reiteration of the base standard. Purchasers should seek complete and correct interface documentation. One obstacle to gaining vendors’ cooperation in providing these specifics, however, is the fact that when a vendor "customizes" an interface, the hidden configuration details that they may consider proprietary must be made transparent and available to possible competitors as well as the customer.
Figure 9.1 shows the relationships between profiles and implementations and the associations that can be drawn among them. Profiles at the various levels provide a source of the documentation about what is to be implemented or what has been implemented. Purchasers can use profiles to express their requirements. Likewise, vendors can use profiles to convey system capabilities. Such documentation can be used to assess needs and capabilities. If the documentation is provided in a standardized computable format, then efficiencies can be gained in assessing compatibilities. Comparisons also can be made between vendor interface implementations for a given use. In Figure 9.1, implementable profile (E) and implementable profile (F) can be assessed for profile compatibility (shown as point 6). Profile E and F are documentation of requirements for what is to be implemented or what has been implemented.
These profiles are derived from the constrainable profile (B), shown as points 2 and 4. These implementable profiles constrain the national (or realm/hospital) profile that is typically specified in an implementation guide. Profiles (E) and (F) are said to be compliant with profile (B) if the rules for adding constraints are followed faithfully. Likewise, profile (B) is said to be compliant with profile (A) if the rules for adding constraints are followed faithfully. In this case profile (B) is constraining the base standard (A). The base standard can be considered a constrainable profile.
Implementations (C) and (D) are conformant to profiles (E), (F), and (B) if the software implements the requirements as stated in the specification (shown as points 1, 3, 7, and 8 in Figure 9.1).
Finally, implementations (C) and (D) are said to be interoperable if they can exchange information and use the exchanged information as intended (shown as point 5).
Compliance and compatibility are terms that are used to indicate relationships between profiles (that is, its documentation). Conformance is a term that is used to indicate the relationship between a profile and an implementation. Interoperability is a term that is used when discussing the relationship between implementations. Table 9.1 summarizes these relationships and the various assessments that can be made.
|
Test Type |
Dimension |
Artifact |
Description |
|---|---|---|---|
|
Profile Compliance (Points 0,2,4) |
Hierarchical |
Profile |
Profiles are tested against each other to determine whether one is a constraint of (i.e., consistent with) the other. Profile compliance testing is appropriate when additional constraints are specified to successive profiles in the hierarchy (e.g., standard to a constrainable profile to an implementable profile). |
|
Implementation Conformance (Points 1,3,7,8) |
Hierarchical |
Implementation |
Provides an assessment of how well the application fulfills the requirements specified in a profile. This is conformance testing. ONC Health IT Certification is an example of conformance testing. |
|
Profile Compatibility (Point 6) |
Peer |
Profile |
Profiles are tested against each other to determine whether the pair can be used by applications to successfully exchange information (interoperate). If a profile pair that constrains the same underlying profile conflict with each other, chances of interoperability for applications that implement these profiles are diminished. |
|
Implementation Interoperability (Point 5) |
Peer |
Implementation |
Applications are tested with each other to determine whether they can successfully exchange information (interoperate). Applications that implement the same profile or compatible profiles and have successfully passed conformance tests have increased likelihood of interoperating. IHE connect-a-thons are an example of interoperability testing. |
The ultimate goal in development of an interface is to ensure that two implementations of that interface are interoperable with each other. The final step before deployment of an interface is to test the interface. In the simplest case, this test is performed "live"; i.e., two systems are directly tested with each other, for example, as in the IHE Connect-a-thon. For a hospital, performing a live test is often difficult to accomplish because of their implementation-specific requirements, undocumented requirements, or documented requirements that deviate from the standard. At minimum, however, one would like to know whether a system can exchange data with another system or not. Documentation is the key.
Claims such as "The system can support/speaks HL7" that are made by vendors are not informative or helpful to potential purchasers, nor is a statement like "the system is compliant to the guidance" useful, because each system participating in the interface may have been designed with different interface requirements. As an example, usage for an element could be set to "Required" in one of the systems. This usage setting is not a significant issue as long as this system is not acting as the receiver of information and the partner sending-systems are not transmitting this information; however, this scenario could result in a problematic mismatch between a sender and receiver if the sender transmits a message without required information and the receiver subsequently cannot process the message.
To mitigate the downside of not being able to perform live testing of two systems (Figure 9.1 and Table 9.1, Point 5), an alternative must be sought (Figure 9.1 and Table 9.1, Point 6). Table 9.1 gives possible avenues for making this assessment as long as sufficient documentation is provided.
A standard provides the foundation for implementers and includes many options, which can lead to multiple interpretations and implementations that exhibit different behaviors depending on the options chosen by the implementers. Given the possible variations in implementation behavior, it is essential that vendors’ claims for conformance to the standard are backed by documentation that clearly describes the capabilities supported. This documentation can be articulated in varying degrees of completeness and quality. To emphasize the importance of documentation, we describe characteristics of documentation quality. Some aspects are hierarchical in nature, forming levels of quality. Other aspects are necessary to ensure completeness. Table 9.2 presents documentation quality levels.
|
Documentation Claim |
Description |
|---|---|
|
Undocumented Unsubstantiated Claim |
A developer of an implementation claims conformance to a given standard; however, the claim is unsubstantiated. |
|
Documented Unsubstantiated Claim |
A developer of an implementation provides evidence of a claim with documentation of the interface. The documentation can be in any format (e.g., a text document) and the contents of the claim are not substantiated. Note: For example, the provider of the documentation (e.g., a vendor) may copy paragraphs from the original standard; this approach represents the type of documentation at this level. |
|
Documented Standard Unsubstantiated Claim |
The documentation fulfills the requirements of the conformance profiling mechanism provided by the underlying standard. A text document of a state-level (e.g., guide for Immunization for Texas) profile is an example. |
|
Documented Standard Machine Processable Unsubstantiated Claim |
The documentation is machine process-able, such as HL7 v2.x XML message profile definition (See Section 10 and Appendix C). Tools can aid in the development of machine process-able documentation. Documentation at this level enables automated comparison of specifications and implementations. |
|
Documented Standard (Implementable Profile Level) Machine Process-able Unsubstantiated Claim |
The documentation is a message profile fulfilling the criteria for implementable profiles in a machine process-able format. Such documentation defines precisely the capabilities of the implementation. Tooling, as mentioned previously, can provide the machine process-able documentation and also allows for verification that this claim is an implementable profile. |
|
Substantiated Claim |
The implementation is verified to a claim (i.e., a claim in this list) made in the documentation. The verification is performed in a testing or certification program. |
It is important to note that this quality hierarchy is designed for assessment of documentation and not for systems. The primary goal is to support assessment of the compatibility capabilities of proposed interface implementations. This determination of the quality of the associated documentation provides a first-level review prior to interoperability testing with implementations.
Table 9.2 represents a tiered structure of the steps for determining the quality of documentation related to vendors’ conformance claims. Substantiating a claim is most meaningful when applied to an implementable profile.