what is the role of Connectathon testing?

dritz · July 22, 2015, 7:40pm

Hi all.

I know we’ve broached this topic a few times on Architecture Community calls – but I think it will be useful to get our arms around definitively answering an important question: what is the role, for OpenHIE, of Connectathon (CAT) testing?

Folks in the OpenHIE Architecture Community know I have a point of view on this topic. I believe that we have created important value through our CAT participation to date; our various software infrastructure “puzzle pieces” have established their conformance to the IHE profiles that we’ve spec’d for OpenHIE. Usefully, CAT participation also has simplified the OpenHIE integration testing that we need to do. Because our components separately are tested at CAT, we didn’t need to replicate in our OpenHIE environment the full array of tests that a component would need to pass if they were enrolled at CAT. Lastly, it is important (in my view) for OpenHIE’s credibility that the software pieces in our infrastructure enjoy the imprimatur of having passed the IHE CAT process. CAT is an external, mature, well-recognized, well-governed process – and that is helpful both as a genuine risk mitigator and in terms of our “positioning” for OpenHIE. I’d go so far as to suggest we should help our potential “customers” (implementing jurisdictions) appreciate the importance of the fact that the OpenHIE infrastructure is standards-based and independently conformance-tested.

What are the downsides? Clearly, one of them is the cost and effort needed to attend and pass at CAT. We have enjoyed IHE scholarships the last couple of years that covered the entry fees, but there are still other costs that must be covered. It would be interesting to ask whether our OpenHIE reference implementation software developers would seek CAT conformance statements if there wasn’t funding to cover their participation. Obviously, some would (and have, in years past… of their own accord and using their own funds). But if the 3rd party testing isn’t important to our customers it won’t be important to us either… and maybe vice versa.

Another downside, related to the first one, is that CAT testing creates barriers. One is cost… another is technical capability. CAT testing isn’t an “easy pass”. Development shops that are new to eHealth and/or new to health informatics standards will need to ramp up the learning curve before they are able to “do it right”… and if they don’t “do it right” they won’t pass at CAT. Notwithstanding that I think we should be leveraging our existing centres of excellence within the OpenHIE community to give new OpenHIE participants a “leg up”, it is still a high bar and not everyone will get over it. This winnowing has its benefits, but it is less community-inclusive that favouring a low/no bar approach.

Anyway… before this is TLDR (sadly, maybe it already is!), I’ll leave it to others to carry on the pros/cons discussion – especially some who believe (on the con side) that CAT conformance shouldn’t be a prerequisite to being “OpenHIE-conformant”.

-Derek.

pbiondich · July 23, 2015, 12:35pm

Hi Derek, thanks for this note.

I think underneath your point is a more fundamental one that we need to address first as a group:

“When we judge ‘conformance’ with OpenHIE, what qualities do we want that conformance to have?”

That the technical component has some measure of “quality”… that includes attributes like “secure”, “does what it purports to do”, and “has a well understood maintenance/further development trajectory which allows the software to be more functional and heal itself over time”
That it complies with our integration testing process… i.e., “it can demonstrate the ability to interact with our architectural model in ways that we as a community formally specify”… for now that means: can support the workflows we agree to release as a community, but could include other attributes over time
That it has the ability to be included as a part of a packaging/distribution strategy, and that the “owners” of that component will assent and ideally help us with that process.

These are just my opinions… I know that others might have different perspectives.

So my stance is to agree upon the high-level components of conformance, and then we collectively as a group can decide how we want to address each of these components, one by one.

What are other people’s thoughts here?

-Paul

···

On Wed, Jul 22, 2015 at 3:40 PM, Derek Ritz (ecGroup) derek.ritz@ecgroupinc.com wrote:

Hi all.

I know we’ve broached this topic a few times on Architecture Community calls – but I think it will be useful to get our arms around definitively answering an important question: what is the role, for OpenHIE, of Connectathon (CAT) testing?

Folks in the OpenHIE Architecture Community know I have a point of view on this topic. I believe that we have created important value through our CAT participation to date; our various software infrastructure “puzzle pieces” have established their conformance to the IHE profiles that we’ve spec’d for OpenHIE. Usefully, CAT participation also has simplified the OpenHIE integration testing that we need to do. Because our components separately are tested at CAT, we didn’t need to replicate in our OpenHIE environment the full array of tests that a component would need to pass if they were enrolled at CAT. Lastly, it is important (in my view) for OpenHIE’s credibility that the software pieces in our infrastructure enjoy the imprimatur of having passed the IHE CAT process. CAT is an external, mature, well-recognized, well-governed process – and that is helpful both as a genuine risk mitigator and in terms of our “positioning” for OpenHIE. I’d go so far as to suggest we should help our potential “customers” (implementing jurisdictions) appreciate the importance of the fact that the OpenHIE infrastructure is standards-based and independently conformance-tested.

What are the downsides? Clearly, one of them is the cost and effort needed to attend and pass at CAT. We have enjoyed IHE scholarships the last couple of years that covered the entry fees, but there are still other costs that must be covered. It would be interesting to ask whether our OpenHIE reference implementation software developers would seek CAT conformance statements if there wasn’t funding to cover their participation. Obviously, some would (and have, in years past… of their own accord and using their own funds). But if the 3rd party testing isn’t important to our customers it won’t be important to us either… and maybe vice versa.

Another downside, related to the first one, is that CAT testing creates barriers. One is cost… another is technical capability. CAT testing isn’t an “easy pass”. Development shops that are new to eHealth and/or new to health informatics standards will need to ramp up the learning curve before they are able to “do it right”… and if they don’t “do it right” they won’t pass at CAT. Notwithstanding that I think we should be leveraging our existing centres of excellence within the OpenHIE community to give new OpenHIE participants a “leg up”, it is still a high bar and not everyone will get over it. This winnowing has its benefits, but it is less community-inclusive that favouring a low/no bar approach.

Anyway… before this is TLDR (sadly, maybe it already is!), I’ll leave it to others to carry on the pros/cons discussion – especially some who believe (on the con side) that CAT conformance shouldn’t be a prerequisite to being “OpenHIE-conformant”.

-Derek.

–

You received this message because you are subscribed to the Google Groups “OpenHIE Architecture” group.

To unsubscribe from this group and stop receiving emails from it, send an email to ohie-architecture+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

sgrannis · July 23, 2015, 1:08pm

I would also add a fourth dimension to conformance - that the stakeholder assents to the mission/vision/values of the OpenHIE community.

Shaun

···

On Wed, Jul 22, 2015 at 3:40 PM, Derek Ritz (ecGroup)
derek.ritz@ecgroupinc.com wrote:

Hi all.

I know we’ve broached this topic a few times on Architecture Community calls – but I think it will be useful to get our arms around
definitively answering an important question: what is the role, for OpenHIE, of Connectathon (CAT) testing?

Folks in the OpenHIE Architecture Community know I have a point of view on this topic. I believe that we have created important value through our CAT participation to date; our various software infrastructure “puzzle pieces” have established their conformance
to the IHE profiles that we’ve spec’d for OpenHIE. Usefully, CAT participation also has simplified the OpenHIE integration testing that we need to do. Because our components separately are tested at CAT, we didn’t need to replicate in our OpenHIE environment
the full array of tests that a component would need to pass if they were enrolled at CAT. Lastly, it is important (in my view) for OpenHIE’s credibility that the software pieces in our infrastructure enjoy the imprimatur of having passed the IHE CAT process.
CAT is an external, mature, well-recognized, well-governed process – and that is helpful both as a genuine risk mitigator and in terms of our “positioning” for OpenHIE. I’d go so far as to suggest we should help our potential “customers” (implementing jurisdictions)
appreciate the importance of the fact that the OpenHIE infrastructure is standards-based and independently conformance-tested.

What are the downsides? Clearly, one of them is the cost and effort needed to attend and pass at CAT. We have enjoyed IHE scholarships the last couple of years that covered the entry fees, but there are still other costs that must be covered. It would
be interesting to ask whether our OpenHIE reference implementation software developers would seek CAT conformance statements if there
wasn’t funding to cover their participation. Obviously, some would (and have, in years past… of their own accord and using their own funds). But if the 3rd party testing isn’t important to our customers it won’t be important to us either… and maybe
vice versa.

Another downside, related to the first one, is that CAT testing creates barriers. One is cost… another is technical capability. CAT testing isn’t an “easy pass”. Development shops that are new to eHealth and/or new to health informatics standards will
need to ramp up the learning curve before they are able to “do it right”… and if they don’t “do it right” they won’t pass at CAT. Notwithstanding that I think we should be leveraging our existing centres of excellence within the OpenHIE community to give
new OpenHIE participants a “leg up”, it is still a high bar and not everyone will get over it. This winnowing has its benefits, but it is less community-inclusive that favouring a low/no bar approach.

Anyway… before this is TLDR (sadly, maybe it already is!), I’ll leave it to others to carry on the pros/cons discussion – especially some who believe (on the con side) that CAT conformance shouldn’t be a prerequisite to being “OpenHIE-conformant”.

-Derek.

–

You received this message because you are subscribed to the Google Groups “OpenHIE Architecture” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
ohie-architecture+unsubscribe@googlegroups.com.

For more options, visit
https://groups.google.com/d/optout.

Carl_Fourie1 · July 23, 2015, 3:03pm

I think Paul and Shaun have raised a good point around what qualifies a tool to be part of OpenHIE; but I see that Derek’s question is really focusing at understanding how to qualify when/if a tool is conforming to OHIE connections and workflows. It struck me that we may be looking at a maturity matrix of tools for OpenHIE and a way to review new comers etc.

I’ve attached a very rough first draft of what I’ve read here and tried to put a few measures in. I think where Derek’s question comes in is what is OpenHIE’s role in evaluating connectathon testing? Are we going to do it ourselves? if so how and do we have the bandwidth to do that? Or do we rely on international groups like IHE (actually IHE) to do this for us? and what does that do to the teams getting started with tools?

I think we may have space for both in the model attached.

Just my 2c

Cheers

OHIE-Conformance Maturity Model.xlsx (9.07 KB)

···

On Thu, Jul 23, 2015 at 3:08 PM, Grannis, Shaun J sgrannis@regenstrief.org wrote:

I would also add a fourth dimension to conformance - that the stakeholder assents to the mission/vision/values of the OpenHIE community.

Shaun

From: ohie-architecture@googlegroups.com on behalf of Paul Biondich pbiondic@regenstrief.org

Date: Thursday, July 23, 2015 at 8:35 AM

To: “ohie-architecture@googlegroups.com” ohie-architecture@googlegroups.com

Subject: Re: what is the role of Connectathon testing?

Hi Derek, thanks for this note.

I think underneath your point is a more fundamental one that we need to address first as a group:

“When we judge ‘conformance’ with OpenHIE, what qualities do we want that conformance to have?”

From my perspective, those qualities should include a few things:

That the technical component has some measure of “quality”… that includes attributes like “secure”, “does what it purports to do”, and “has a well understood maintenance/further development trajectory which allows the software to be more functional
and heal itself over time”

That it complies with our integration testing process… i.e., “it can demonstrate the ability to interact with our architectural model in ways that we as a community formally specify”… for now that means: can support the workflows we agree to release
as a community, but could include other attributes over time

That it has the ability to be included as a part of a packaging/distribution strategy, and that the “owners” of that component will assent and ideally help us with that process.

These are just my opinions… I know that others might have different perspectives.

So my stance is to agree upon the high-level components of conformance, and then we collectively as a group can decide how we want to address each of these components, one by one.

From my perspective, your comments relate to #1, or could at least serve as a proxy for #1.

What are other people’s thoughts here?

-Paul

–

You received this message because you are subscribed to the Google Groups “OpenHIE Architecture” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
ohie-architecture+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

–

You received this message because you are subscribed to the Google Groups “OpenHIE Architecture” group.

To unsubscribe from this group and stop receiving emails from it, send an email to ohie-architecture+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Carl Fourie
Programs Development Manager

Jembi Health Systems | SOUTH AFRICA
Mobile: +27 71 540 4477 | Office: +27 21 701 0939 | Skype: carl.fourie17
E-mail: carl.fourie@jembi.org

Email Disclaimer:

This e-mail contains proprietary and confidential information some or all of which may be legally privileged. It is for the intended recipient only. If an addressing or transmission error has misdirected this e-mail, please notify the author by replying to this e-mail and then deleting same. If you are not the intended recipient you must not use, disclose, distribute, copy, print or rely on this e-mail. Jembi Health Systems NPO, its subsidiaries and associated companies is not liable for the security of information sent by e-mail and accepts no liability of whatsoever nature for any loss, damage or expense resulting, directly or indirectly, from the access of this e-mail or any attachments hereto.

On Wed, Jul 22, 2015 at 3:40 PM, Derek Ritz (ecGroup)
derek.ritz@ecgroupinc.com wrote:

Hi all.

I know we’ve broached this topic a few times on Architecture Community calls – but I think it will be useful to get our arms around
definitively answering an important question: what is the role, for OpenHIE, of Connectathon (CAT) testing?

Folks in the OpenHIE Architecture Community know I have a point of view on this topic. I believe that we have created important value through our CAT participation to date; our various software infrastructure “puzzle pieces” have established their conformance
to the IHE profiles that we’ve spec’d for OpenHIE. Usefully, CAT participation also has simplified the OpenHIE integration testing that we need to do. Because our components separately are tested at CAT, we didn’t need to replicate in our OpenHIE environment
the full array of tests that a component would need to pass if they were enrolled at CAT. Lastly, it is important (in my view) for OpenHIE’s credibility that the software pieces in our infrastructure enjoy the imprimatur of having passed the IHE CAT process.
CAT is an external, mature, well-recognized, well-governed process – and that is helpful both as a genuine risk mitigator and in terms of our “positioning” for OpenHIE. I’d go so far as to suggest we should help our potential “customers” (implementing jurisdictions)
appreciate the importance of the fact that the OpenHIE infrastructure is standards-based and independently conformance-tested.

What are the downsides? Clearly, one of them is the cost and effort needed to attend and pass at CAT. We have enjoyed IHE scholarships the last couple of years that covered the entry fees, but there are still other costs that must be covered. It would
be interesting to ask whether our OpenHIE reference implementation software developers would seek CAT conformance statements if there
wasn’t funding to cover their participation. Obviously, some would (and have, in years past… of their own accord and using their own funds). But if the 3rd party testing isn’t important to our customers it won’t be important to us either… and maybe
vice versa.

Another downside, related to the first one, is that CAT testing creates barriers. One is cost… another is technical capability. CAT testing isn’t an “easy pass”. Development shops that are new to eHealth and/or new to health informatics standards will
need to ramp up the learning curve before they are able to “do it right”… and if they don’t “do it right” they won’t pass at CAT. Notwithstanding that I think we should be leveraging our existing centres of excellence within the OpenHIE community to give
new OpenHIE participants a “leg up”, it is still a high bar and not everyone will get over it. This winnowing has its benefits, but it is less community-inclusive that favouring a low/no bar approach.

Anyway… before this is TLDR (sadly, maybe it already is!), I’ll leave it to others to carry on the pros/cons discussion – especially some who believe (on the con side) that CAT conformance shouldn’t be a prerequisite to being “OpenHIE-conformant”.

-Derek.

–

You received this message because you are subscribed to the Google Groups “OpenHIE Architecture” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
ohie-architecture+unsubscribe@googlegroups.com.

For more options, visit
https://groups.google.com/d/optout.