Perhaps an approach we could take would be for the HIM to provide a mechanism (or incorporate a tool) that allows metrics to be reported from the registries and itself. This could give a single view into the performance of the HIE.
···
On Wed, Feb 12, 2014 at 2:07 AM, Eduardo Jezierski edjez@instedd.org wrote:
+1
On Feb 11, 2014, at 1:35 PM, Bob Jolliffe bobjolliffe@gmail.com wrote:
Yes MOH has munin running on one of the dhis servers in Rwanda as well (in fact the postgres db server).
Having such monitoring service available as a “standard” box somewhere in the enterprise could certainly be part of a “best practice” picture. Not necessarily munin of course, but a placeholder for a couple of options.
–
Ryan Crichton
Software Developer, Jembi Health Systems | SOUTH AFRICA
Mobile: +27845829934 | Skype: ryan.graham.crichton
E-mail: ryan@jembi.org
On 11 February 2014 18:31, Eduardo Jezierski edjez@instedd.org wrote:
Agree
For CPU, uptime, free disk space, and the likes we have been super happy with Munin.
For example; you can see our stats for one of our mobile routing systems here
http://munin.instedd.org/instedd.org/nuntium.instedd.org/index.html
you can see super detailed things but also important things like uptime days etc there. The services contributing data need not live in the same physical subnet/environment.
On Feb 11, 2014, at 10:04 AM, Bob Jolliffe bobjolliffe@gmail.com wrote:
–
You received this message because you are subscribed to the Google Groups “OpenHIE Architecture” group.
To unsubscribe from this group and stop receiving emails from it, send an email to ohie-architecture+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
On 11 February 2014 17:45, Eduardo Jezierski edjez@instedd.org wrote:
Thanks Carl these are great.
Stepping back and looking at this thread, seing OHIE a toolkit for an integrated architecture, I would expect coherent treatment of crosscutting themes - of course security related aspects and management as well as infrastructure management and other governance topics.
By coherent I mean that there are sound principles and guidelines outlined for each of these crosscutting areas; and clear identification of standards & IHE profiles where needed; so that each sub-community can prioritize and address ‘blending in’ in appropriate ways. Furthermore; the capacity building in country around these topics is important and part of having an OHIE implementation is understanding the capabilities and contextualizing them to local needs. A CMM / Zachmannish approach can be used as a guideline to produce some artifacts when we hear countries demand them.
Specifically to the montoring thread; Suranga also saw how we use NewRelic as well as other tools to track & trace & visualize activities spanning multiple services. It’s helped us a lot - our systems are mostly skins over collections of web services that work alongside each other or are stacked up; so seeing what happened where and when caused by a single activity that meanders its way through them has been very useful.
Ah I see that this is really yet another level of monitoring. Not really to do with cpu cycles and disk utilization at all. More about collecting/processing information about how the system is being used at the level of web interactions.
~ ej
On Feb 11, 2014, at 8:14 AM, Carl Leitner litlfred@gmail.com wrote:
Hi All,
Here is a work-item for the ITI committee for this year that may be relevant to the overall discussion. It allows some basic querying of ATNA logs.
ftp://ftp.ihe.net/IT_Infrastructure/iheitiyr12-2014-2015/Technical_Cmte/Detailed-Proposals/IHE_Profile_ATNA_federation_detailed.docx
Cheers,
-carl
On Feb 11, 2014, at 11:05 AM, Bob Jolliffe bobjolliffe@gmail.com wrote:
widely supported network and system monitoring tools (like zabbix) also typically provide bridges to jmx as well as more established standards like snmp.
I think that such network monitoring tools should really be part of the essential infrastructure upon which architectural artefacts like openHIE should sit. Remote tools are useful for monitoring availability but I am not sure if they replace this traditional need.
+1 to central logging. And you can of course also aggregate them after the fact - http://logio.org/ for example looks like a very good tool for this sort of thing.
Clinical indicators are a different kettle of fish, with different tools and mechanisms - such as those we might conceive in the hmis box. Useful things but aimed at different layers of management.
Bob
–
You received this message because you are subscribed to the Google Groups “OpenHIE Architecture” group.
To unsubscribe from this group and stop receiving emails from it, send an email to ohie-architecture+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
–
You received this message because you are subscribed to the Google Groups “OpenHIE Architecture” group.
To unsubscribe from this group and stop receiving emails from it, send an email to ohie-architecture+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
On 11 February 2014 14:55, Odysseas Pentakalos, Ph.D. odysseas@sysnetint.com wrote:
Here are a couple of contributions from the Client Registry
perspective and how we are thinking of supporting these
requirements in the future.
Within the Java world there is a standard technology called JMX
(for Java Management Extensions) that provides the means to add
standards-based
monitoring and management capabilities to an application while
isolating the aspect of instrumenting an application from the
aspect of collecting
metrics and presenting them to interested users. The standard
defines a mechanism for instrumenting an application and by
following this
mechanism, end-users can use tools that comply with this standard
to collect and view metrics. Many monitoring tools are available
and quite
a few of them are free so by utilizing this standard for
instrumentation we can automatically make use of the end-user
tools.
The standard is fairly abstract so it can be used to collect
metrics across the spectrum described by Shaun and r.friedman. It
is up to the application to
expose the appropriate metrics whether they are technical in
nature, such as the number of connections, number of threads, CPU
utilization, etc. or
functional, such as the total number of patient registration
messages, number of unique patients in the client registry, and so
on.
This standard is widely adopted within the Java world with most
application servers and enterprise applications already supporting
it. The ESB that
we utilized for the RHEA deployment itself supports the JMX
standard and makes many management capabilities available through
it. Being though
that it is a Java standard, I am not sure to what extend this
would be useful to all the registries. There are now libraries
that support this standard
within the Python and .NET platforms.
Another useful technical solution that we can consider is Apache
Flume. The first paragraph from their site describes it very well
so I have copied it
down below:
Flume is a distributed, reliable, and available service for
efficiently collecting, aggregating, and moving large amounts of
log
data
It is open source and will make it easy for us to aggregate log
files from multiple applications in one place for root-cause
analysis or for collecting metrics
for reporting purposes. The Client Registry is moving towards a
distributed architecture for large deployments and even within
this single registry it
will be very useful to be able to aggregate in an automated
approach the logs from multiple servers in one place.
Best regards,
Odysseas
On 02/11/2014 07:13 AM, Ryan Crichton wrote:
Hi Shaun,
I agree, limiting scope sounds best. Perhaps, we could put
place holder pages on the wiki for each of these types of
motoring where people could dump thoughts and requirements
that they would like to see. Maybe this would allow us to get
started.
Cheers,
Ryan
–
You received this message because you are subscribed to the Google
Groups “OpenHIE Architecture” group.
To unsubscribe from this group and stop receiving emails from it,
send an email to ohie-architecture+unsubscribe@googlegroups.com.
For more options, visit [https://groups.google.com/groups/opt_out](https://groups.google.com/groups/opt_out).
-- Odysseas Pentakalos, Ph.D., PMP
Chief Technology Officer
SYSNET International, Inc.
2930 Oak Shadow Drive
Oak Hill, Virginia 20171
mailto:odysseas@sysnetint.com
(703) 855-2029
–
You received this message because you are subscribed to the Google Groups “OpenHIE Architecture” group.
To unsubscribe from this group and stop receiving emails from it, send an email to ohie-architecture+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
On Tue, Feb 11, 2014 at 1:11 AM, r.friedman@mindspring.com
wrote:
I agree with Shaun. But
there do seem to be two layers. The most
“technical” layer is monitoring connections,
transaction rate, response time; the monitored
values are compared against service levels. The
second layer is more content oriented – the number
of encounters per visit type per time unit should
stay relatively constant; the number of encounters
per provider per time unit should stay relatively
constant; the number of common diagnoses or
prescriptions for common drugs or orders for common
lab tests at a facility per unit of time should stay
relatively constant or be seasonal; the number of
patient creation events per time unit should taper
off from initial values when the site came on line.
Anomalies at this level may indicate
software/hardware malfunctions or user/site
problems. This second layer requires anomaly
detection algorithms and has many similarities to
epidemic detection. And the distance from this
second layer to performance indicators is small.
-----Original Message-----
From: Shaun Grannis
Sent: Feb 10, 2014 5:05 PM
To: ohie-architecture@googlegroups.com
Cc: Eduardo Jezierski , Suranga Kasthurirathne
Subject: Re: Discussion on OpenHIE Monitoring and
Evaluation design
This is a great conversation, to which I
would offer a few thoughts.
Recall that there seems to be two distinct
ends of the “monitoring” spectrum: at one end of
the spectrum “monitoring” relates to tracking
systems to ensure they’re functioning as
expected (‘technical’ monitoring); the other end
of the spectrum is “monitoring” as it pertains
to “monitoring and evaluation” (“M&E”) -
assessing the impact of systems on clinical and
population health (‘clinical’ monitoring). These
two senses of the word “monitoring” seem to have
clear distinctions, even though they might use
overlapping technologies or systems in some
parts.
To make early and significant gains, I wonder
if it would make sense to begin the “OpenHIE
monitoring” discussions with well defined and
limited initial scope. I would prefer we begin
with an initial scope that focuses primarily on
the first sense, “technical” monitoring of the
system’s function rather than trying to also
digest the more classical clinical M&E
sense.
Does that sound like a reasonable way to
begin this work?
Best,
Shaun
–
You received this message because you are subscribed
to the Google Groups “OpenHIE Architecture” group.
To unsubscribe from this group and stop receiving
emails from it, send an email to ohie-architecture+unsubscribe@googlegroups.com.
For more options, visit [https://groups.google.com/groups/opt_out](https://groups.google.com/groups/opt_out).
–
You received this message because you are subscribed to
the Google Groups “OpenHIE Architecture” group.
To unsubscribe from this group and stop receiving emails
from it, send an email to ohie-architecture+unsubscribe@googlegroups.com.
For more options, visit [https://groups.google.com/groups/opt_out](https://groups.google.com/groups/opt_out).
–
Ryan
Crichton
Software
Developer, Jembi Health Systems | SOUTH
AFRICA
Mobile:
+27845829934 | Skype: ryan.graham.crichton
E-mail: ryan@jembi.org
Shaun J. Grannis, MD MS FACMI FAAFP
Biomedical Research Scientist, The
Regenstrief Institute
Associate Professor, I.U. School of Medicine
410 West 10th Street, Suite 2000
Indianapolis, IN 46202
(317)
274-9092 (Office)
(317)
274-9305 (Fax)
On Thu, Feb 6, 2014 at
5:19 PM, Suranga Kasthurirathne surangakas@gmail.com
wrote:
Hi,
Thanks Ed, I did have a good
conversation with Mario on this matter.
Unfortunately, we couldn't actually
look at the NewRelic demo in action… i’m
also interested in Poirot, which we may be
able to use, at least in terms of learning
how/what to do.
I'm definitely sure that RHIE would
benefit from adopting NewRelic for their
FR. Mario thinks that we may be able to
use a free account, which is what i’m also
rooting for. I will try to raise this
matter with the RHIE team, and see their
response.
On a separate tangent, it might be
useful to find some time on a call,
especially to discuss our
boundaries/scope, as @Roger pointed out.
Best regards,
Suranga
--
You received this message because you are
subscribed to the Google Groups “OpenHIE
Architecture” group.
To unsubscribe from this group and stop
receiving emails from it, send an email to
ohie-architecture+unsubscribe@googlegroups.com.
For more options, visit [https://groups.google.com/groups/opt_out](https://groups.google.com/groups/opt_out).
On Thu, Feb 6, 2014
at 3:20 PM, Eduardo Jezierski edjez@instedd.org
wrote:
Thanks
Suranga.
Please feel free to ask more
about poirot and NewRelic; we’ll be
glad to explain to whoever is
appropriate in OHIE how these tools
we have been using (not building)
allows us to see how things are
working in multi-service
environments, maintain health server
installations, etc…
I think Martin demo'd some stuff
while he was still around. If
NewRelic is an option you all want
to look into I’d love to do
introductions to explore CSR
subsidies or pricing - we know the
team well and it may be an excellent
ROI for MOHs who don’t have large or
mature IT teams and operations. We
use a lot of advanced features but
even without these they may find it
beneficial.
~
ej
On Feb 6, 2014, at
2:13 PM, Suranga
Kasthurirathne <surangakas@gmail.com >
wrote:
Hi,
During
Wednesday’s
session of the
OpenHIE
architecture
meeting we were
able to briefly
discuss OpenHIE
monitoring and
evaluation
methods.
Since a number
of people were
unable to attend
this session, I
thought that i’d
list down our
observations so
that we could
discuss / decide
on a common path
forward.
So basically,
1) We need to
determine
boundaries for
Monitoring and
Evaluation, and
what they
represent.
2) Monitoring
and evaluation
includes,
-
Server
performance /
up time
(hardware /
software
issues)
-
Clinical
evaluation
(etc. number
of HIV
positive
patients not
receiving
appropriate
treatment)
3) Standards
base : Our
approach should
adhere to
standards wherever
possible
(question: what
standards are
available/usable
?)
4) Adaptability
/ re-usability
(can we re-use ?
Note : Can we
re-use any of the
monitoring tools
Instedd already
uses / is building
for certain tasks
? etc. newRelic
and Poirot
5) Need to
evaluate adherence
to guideline based
care
One of many
potential
solutions
discussed were to
centralize logging
/ monitoring for
all components
within the HIM.
This implies that
the results / log
for what each
transaction
undergoes at each
component is
transferred back
to the HIM, and
stored there.
On the plus
side, this makes
debugging very
easy because each
transaction can be
seen as a single
unit. On the
negative side,
moving heavy logs
back to the HIM
requires even more
bandwidth usage,
which may not work
well 
I just wanted
to share this here
and make some
waves so that we
could schedule
some more time to
discuss this at
length later.
Best regards,
Suranga
–
Best Regards,
Suranga
–
You received this
message because you are
subscribed to the Google
Groups “OpenHIE
Architecture” group.
To unsubscribe from this
group and stop receiving
emails from it, send an
email to ohie-architecture+unsubscribe@googlegroups.com.
For more options, visit
https://groups.google.com/groups/opt_out.
–
You received this message because
you are subscribed to the Google
Groups “OpenHIE Architecture” group.
To unsubscribe from this group and
stop receiving emails from it, send
an email to ohie-architecture+unsubscribe@googlegroups.com.
For more options, visit [https://groups.google.com/groups/opt_out](https://groups.google.com/groups/opt_out).
Suranga
–
Best Regards,