National Research Network for EHR Audit Log Data

The National Research Network for EHR Audit Log Data is focused on use of EHR Audit-log and Meta-data to support health services research. Kicking off on March 16, 2018, a group of researchers convened to share their experiences and ideas, learn from each other, and pursue collaborative projects. 

The monthly NRN webinars are focused on these three topics:

• Research Topics and Funding (types of research questions well suited to audit log data, funding sources for audit log research)
• Research Methodologies and Skillsets (methodological/modeling approaches well suited to audit log data, skillsets of research team)
• Data Quality and Creating Meaningful Measures (accuracy, usefulness of data documentation, various levels of granularity, strategies for reducing noise/errors)

NRN Workgroups

In early 2019, the network convened 3 workgroups in response to the substantial heterogeneity in approaches to tackling similar issues and the many examples of different researchers and sites replicating the same foundational work. Each of the workgroups take on a different domain in which developing common methods and measures along with methodological guidance would be valuable. The workgroups include:

Developing Measures Relevant to Audit Log Work

The audit log measures workgroup is addressing the lack of standardization for timings derived from audit logs.  The group includes researchers from diverse institutions including universities, healthcare centers, government, and industry. The goal of this workgroup is to produce a catalog of measures with definitions, validations, and applications across different medical settings, EHR vendors and EHR implementations.  We will be defining measures already included in the literature as well as creating definitions of new measures. Limitations and challenges for defining these measures will be documented as well as recommendations for improving their accuracy and reproducibility.

Sign up here if you are interested in joining or hearing more about this workgroup.

The NRN Measures Workgroup curates and maintains a public database of peer-reviewed publications that use EHR audit log data: the EHR Audit Log Evidence Repository (EALER). This database includes the papers presented in the monthly webinars, publications of NRN members, and other research utilizing audit logs or similar data sources derived from EHR metadata. Please use the below links to submit recommendations or view the repository. Feel free to direct any inquiries about this effort to Nate C. Apathy, PhD at [email protected].

Submit a paper to the repository

View the repository

Developing a Framework to Characterize Task Types

Task types workgroup has 19 members coming from 14 different institutions including academic and industrial medical centers, universities and EHR vendors. The goal of the workgroup is to develop a common framework incorporating informatics approaches and knowledge of clinical experts to learn task types which will be leveraged to characterize common clinical activities occurring in a variety of EHR systems. The anticipated output would be a task type catalog, within which, each task type has its own definition, affiliated clinical setting and a list of corresponding access actions that are commonly available in EHR log data.

Developing Standards to Support Use of Audit Log Data to Characterize a Specific Use Case

The goal of this workgroup is to identify and leverage existing/emerging models, such as FHIR resources, to support log data standards-development around a use-case. The selected use-case is, “Characterizing patterns of EHR use across clinical departments and/or types of staff”. The group will discuss resources needed to address this use-case (e.g., Audit Events, Tasks, Practitioners), describe the metrics used with those resources (e.g., Login/Logout), address differences discovered in log data capture and meaning across systems (e.g., whether timestamps associated with Login/Logout represent the same event, such as a user-submit or an authentication), and propose the standards to address the use-case. The group will also describe limitations and challenges to developing standards across these resources and events, as well as future work needed to scale and extend them to additional use-cases.

Upcoming Presentations

Effects of a QI Intervention on Pediatric Asthma Treatment in Pediatric Emergency Departments
January 20, 2023: Mustafa Ozkaynak, University of Colorado

Comparison of audit-log-derived clinical workload and reimbursement in anesthesiology
February 3, 2023: Sunny Lou, Washington University St. Louis