National Research Network for EHR Audit Log Data



Members from over 50 organizations


The National Research Network for EHR Audit Log Data is focused on use of EHR Audit-log and Meta-data to support health services research. Kicking off on March 16, 2018, a group of researchers convened to share their experiences and ideas, learn from each other, and pursue collaborative projects. 

The monthly NRN webinars are focused on these three topics:

  • Research Topics and Funding (types of research questions well suited to audit log data, funding sources for audit log research)
  • Research Methodologies and Skillsets (methodological/modeling approaches well suited to audit log data, skillsets of research team)
  • Data Quality and Creating Meaningful Measures (accuracy, usefulness of data documentation, various levels of granularity, strategies for reducing noise/errors)

NRN Workgroups

In early 2019, the network convened 3 workgroups in response to the substantial heterogeneity in approaches to tackling similar issues and the many examples of different researchers and sites replicating the same foundational work. Each of the workgroups take on a different domain in which developing common methods and measures along with methodological guidance would be valuable. The workgroups include:

Developing Measures Relevant to Audit Log Work

The audit log measures workgroup is addressing the lack of standardization for timings derived from audit logs.  The group includes researchers from diverse institutions including universities, healthcare centers, government, and industry. The goal of this workgroup is to produce a catalog of measures with definitions, validations, and applications across different medical settings, EHR vendors and EHR implementations.  We will be defining measures already included in the literature as well as creating definitions of new measures. Limitations and challenges for defining these measures will be documented as well as recommendations for improving their accuracy and reproducibility.

Sign up here if you are interested in joining or hearing more about this workgroup.

The NRN Measures Workgroup curates and maintains a public database of peer-reviewed publications that use EHR audit log data: the EHR Audit Log Evidence Repository (EALER). This database includes the papers presented in the monthly webinars, publications of NRN members, and other research utilizing audit logs or similar data sources derived from EHR metadata. Please use the below links to submit recommendations or view the repository. Feel free to direct any inquiries about this effort to Nate C. Apathy, PhD at [email protected].

Submit a paper to the repository

View the repository

Developing a Framework to Characterize Task Types

Task types workgroup has 19 members coming from 14 different institutions including academic and industrial medical centers, universities and EHR vendors. The goal of the workgroup is to develop a common framework incorporating informatics approaches and knowledge of clinical experts to learn task types which will be leveraged to characterize common clinical activities occurring in a variety of EHR systems. The anticipated output would be a task type catalog, within which, each task type has its own definition, affiliated clinical setting and a list of corresponding access actions that are commonly available in EHR log data.

Developing Standards to Support Use of Audit Log Data to Characterize a Specific Use Case

The goal of this workgroup is to identify and leverage existing/emerging models, such as FHIR resources, to support log data standards-development around a use-case. The selected use-case is, “Characterizing patterns of EHR use across clinical departments and/or types of staff”. The group will discuss resources needed to address this use-case (e.g., Audit Events, Tasks, Practitioners), describe the metrics used with those resources (e.g., Login/Logout), address differences discovered in log data capture and meaning across systems (e.g., whether timestamps associated with Login/Logout represent the same event, such as a user-submit or an authentication), and propose the standards to address the use-case. The group will also describe limitations and challenges to developing standards across these resources and events, as well as future work needed to scale and extend them to additional use-cases.

Upcoming Presentations

Effects of a QI Intervention on Pediatric Asthma Treatment in Pediatric Emergency Departments
January 20, 2023: Mustafa Ozkaynak, University of Colorado

Comparison of audit-log-derived clinical workload and reimbursement in anesthesiology
February 3, 2023: Sunny Lou, Washington University St. Louis

Past Presentations

Empowering Data-Driven Trainee Improvement: Interactions with the Electronic Health Record
Feb 8, 2019: Adam Dziorny and Mark Mai 

Recognition of Hospital Work System Strain through Knowledge Elicitation, Multi-Source Data Integration and Analysis
March 8, 2019: Dana Womack

Imaging Order Dwell Time
April 12, 2019: Nicholas Riley 

Leveraging EHR Audit Logs to Discover Effective Care Coordination Practice Patterns
May 10, 2019: You Chen 

NRN Workgroup Leads Report out on Workgroup Progress and AMIA Presentations
September 13, 2019

ASPE Burden Measurement Project
October 10, 2019: Genna Cohen, Llew Brown and Megan Fitzgerald of Mathematica

Monitoring EMR Accesses for Inappropriate Use
January 17th, 2020: Daniel Fabbri 

Documentation and Consumption of Social Needs Data in the EHR: Early Findings, Challenges, and Opportunities
February 28th, 2020: Michael WangUCSF

Characterizing Styles of Clinical Note Production
April 24th, 2020: Jen Gong, UCSF

Derivation of ambulatory physician normalized electronic health record use across two healthcare systems using vendor audit log data from different vendor products: A cross-sectional analysis
October 16th, 2020: Shawn Ong and Ted Melnick

Assessing Single Sign-On Authentication Behaviors to Inform Customized Solutions Using Audit Log Data – While Maintaining Privacy and Promoting Standards
November 13, 2020: Jason Thomas, University of Washington

Automated Measurement of Resident Work Hours
December 4, 2020: Hossein Soleimani, PhD and Sara Murray, MD, UCSF

Physicians' Electronic Inbox Work Patterns and Factors Associated With High Inbox Work Duration
Jan 22, 2021: Fatema Akbar, PhD Candidate, University of California, Irvine

Pediatric ICU Electronic Health Record Usage as Measured by Audit Logs and Screen Recordings
Feb 19, 2021: Amrita Sinha, Stanford University

Evaluation of Attention Switching and Duration of Electronic Inbox Work Among Primary Care Physicians
March 19, 2021: Mary Reed and Margaret Wharton, Kaiser Permanente

Mining Tasks and Task Characteristics from Electronic Health Record Audit Logs with Unsupervised Machine Learning
April 23, 2021: Bob Chen and You Chen, Vanderbilt University

E/M Guideline Changes and Physician Documentation Burden
June 25, 2021: Nate Apathy, University of Pennsylvania and Dori Cross, University of Minnesota

Barriers and Challenges in Making Clinical Sense of Audit Log Data
August 27, 2021: Deevakar Rogith, UT Health

Use of EHR Audit Log Metrics to decipher contributions of EHR vs. Local Work Culture to Clinician Burnout
October 15, 2021: Genie McPeek Hinz, Duke University

Predicting Next Day Discharge via EHR Access Logs
November 19, 2021: Xinmeng Zhang and Chao Yan, Vanderbilt University

Examining Physician Metrics Before and After Working with Virtual Scribes
December 10, 2021: Shawn Ong and Ted Melnick, Yale University

Documentation Dynamics: Note composition and provider efficiency
January 21, 2022: Nate Apathy, University of Pennsylvania

Using Audit Log Data to Advance Diagnostic Excellence
February 11, 2022: Julia Adler-Milstein, UCSF

Audit logs for the measurement of physician workload, cognitive burden, and burnout
March 11, 2022: Sunny Lou and Thomas Kannampallil, Washington University in St. Louis

Accounting for Variation in EHR Use Within vs. Across Provider Organizations
April 29, 2022: Dori Cross, University of Minnesota

Are Longer Visits Better? Evidence from Primary Care Prescribing
May 20, 2022: Hannah Neprash, University of Minnesota

Using the EHR to understand patterns and quality of ambulatory care delivery
July 22, 2022: Lisa Rotenstein, Harvard University

Opportunities and Challenges Integrating Patient Scheduled Hours into Measures of EHR Time
September 30, 2022: Natalya Maisel, UCSF

Transforming Healthcare from the Outside: the OBSERVER Project
October 28, 2022: Kevin Johnson and Eric Eaton, University of Pennsylvania

Differences in Physician EHR Use Between Virtual and In-Person Care Delivery
November 18, 2022: A Jay Holmgren, UCSF