Scientific publications describing what we learned is one of the important outputs for SafePAT.
Here’s a list of published abstracts and articles describing what we have been working on.


Going the Extra Mile. Cross-border Patient Handover in a European Border Region from a Healthcare Professional Perspective.

Beuken, J.A., Verstegen D.M.L., Dolmans, D.H.J.M., Van Kersbergen, L.J.M., Losfeld, X. , Sopka, S. , Vogt, L. and Bouwmans, M.E.J.


Background Cross-border healthcare is complex, increasingly frequent and causes potential risks for patient safety. In this context, cross-border handovers or the transfer of patients from one country to another deserves particular attention. Although general handover has been the topic of extensive research, little is known about the challenges of handover across national borders, especially as perceived by stakeholders. In this study, we aimed to gain insight into healthcare professionals’ perspectives on cross-border handover and ways to support this.

Methods We conducted semistructured interviews with healthcare professionals (physicians, nurses, paramedics and administrative staff) in a European border region to investigate their perspectives on cross-border handover. The interviews were aimed to investigate settings of acute and planned handover. Informed by the theory of planned behaviour (TPB), interviews focused on participant perspectives. We summarised all interviews and inductively identified healthcare professionals’ perspectives. We used elements of the TPB as sensitising concepts.

Results Forty-three healthcare professionals participated. Although respondents had neutral to positive attitudes, they often did not know very well what was expected of them or what influence they could have on improving cross-border handover. Challenges covered five themes: information transferlanguage barrierstask division and educationpolicy and financial structures and cultural differences. To overcome these challenges, we proposed strategies such as providing tools and protocols, discussing and formalizing collaboration, and organising opportunities to meet and get to know each other.

Conclusion Healthcare professionals involved in cross-border handovers face specific challenges. It is necessary to take measures to come to a shared understanding while paying special attention to the above-mentioned challenges. Meeting in person around meaningful activities (eg, training and case discussions) can facilitate sharing ideas and community building.

Full text:


Beuken JA, Verstegen DML, Dolmans DHJM, et al. Going the extra mile — cross-border patient handover in a European border region: qualitative study of healthcare professionals’ perspectives. BMJ Quality & Safety

Impact of a blended curriculum on the quality of handovers: a quality improvement project.

Losfeld X., Istas L., Schoonvaere Q., Vergnion M., Bergs J.


Context and objective The negative consequences of inadequate nursing handovers on patient safety are broadly acknowledged, both within the literature as in practice. Evidence regarding strategies to improve nursing handover is however lacking. This study investigates the effect of a tailored, blended, curriculum on professional communication nurses’ perception of handover quality.

Methods We used a pre-test/post-test design within four units of a Belgian general hospital. Nurses (n = 120) from Maternal Intensive Care Unit, two rehabilitation wards, and geriatrics completed an interactive e-learning and a face-to-face session. We used the Handover Evaluation Scale (HES) to evaluate nurses’ perception of handover quality before and after curriculum completion. We used Confirmatory Factor Analysis (CFA) to assess validity of the HES.

Results Perception of nurses in Relevance of information showed a significant improvement (M = 53.19 ± 18.93 vs. M = 61.03 ± 19.75; U = 1274, P = 0.04). In a significant order, nurses also feel to be saving time on handovers after the intervention completion (M = 4.05 ± 1.47 vs M = 5.16 ± 1.31, p = 0.01).

Conclusion The applied intervention resulted in an improved awareness on the importance of Relevance of information during handovers. Besides, a significantly number of nurses thought that handovers took place in a timely fashion after the intervention. We are aware that the educational intervention is only the first step to achieve the long-term implementation of a culture of professional communication based on mutual support. Further research, using insight from implementation science, is needed.


Losfeld X., Istas L., Schoonvaere Q., Vergnion M., Bergs J.Losfeld X., Istas L. Impact of a blended curriculum on the quality of handovers: a quality improvement project. Submitted manuscript on BMJ Open Quality, 2020, May 20th .

A Comprehensive Literature Overview on Clinical Handover: A Systematic Review of Systematic Reviews

Melissa Desmedt, Dorien Ulenaers, Joep Grosemans, Johan Hellins, Jochen Bergs


Background In recent years, the interest in patient safety has picked up momentum among scholars, healthcare professionals and policy makers. A process central to the delivery of high-quality and safe care is handover. Handover has been recognised internationally as a high-risk area for patient safety and the call for interventions to improve the handover process has increased. Aim To appraise and summarize existing literature on this clinical handover.

Method This paper describes a systematic review of systematic reviews on clinical handover. We followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analysis) guidelines for identifying, screening, selecting, and including papers.

Results First, research indicates that poor handover is associated with multiple potential hazards such as required equipment for patients not readily available, information omissions, diagnosis errors, treatment errors, disposition errors, treatment delays, … Second, our systematic review indicates that no single tool arises as best for any particular speciality or use to evaluate the handover process. Third, there is little empirical evidence delineating what constitutes best handoff practices. Most efforts are primarily aimed at facilitating the coordination of care and communication between healthcare professionals by using electronic tools or a standardized form. Fourth, our systematic review indicates that the principal teaching method are role-play and simulation; which may result in better knowledge transfer to the work environment, better health and well-being of patients, …

Conclusion This review increases the emphasis on the importance of staff education (including simulation-based and team training), non-technical skills, and the implementation process. In addition, structured handovers and multidisciplinary education may improve information exchange and encourage teamwork, but further high-quality research is necessary.


Desmedt M., Ulenaers D., Grosemans J., Hellins J, Bergs J. A Comprehensive Literature Overview on Clinical Handover: A Systematic Review of Systematic Reviews. preprint. 2020:1-15.

Psychometric Properties of the Dutch Version of the Health Professional Education in Patient Safety Survey (HPEPSSDutch)

Jochen Bergs, Isabel Kortleven, Katrien Peeters, Melissa Desmedt, Dorien Ulenaers, Sarah Creemers, Ward Schrooten


Aim This study sought to validate the Dutch version of the Health Professional Education in Patient Safety Survey (H-PEPSSDutch), an instrument used to assess self-efficacy of health profession students regarding patient safety competence.

Method The H-PEPSSDutch was administered to a sample of 610 nursing students in two Belgian nursing schools. The psychometric properties were examined using confirmatory factor analysis, both for classroom and clinical learning. Cronbach’s alpha was used to ascertain the internal consistency reliability. Convergent and discriminant validity of the models were also tested.

Results Data were collected from 484 students. The analysis of construct validity showed good fit to the hypothesized models. The analysis yielded satisfactory factor loadings, suggesting that the H-PEPSSDutch reflects the constructs of the underlying six domains of the CPSI safety competencies framework. Internal consistency for the six constructs in terms of Cronbach’s alpha values ranged from 0.70 to 0.87 for classroom learning and from 0.56 to 0.86 for clinical learning, indicating good reliability. Discriminant validity was not supported for all dimensions. Especially, the model regarding learning in the clinical setting, which showed difficulties in differentiating between the H-PEPSS constructs.

Conclusion Overall, this provides us with a reliable instrument to measure self-reported patient safety competence among nursing students. Further research is needed to validate the H-PEPPS as a longitudinal monitoring tool and as a pre-and-post measurement on the impact of interventions related to patient safety in the nursing curricula.


Bergs J., Kortleven I., Peeters K., Desmedt M., Ulenaers D., Creemers S., Schrooten W. Psychometric Properties of Dutch Version of the Health Professional Education in Patient Safety Survey (H-PEPSSDutch). preprint. 2019:1-5.

Detecting Mistakes in CPR Training with Multimodal Data and Neural Networks

Di Mitri D, Schneider J, Specht M, Drachsler H

This study investigated to what extent multimodal data can be used to detect mistakes during Cardiopulmonary Resuscitation (CPR) training. We complemented the Laerdal QCPR ResusciAnne manikin with the Multimodal Tutor for CPR, a multi-sensor system consisting of a Microsoft Kinect for tracking body position and a Myo armband for collecting electromyogram information. We collected multimodal data from 11 medical students, each of them performing two sessions of two-minute chest compressions (CCs). We gathered in total 5254 CCs that were all labelled according to five performance indicators, corresponding to common CPR training mistakes. Three out of five indicators, CC rate, CC depth and CC release, were assessed automatically by the ReusciAnne manikin. The remaining two, related to arms and body position, were annotated manually by the research team. We trained five neural networks for classifying each of the five indicators. The results of the experiment show that multimodal data can provide accurate mistake detection as compared to the ResusciAnne manikin baseline. We also show that the Multimodal Tutor for CPR can detect additional CPR training mistakes such as the correct use of arms and body weight. Thus far, these mistakes were identified only by human instructors. Finally, to investigate user feedback in the future implementations of the Multimodal Tutor for CPR, we conducted a questionnaire to collect valuable feedback aspects of CPR training. Full text available on:


Di Mitri, D.; Schneider, J.; Specht, M.; Drachsler, H. (2019) Detecting Mistakes in CPR Training with Multimodal Data and Neural Networks. Sensors 2019, 17, 3099.

Implémentation du SBAR à l’hôpital : au-delà d’un curriculum mixte.

CHR Citadelle, Liège (Belgique)


Contexte – Le Centre Hospitalier Régional (CHR) Citadelle (Liège, Belgique) est en phase d’accréditation par le Joint Commission International (JCI). En janvier 2018, une équipe de quatre référents a bénéficié d’une formation d’Instructeurs TeamSTEPPS. Notre institution a retenu l’optimalisation de la communication
professionnelle, pilier transversal du travail en équipe, comme le premier projet basé sur les outils proposés. Un programme mixte de formation n’est que la première étape pour améliorer les comportements visant la sécurité du patient. Les enjeux réels résident dans les stratégies pour une implémentation efficace et durable.

Objectifs – Le premier objectif de l’étude que nous menons (Cf. annexe) est d’évaluer l’impact perçu d’un programme mixte d’enseignement et de mise en oeuvre de l’outil SBAR au sein des départements infirmiers et (para)médicaux du CHR Citadelle Liège.

Intervention(s) réalisée(s) – Dans une phase test, nous avons formé et comparé des groupes équivalents issus du service des urgences, en gériatrie, dans deux salles de revalidation et en soins intensifs de gynécologie/obstétrique (n=136). Avant le déploiement institutionnel en février 2019 (n≈1200), des managers, des leaders, des formateurs, des coaches et des utilisateurs finaux seront formés (n=278).
Méthodes d’évaluation – La première phase de notre étude s’appuie sur une Handover Evaluation Scale (HES) adaptée. La deuxième phase s’appuiera sur la validation d’un indicateur qualité observationnel des transferts de charge.

Résultats – Les résultats seront présentés à l’occasion du Deuxième Forum Francophone TeamSTEPPS (7 mars 2019).

Leçons retenues – Nous proposons d’ores et déjà une stratégie à plusieurs niveaux ciblant en première intention des départements ou spécialités spécifiques. Il s’agit d’une approche pragmatique vers une nouvelle culture de la communication dans les établissements de santé. Il semble obligatoire de créer un processus de développement du leadership ayant une incidence sur tous les niveaux de l’organisation.

Impact of digital handover checklists on clinical outcome parameters in ICU patients: a randomized controlled pilot trial. Digital handover checklists: what is the influence on clinical outcome parameters in ICU patients? A randomized controlled pilot trial

Verholen N., Vogt L., Beckers S., Marx G., Klasen M., Sopka S.

Abstract, to be added
Reference, to be added


Within the framework of the SafePat project, three interdisciplinary and interprofessional team trainings were established at the Center for teaching and training of the University Hospital RWTH Aachen. All three team trainings are designed to improve processes in medical settings, teamwork, communication, and thus also patient safety.

  • Team Training Anesthesia: in this training young anesthetists in their specialist training as well as participating nurses are trained and debriefed in anesthesia-relieving situations.
  • Team Training Neuroradiology: in this training Neuroradiologist and Anesthesiologist and their teams are trained and debriefed interdisciplinary to “shorten the door to needle time” in stroke patients.
  • Team Training pediatric Extra-corporal membrane oxygenation (ECMO): ECMO implantation in small patients is a ultima ratio therapy and a high-risk, error-prone situation – therefore, these interdisciplinary, interprofessional team trainings are essential to improve patient safety.







Out of sight, out of mind? International patient handover from a patient perspective.

Beuken JA, Bouwmans MEJ, Verstegen DML and Dolmans DHJM

Abstract, to be added
Reference, to be added


Real-time Multimodal Feedback with the CPR Tutor.


We developed the CPR Tutor, a real-time multimodal feedback system for cardiopulmonary resuscitation (CPR) training. The CPR Tutor detects mistakes using recurrent neural networks for real-time time-series classification. From a multimodal data stream consisting of kinematic and electromyographic data, the CPR Tutor system automatically detects the chest compressions, which are then classified and assessed according to five performance indicators. Based on this assessment, the CPR Tutor provides audio feedback to correct the most critical mistakes and improve CPR performance. To test the validity of the CPR Tutor, we first collected the data corpus from 10 experts that was used for model training. Hence, to test the impact of the feedback functionality, we ran a user study involving 10 participants. The CPR Tutor pushes forward the current state of the art of real-time multimodal tutors by providing: 1) an architecture design, 2) a methodological approach to design multimodal feedback and 3) a field study on real-time feedback for CPR training.


Di Mitri, D., Schneider, J., Trebing, K., Sopka, S., Specht, M. & Drachsler, H. (2020) Real-time Multimodal Feedback with the CPR Tutor. In: Bittencourt, I.I., Cukurova, M. & Muldner, K. (eds) Artificial Intelligence in Education. AIED 2020. Lecture Notes in Computer Science. Springer, Cham.

Prediction of outcome after acute stroke: comparison of CT-DRAGON score and a simplified score

Anouk Lesenne, Jef Grieten, Alain Wibail, Ludovic Ernon, Luc Stockx, Patrick Wouters, Leentje Dreesen, Elly Vandermeulen, Sam Van Boxstael, Pascal Vanelderen, Sven Van Poucke, Joris Vundelinckx, Sofie Van Cauter, Dieter Mesotten

Background – Acute ischaemic stroke is not only a major cause of mortality, it bears also a heavy burden of morbidity and healthcare-related costs. Accurate prognostication of long-term outcome in these patients may be useful in advising stroke patients and in the allocation of the right stroke treatment.
As for all prognostic stroke scores, the CT-DRAGON was developed to predict long-term functional outcome after acute stroke, at a time when only intravenous thrombolysis was available as treatment. Moreover, their implementation in clinical practice is hampered because of the many variables and by lack of validation in the context of all stroke treatments.

Relevance – This study therefore aims to investigate whether a simplified score, that incorporates less variables and is applicable independently of stroke treatment, may be as accurate and reliable as the CT-DRAGON Score in predicting both favourable and miserable long-term functional outcome in patients after acute ischaemic stroke.

Methods – This single-centre retrospective study analyses 564 patients admitted for stroke between January 2017 and February 2019, in the anterior and posterior cerebral circulation and receiving all stroke treatments (thrombolysis, thrombectomy, combination of thrombolysis and thrombectomy and conservative therapy). At 90 days, favourable (modified Rankin Scale (mRS): 0-2) and miserable outcome (mRS: 5-6) were predicted by CT-DRAGON. Selection of predictors from the CT-DRAGON to build a simplified score was done by machine learning techniques, suited for dimensionality reduction: logistic regression, bootstrap forest and decision tree analysis. Discrimination, calibration and misclassification of CT-DRAGON and its simplified score were tested.

Results – 13% (n=71) of patients had thrombectomy, 17% (n=96) received thrombolysis and 9% (n=53) underwent combination-therapy. 61% (n=344) was treated conservatively. AUC-ROC for CT-DRAGON was 0.78 (95%CI 0.74-0.81) for favourable and 0.78 (95%CI 0.72-0.83) for miserable outcome. NIHSS, pre-stroke mRS and age were the strongest contributors to outcome, and included in the simplified score. AUC-ROC was respectively 0.82 (95CI% 0.79-0.86) and 0.83 (95CI% 0.77-0.87) for the prediction of favourable and miserable outcome. Both scores had better misclassification rates and higher specificity for miserable outcome. The simplified score had better discrimination than CT-DRAGON for both outcomes (both p<0.005).

Conclusion – The CT-DRAGON score revealed acceptable discrimination in our cohort of both anterior and posterior circulation strokes, receiving all treatment modalities. The simplified score had a better discrimination, while maintaining comparable and good specificity and misclassification rate for miserable outcome. The simplified score needs further validation in a prospective, multi-centre study


Lesenne, Anouk & Grieten, Jef & Wibail, Alain & Ernon, Ludovic & Stockx, Luc & Wouters, Patrick & Dreesen, Leentje & Vandermeulen, Elly & Boxstael, Sam & Vanelderen, Pascal & Van Poucke, Sven & Vundelinckx, Joris & Van Cauter, Sofie & Mesotten, Dieter. (2019). Prediction of outcome after acute stroke: comparison of CT-DRAGON score and a simplified score. Euregio Intensive Care Symposium.

Implementation of the SBAR in the hospital: beyond a blended curriculum – Culture of change and change of culture.

Xavier Losfeld and Simon Bataille

Background Regional Hospital Centre (CHR) of Citadelle Liege (BE) is in the accreditation phase of the Joint International Commission (JCI). In January 2018, a team of four referents received training from TeamSTEPPS Instructors. Our institution has selected the optimization of professional communication, a cross-cutting pillar of teamwork, as the first project based on the tools proposed by TeamSTEPPS. A blended training program is only the first step in improving patient safety behaviours. The real issues lie in strategies for effective and sustainable implementation.

Goals The first objective of our study is to assess the perceived impact of a joint program of teaching and implementation of the SBAR tool within the nursing and (para)medical departments of CHR Citadelle Liege.

Intervention In a test phase, we trained and compared equivalent groups from the emergency department, in geriatrics, in two revalidation rooms and in intensive care for gynecology/obstetrics (n=136). Prior to the institutional deployment in February 2019 (n ≅ 1500), managers, leaders, trainers, coaches and end users have been trained (n = 278).

Evaluation methods The first phase of our study is based on an adapted Handover Evaluation Scale (HES). The second phase will be based on the validation of an observational assessment tool of handovers.

Results The results have been presented at the Second Francophone TeamSTEPPS Forum (March 7, 2019).

Lessons learned We propose first a multi-level strategy targeting specific departments or medical specialties. It is a pragmatic approach to a new culture of communication in health care facilities. It seems mandatory to create a leadership development process that affects all levels of the organization.


Second French speaking TeamSTEPPS Forum, Morges (CH), 2019, March 7th.