Improving the accuracy and efficiency of maternal venous thromboembolism risk assessments using automation: a simulation-based study

Abstract Introduction Venous thromboembolism (VTE) is the leading cause of maternal death.1 Current risk assessment tools often rely on manual data entry by healthcare professionals (HCPs), which can reduce accuracy and influence thromboprophylaxis decisions.2 Building on an established manual VTE risk assessment (VTERA) tool, we developed a semi-automated VTERA app using SMART on FHIR. The app integrates with the electronic health record (EHR) to automatically extract 11 risk factors, while allowing manual entry for additional variables. Aim To evaluate the accuracy, efficiency, and usability of a semi-automated VTERA app. Methods A randomised crossover study was conducted at a tertiary maternity teaching hospital. Eligible participants included midwives, non-consultant hospital doctors (NCHDs) and pharmacists—reflecting the multidisciplinary staff responsible for VTE risk assessment in this institution. Recruitment used convenience and purposive sampling. Participants completed ten simulated patient assessments using both the manual and semi-automated VTERA tools in randomised order. After clinical handover, they reviewed each chart and completed the risk assessment with the assigned tool. Accuracy was evaluated based on the inclusion of appropriate risk factors, accuracy of VTE risk scores, and appropriateness of thromboprophylaxis recommendations, specifically regarding indication, duration, and dose. Mouse clicks, keystrokes, and completion times were logged to assess efficiency. Usability of the semi-automated tool was evaluated using the System Usability Scale (SUS), and participant comments/observations were documented. Linear and logistic mixed-effects models were applied to accuracy and efficiency outcomes, adjusting for tool type, tool order sequence, familiarity with the existing manual VTERA tool, and years of EHR experience. Results Thirty-one participants (21 midwives, five pharmacists, and five NCHDs) took part. Nearly half (48%) had seven or more years of general EHR experience and 74% were highly familiar with the existing manual VTERA tool. Accurate risk scores occurred in 63% of manual assessments versus 87% of semi-automated assessments, while accurate thromboprophylaxis recommendations occurred in 79% and 91%, respectively. Logistic mixed-effects models showed participants were more likely to obtain accurate risk scores (OR 6.99; 95% CI: 3.31–14.67) and recommendations (OR 3.59; 95% CI: 1.64–7.88) with the semi-automated tool. Linear mixed-effects modelling indicated that automation reduced task time by 49 seconds (95% CI: 32.8–65) and mouse clicks by 12 clicks (95% CI: 9.01–15.37). Keystroke requirements were reduced by 94% (95% CI: 93.4%–94.2%). The semi-automated tool achieved a mean SUS score of 89.8, indicating excellent usability. Participants’ comments emphasised the improved workflow and efficiency associated with the semi-automated tool. They also requested that more risk-factor definitions be incorporated into its design. Conclusion This is the first study to assess the impact of automation on a postpartum VTERA tool’s performance—a key strength of our research. However, a limitation of our study is the simulation environment’s lack of real-world clinical complexity, which may have influenced user behaviour. Overall, automation improved the efficiency and accuracy of VTE risk assessments, reducing manual burden and enhancing the consistency of risk scores and recommendations. The semi-automated VTERA tool also showed excellent usability, though optimal performance requires reliable data, workflow integration, and HCP oversight.