User experience, knowledge and performance are different measures. A short guide to reading evidence and planning evaluation, alongside a public example from Magen David Adom.
General information for planning and implementation. Not medical guidance, a systematic review or clinical approval of a product.
An example from the field: Magen David Adom's learning system
In June 2022, Magen David Adom published information about a virtual reality learning system for emergency teams. This is a public source documenting the system's existence—not a publication proving improved patient care. The simulation environment shown in ARVR's presentation is available on the project page.
What exactly is being measured?
Distinguish experience and usability from knowledge retention, task performance and longer-term retention. A satisfaction questionnaire is not a measure of professional competence. Success in a virtual environment is not, by itself, evidence that skills transfer to field work.
What should you look for before drawing a conclusion?
Identify the participants, practice task, number of repetitions, measurement tool and comparison group. Check whether the report presents interim findings or a completed study, and what its limitations are. When details are missing, do not fill them in with assumptions or turn a finding into a promise for every organization.
How do these questions become an evaluation plan?
Before a pilot, define one learning objective and a suitable measure, record technical difficulties and interruptions, and review the result with the training team. Keep instructor time, participant time and operating costs separate. Expansion should be based on local needs and the data collected.
Continue to the example and guide
Further reading and sources
The publication describes a learning system; it is not a research paper or evidence of a clinical outcome.
Want to explore suitability for your organization?
Start with a focused demo, choose one scenario and define together what needs to be checked before a pilot.
