Show Me The Evidence

Guest: Dr Jorio Mascheroni, biomedical engineer, with a Master's in Technology Enhanced Learning and a PhD on how best to implant cardiac resynchronisation therapy (CRT) devices. Jorio has spent his career in the medical device industry, where he develops and validates training for cardiac device implanters. Interviewed by: Professor Anthony G. Gallagher Topic: Moving the Learning Curve Away from the Patient: Proficiency-Based Progression for Cardiac Device Implantation Episode Summary Professor Tony Gallagher talks with Dr Jorio Mascheroni, a friend and collaborator of around 20 years, about a career spent trying to make device training safer and more effective. Jorio was there when cardiac resynchronisation therapy was introduced and saw early on that getting specific steps right, and avoiding specific mistakes, decided whether a procedure went well. The conversation traces how training moved from knowledge-based courses and "learning on patients", through the early days of virtual reality simulators, to a proficiency-based progression (PBP) programme with validated metrics and a benchmark that every trainee must reach before moving on. Jorio shares what his randomised trials found, why the approach does not take longer or cost more over time, and why he would change one thing above all: everyone should start in a protected environment and only then move to the operating room. Key Topics Covered Introducing Dr Jorio Mascheroni (0:00) An engineer who moved into education, with a Master's in Technology Enhanced Learning and a PhD on optimal CRT implantation Twenty years of working together From Engineer to Educator: The Arrival of CRT (0:34) Witnessing the introduction of cardiac resynchronisation therapy, a completely new type of procedure Working with the pioneers and realising that performing certain steps properly and avoiding mistakes determined success Early involvement in building a virtual reality simulator for CRT, and the chance to move training away from the patient Training 25 Years Ago: Impression, Not Data (2:55) Training was mostly knowledge-based and theoretical Physicians learning a new procedure were, and still often are, learning on patients Different experts taught different content and techniques for the same procedure Nobody measured what trainees had actually learned: "we had an impression that people were getting better" Simulators: From Room Decoration to a Standard Workflow (6:51) Early simulators were often "the attraction of the training event" Use ranged from decoration to the first attempts at standardising a procedural workflow Bringing the pioneers together to agree an optimal approach, and why a group of experts beats a single speaker telling their own story Was Simulation Seen as a Magic Bullet? (10:04) Industry valued simulation for demonstrating product workflow in a safe setting Effectiveness depended on the faculty delivering the training "Simulators by themselves don't teach much if there is no solid curriculum around" Shared Shortcomings Across Industry and Medicine (11:54) Underestimating the size of skill gaps and taking foundational skills for granted Training differed from one hospital to the next, even for the same therapy Speaker-centred training: "listening to an expert does not mean that you become an expert" Learning curves happening on patients in the clinical environment An Accumulation of Evidence (15:04) No single turning point: evidence built up that different techniques produced different patient outcomes The way you teach someone to do something may determine the outcome for the patient Reading Tony's work on minimally invasive surgery and realising the approach could apply to device implantation The aggregation of marginal gains, and why it sits at the core of proficiency-based progression Related reading: Gallagher AG, Ritter EM, Champion H, et al. Virtual reality simulation for the operating room: proficiency-based training as a paradigm shift in surgical skills training. Annals of Surgery, 2005; 241(2): 364 to 372 What Pacing and CRT Are (21:22) Treatments for selected patients with cardiac rhythm abnormalities A generator placed under the skin of the chest, connected to the heart through implanted leads Permanent pacing has been around for about 70 years; CRT for 25 to 30 years Where Device Training Fell Short (23:41) Challenge one: training or coaching in the cath lab is too late, and it is not fair for the learning curve to happen on a patient Challenge two: many interpretations of optimal implantation, never compared, varying between countries and even neighbouring hospitals Licensed physicians finishing training with wide variability in performance Complications that looked like technology problems were often a suboptimal use of the technology Tony on the surgical side of device procedures: femoral access complications in carotid stenting at Emory in 2003, and the same complications two decades later with TAVR Publication: Mascheroni J, Stockburger M, Patwala A, Retzlaff H, Gallagher AG. Beyond the badge: quantitative insights of skill gaps in licensed early-career cardiac device implanters and evidence-based pathways to proficiency. Journal of Interventional Cardiac Electrophysiology, 2026; 69: 267 to 276. Most licensed early-career implanters did not meet the proficiency benchmark on a simulated assessment. What a PBP Programme for Device Implantation Involves (28:26) A structured, consensus-based description of how the procedure should be performed Metrics, and a proficiency benchmark set from experienced operators A blended curriculum: trainees must pass a knowledge benchmark before hands-on training Faculty training is critical: "If you stick to the methodology the results are generally very predictable" Publication: Mascheroni J, Mont L, Stockburger M, Patwala A, Retzlaff H, Gallagher AG, et al. International expert consensus on a scientific approach to training novice cardiac resynchronization therapy implanters using performance quality metrics. International Journal of Cardiology, 2019; 289: 63 to 69. Experts broke the procedure into 13 phases, 196 steps, 122 errors and 50 critical errors, agreed through a Delphi process with an international panel. Why Validation Matters, and How Trainees Respond (31:44) Validation makes the approach objective, so it is not "the opinion of 10 people" The metrics must be objective, transparent and fair for trainers and trainees Trainees have responded very positively because they know exactly what is expected "Proficiency-based progression is not meant to fail people": it gives each person the time they need to reach the benchmark The assessment exists to give explicit, formative feedback; reaching the benchmark is non-negotiable What the Data Show (38:44) PBP-trained implanters made significantly fewer errors than peers trained on the same simulators with the same content using traditional methods J...

In this episode, Professor Anthony G. Gallagher is joined by Dr Rui Farinha, a senior consultant urologist in Lisbon who has completed fellowships in Spain, Germany and Belgium in laparoscopic and robotic surgery, and who trained with Professor Alexandre Mottrie in Aalst and at Orsi Academy in Belgium. The two met at Orsi in 2019. The conversation begins with a practical question. Robots are expensive, so what do they actually give the surgeon? Rui sets out the case carefully: tremor filtering and motion scaling for precision, wristed instruments for dexterity in tight anatomy such as the pelvis, magnified stereoscopic vision, better ergonomics over long procedures, and a digital platform capable of recording and analysing performance. Then he adds the caveat that frames the rest of the hour. None of these technical advantages automatically produces a better clinical outcome. From there the discussion turns to how surgeons are trained. Rui traces his own path through three eras: an apprenticeship model in open surgery that depended on which cases turned up and which consultant happened to be supervising, a laparoscopic era in which he discovered that open skills did not transfer and that basic skills belonged outside the operating theatre, and a robotic era that was structured from the start around defined objectives, simulation, proximate feedback and a demonstrated standard. The heart of the episode is Rui's programme of research on robot-assisted partial nephrectomy (RAPN). He explains why he chose a technically demanding, high-risk procedure with clearly separable phases and direct consequences for the patient, and he sets out what the studies found. A complex operation can be deconstructed into observable phases, steps, errors and critical errors, with 100 per cent consensus from an international expert panel. Experienced surgeons made 69 per cent fewer total errors than novices. Within the experienced group, the low-error surgeons made 77 per cent fewer errors than the high-error surgeons, and that high-error expert group performed at roughly the level of the better novices. Procedure-specific binary metrics achieved high inter-rater reliability where a global rating scale did not. And a systematic review of partial nephrectomy training models found models widely rated as realistic and useful, but no randomised controlled trials and no evidence of skill transfer. Rui's conclusion is direct. Realism is not evidence. A simulator is a vehicle, not a training programme. Anyone building a curriculum should define the performance they want first and select or construct the simulation second, which is the opposite of what usually happens. He describes a model he developed deliberately as a delivery vehicle for a metric-based curriculum, using readily available animal tissue and emulating eight of the eleven phases of the human procedure, on the grounds that the useful question is not how realistic a model looks but how much high-quality measurable practice it permits. The episode closes on proficiency-based progression in the skills laboratory and in the operating room, on the value of proximate human feedback in an era of AI-delivered training, and on a central principle: the manufacturer's instructions for use teach the surgeon how the robot functions, while PBP determines whether the surgeon can use it proficiently. Key Topics Covered What the robot actually adds, and what it does not | 0:07 Guest introduction: senior consultant urologist in Lisbon, fellowships in Spain, Germany and Belgium, met Professor Gallagher at Orsi Academy in 2019 Robots do not replace the surgeon and do not operate independently; they act as an interface that translates the surgeon's movements Precision through filtering of physiological tremor and scaling of movement, valuable in delicate dissection, suturing and vascular anastomosis | 1:46 Dexterity, vision and endurance | 2:34 Wristed instruments provide additional degrees of freedom where rigid laparoscopic instruments cannot, which matters most in narrow spaces such as the pelvis Magnified high-definition three-dimensional view improves depth perception and tissue plane discrimination; the surgeon controls the camera directly | 3:25 An adjustable console reduces muscular strain and fatigue, supporting concentration and consistency through long procedures | 4:14 The digital platform, and the caveat that matters | 5:04 Because the surgeon's actions pass through a computer-controlled interface, robotic systems can record instrument movement, analyse technical performance and integrate imaging or navigation Rui's caveat: these technical advantages do not automatically produce better clinical outcomes. Performance still depends on training, case selection, team coordination and appropriate use of the technology Professor Gallagher returns to the Dwight Meglan episode and the view that the surgeon, not the platform, makes the decisions | 5:53 Three training eras: apprenticeship, laparoscopy, robotics | 6:43 Open surgery followed the apprenticeship model: observe, assist, then perform increasing parts of the operation under supervision The limitations Rui experienced: dependence on case availability, inconsistent quality of supervision, and progression decided by an individual trainer's subjective judgement | 8:29 Laparoscopy and the skills that did not transfer | 9:20 Competence in open surgery did not translate into competence in minimally invasive surgery Two-dimensional vision and reduced depth perception, the fulcrum effect, long rigid instruments with limited degrees of freedom, and bimanual coordination against an indirect image | 10:10 The realisation that basic laparoscopic skills could and should be learned outside the operating theatre, using box trainers, virtual reality, dry and animal laboratory work and standardised exercises | 10:58 Structured robotic training from 2019 | 11:48 Training with Professor Alexandre Mottrie: console controls, instrument management, camera control and clutching, multiple arms, collision avoidance, bedside communication, troubleshooting and emergency undocking Maintaining situational awareness while physically separated from the patient, supported by online modules, simulation, observation, bedside assistance, supervised console practice and procedural prompting | 12:39 From time served to a defined standard | 13:28 The question shifts from how many cases the trainee has performed to whether the trainee can perform the procedure safely and consistently to a defined standard Clearly defined learning objectives, procedure-specific benchmarks, objective assessment, simulation before patients, and deliberate practice with proximate feedback What simulation cannot teach | 14:19 Clinical judgement, decision making under pressure, recognition of abnormal anatomy, management of bleeding and complications, adapting when the plan changes, and leading the operating team Practice remains very different from centre to centre and department to department | 15:09 Simulation and the Halstedian model | 16:51 The apprenticeship model remains important but is no longer sufficient on its own Professor Gallagher's position: simulation does not end Halstedian training, it improves it, because the consultant now supervises a pre-trained novice who arrives with ...

Professor Stefano Puliatti is Professor of Urology at the University of Modena, a practising robotic surgeon, a faculty member of Surgquest, and the former Medical Director of Orsi Academy in Belgium, where much of the PBP research discussed here was carried out. Key takeaways Training quality should not depend on the luck of being assigned a gifted trainer. A shared methodology makes good outcomes reproducible. Errors, not speed or step count alone, are the real indicator of surgical quality. Process measures on their own do not guarantee it. A proficiency benchmark set at expert level is reachable by nearly all trainees, and usually faster than conventional training allows. Good simulation does not have to be expensive. A validated methodology on a low-cost model can outperform costly kit used without one. Diluting the method, for example by dropping the pre-lab benchmark, measurably slows learning and raises cost. Evidence and further reading The figures cited in this episode come from the peer-reviewed studies below. Please confirm the exact papers you want listed for this episode before publishing. De Groote R, et al. Proficiency-based training and evidence-based methodology: a systematic review and meta-analysis. BJU International. doi:10.1111/bju.70333 Mazzone E, Puliatti S, et al. A Systematic Review and Meta-analysis on the Impact of Proficiency-based Progression Simulation Training on Performance Outcomes. Annals of Surgery, 2021. PMID: 33630473 Puliatti S, et al. Can all surgical trainees be trained to proficiency for a robotic urethro-vesical anastomotic task using a chicken model? A prospective, randomized trial. PMID: 40351291 Randomised trial on the economic impact of proficiency-based progression versus conventional robotic surgical training. PMC12907777 Development and validation of the objective assessment of robotic suturing and knot tying skills for a chicken anastomotic model. Surgical Endoscopy, 2020. doi:10.1007/s00464-020-07918-5 De Groote R, et al. Proficiency-based progression training for robotic surgery skills training: a randomized clinical trial. BJU International, 2022. doi:10.1111/bju.15811 Credits Show Me the Evidence is hosted by Professor Anthony G. Gallagher and produced by Flux Learning.

Show Me the Evidence, Episode 8 Guest: Dr Dwight Meglan Topic: Physics-Based Simulation, Surgical Robotics and Why Simulators Still Don't Measure What Matters Episode Summary In this episode, Professor Tony Gallagher is joined by Dr Dwight Meglan, the engineer who developed one of the first physics-based virtual reality simulators for endovascular procedures in the late 1990s, and who has spent the last two decades building surgical robots. Together they trace 30 years of simulation-based training and ask why so little has changed: simulators are still not verified against real-world physics, the field still measures process rather than skill, and device manufacturers, not educators, still set the agenda. The conversation ranges from haptics and instrumented torquers to Likert scales, credentialing committees, telesurgery risk, and why autonomous surgical robots are a regulatory and financial impossibility rather than a technical one. It closes with the evidence for Proficiency-Based Progression (PBP) and the leadership needed to adopt it. Key Topics Covered 1. Building the first physics-based VR simulators | 0:00 Meeting at Medicine Meets Virtual Reality in the late 1990s Real-time physics of tool and tissue interaction, fluoroscopy and haptic feedback Why the goal was to replicate reality, not design a user experience Physics tests to verify simulator correctness still do not exist, 25 years on 2. Who really drives simulation: the device manufacturers | 2:24 Manufacturers pay for simulation, so manufacturers shape it Training to use the device versus training to perform the procedure Simulators in exhibition booths: marketing tools first, training tools second 3. Haptics and the sensory threshold problem | 3:55 The instrumented torquer: measuring what cardiologists actually feel Just noticeable difference thresholds vary between clinicians Still no published datasets on the forces a cardiologist feels during catheterisation Clinicians praising the haptics on simulators where the haptics were switched off 4. Using devices safely: the stapler and the defibrillator | 8:52 A stapling device with a 6 to 27 per cent leak rate, where one third of patients who develop a leak die Training to the instructions for use is device safety training, not surgical skills training Cardiac defibrillator implantation: clinicians departing from the instructions for use Construct validity findings: some very senior clinicians perform worse than the worst trainee when assessed with objective, peer-derived metrics 5. What should a simulator measure? | 13:19 The original approach: record everything, then find the measures that matter Metrics for mechanical thrombectomy for acute stroke, developed from the human procedure with Mentice How clinician-led metrics forced a redesign of contrast injection, later patented It works when you insist on it, but you must start with the metrics 6. The state of simulation metrics today | 17:35 At a recent conference, almost none of the exhibited simulators had any metrics at all Some manufacturers avoid measurement deliberately: plausible deniability Validated metrics as a purchasing condition: if you cannot build them in, we will not buy 7. From simulation to surgical robotics | 21:52 Why simulation felt like a capped market and robotics did not The analogous DNA of simulators and robots as real-time information processing systems Verification versus validation: robots are bench-tested against dozens of specifications, simulators almost never are 8. Measuring process, not skill | 26:49 Motion tracking and path length: lessons not learned from laparoscopic surgery AI-driven pattern hunting as a fishing expedition without a hypothesis Suturing as the test case: the physics of tissue apposition has never been published Physical intelligence and humanoid robotics will improve simulation from the outside in 9. Likert scales are not measurement | 36:47 Binary, procedure-specific metrics require scoring the entire video, reliably, in pairs One-to-five ratings after watching a few minutes of video are hand waving, not assessment Ring exercises on robotic simulators have never been verified against real forces 10. Whose job is it? Credentialing and privileging | 39:27 Manufacturers certify device use; professional societies and hospitals grant privileges Per-procedure privileging in the United States versus broad qualification in Europe The credentialing committee problem: standards set by the least experienced member Case volume, fellowship length and reputation are social proof, not evidence of competence 11. A jumbo jet a day: the human cost | 44:43 Deaths from surgical skills deficits estimated as equivalent to a full jumbo jet crashing every day, consistent with evidence that around 4.2 million people die within 30 days of surgery each year (Nepogodiev et al., The Lancet, 2019) Why one death at a time never makes headlines the way one crash does The Bristol Royal Infirmary case: peers knew for a decade before the front pages forced action (The Bristol Royal Infirmary Inquiry, Kennedy Report, 2001) Telesurgery at scale: how one underperforming surgeon could soon harm many patients quickly 12. The PBP evidence and the leadership gap | 47:47 Systematic review and meta-analysis evidence: PBP-trained clinicians make approximately 60 per cent fewer objectively assessed intraoperative errors Training to proficiency in one third of the time and at less than half the cost of conventional training "A failure of leadership": adoption at ORSI Academy, AANA, ERUS under Alberto Breda, and increasingly Medtronic cardiovascular came from senior leaders making it non-negotiable The aviation lesson: flight simulation was mandated by government before the evidence existed, while surgery has the evidence and no mandate Publication: Mazzone, E., Puliatti, S., Amato, M., Bunting, B., Rocco, B., Montorsi, F., Mottrie, A. and Gallagher, A.G. (2021). A Systematic Review and Meta-analysis on the Impact of Proficiency-based Progression Simulation Training on Performance Outcomes. Annals of Surgery, 274(2), 281-289. DOI: 10.1097/SLA.0000000000004650 Publication: Puliatti, S., Rodriguez Peñaranda, N., Amato, M., De Groote, R., Farinha, R., Bunting, B., van Cleynenbreugel, B., Mottrie, A. and Gallagher, A.G. (2026). Randomised trial on the economic impact of proficiency-based progression vs conventional robotic surgical training. BJU International, 137, 493-501. https://doi.org/10.1111/bju.70130 13. The next 10 years: AI, automation and the autonomy myth | 54:08 AI will lower the barrier to building simulators and add semi-automated instruction Part-task automation, such as a supervised stapler, is plausible; autonomous procedures are not The barrier is financial and regulatory: manufacturers will not accept liability for practising medicine The self-driving analogy: millions of hours of verified synthetic data would ...

Show Me the Evidence Guest: Dr Ruben De Groote Topic: From Time to Competence: Proficiency-Based Progression and the Reinvention of Robotic Surgical Training Episode Summary In this episode, Professor Tony Gallagher sits down with Dr Ruben De Groote, consultant urologist at OLV in Aalst, Belgium, CEO of 4Health and its digital learning platform Surgquest, and the researcher behind a recently completed PhD on Proficiency-Based Progression (PBP) robotic surgical training across three surgical disciplines. Ruben and Tony first met in 2019 while developing and validating the metrics for the robot-assisted radical prostatectomy. Together they examine an uncomfortable reality: the century-old Halstedian apprenticeship model can no longer produce surgeons who are ready to operate independently. Reduced theatre exposure, rising bureaucracy, and working-hours legislation have hollowed out the "see one, do one, teach one" paradigm, leaving as many as one in three residents unready for independent practice. Ruben makes the evidence-based case for PBP: a standardised, metric-driven approach that measures what a surgeon actually does, gives explicit formative feedback, and trains to a benchmark rather than to a clock. The conversation moves from the failings of subjective assessment through a multi-specialty randomised controlled trial, and on to the harder question of how surgeons progress from proficiency to genuine wisdom. Key Topics Covered 1. Why the apprenticeship model is breaking down (0:50) The Halstedian "see one, do one, teach one" model relied entirely on graded theatre exposure. Over the last 15 to 20 years, bureaucracy and a legal cap on working hours (departments are penalised for exceeding roughly 60 hours a week on average) have eroded that exposure. The result is a vicious circle in which trainees get less time in the operating room, and fellowships originally meant for super-specialisation are being repurposed simply to reach independence. 2. Where the bureaucracy came from (6:06) A wider shift in medicine towards risk aversion and defensive practice. Tasks that were once handled verbally now require written orders, increasing the administrative burden for everyone and pulling ambitious residents out of theatre for half a day or more. 3. Robotics as both a challenge and an opportunity (7:52) Robotic surgery combines complex procedures with the mastery of technology, which raises the training bar. It also places a computer between the surgeon's eyes and the patient, making it possible to store video and surgical data, review procedures, give formative feedback, and measure kinematics. This makes robotics a powerful tool for objectively measuring and improving surgical quality. 4. Exposure is not enough: the case for structure (9:31) Watching a procedure is not the same as being trained to perform it. Ruben and Tony agree that robotics demands the structure the Halstedian approach once imposed, but delivered through universal, evidence-driven standards and benchmarking, rather than the reputation of a single centre or trainer. 5. The systemic problem: a lack of standardisation (12:39) Without standardised curricula, trainees are dependent on the goodwill of whichever consultant they are assigned. Ruben describes a fellowship with six fellows and nine consultants, each teaching the same procedure differently, and warns that patients are effectively used as training models for consultants who were not well trained themselves. 6. Who should set and police the standards (17:37) Standards should be set by rigorous scientific research, not opinion. Scientific societies should define the benchmark and authorities should make it mandatory, in the same way prescribing rights follow formal qualification. Ruben cautions against a large role for industry, citing the conflict of interest in paid proctoring, where a proctor can be pushed to guide a novice through complex steps they have not earned the right to attempt. 7. The multi-specialty randomised controlled trial (23:15) A blinded RCT deliberately included urologists, general surgeons, and gynaecologists to test the belief that some specialties are inherently more skilled. At baseline all three performed equally, and after training all three performed equally well. The methodology, not the specialty, predicted the skill set. As reported in the episode, 67 per cent of PBP trainees reached proficiency by the end of the day, compared with 17 per cent trained by the apprenticeship model. [See PROVESA / De Groote RCT publications below.] 8. Quantifying intraoperative performance: why subjective scales fail (28:18) Likert-based tools such as GEARS are subjective and prone to drift, with a trainer's scoring shifting depending on the video seen just before. For validity, inter-reader agreement should be 80 per cent or higher; in Ruben's study GEARS reached only around 30 per cent, which by default makes it invalid for assessing surgical quality. Binary metrics are the alternative: procedure-specific, zero or one, either a step was performed or a defined error was made. They force assessment of the whole procedure and remove the room to "cheat". 9. Formative feedback in practice (34:59) In Aalst, fellows meet every Thursday to review a recorded procedure on a split screen, with the surgery on one side and the validated metrics on the other. Ruben facilitates, translating the metrics to the procedure and pinpointing exactly where an error occurred. This is transparent, non-subjective feedback that the whole group learns from, and it neutralises the "God complex" that can distort eminence-based teaching, since even high-volume experts sometimes score poorly against objective metrics. 10. Is PBP genuinely better? (38:01) Ruben's position is unambiguous: a methodology associated with a 60 per cent reduction in intraoperative errors compared with the apprenticeship model has to be accepted as better, and fewer errors translate into better patient outcomes. [See Mazzone et al. meta-analysis below.] 11. PBP beyond residents and beyond technical skills (45:14) PBP applies to residents, novice and experienced consultants, and nurses. It has been shown to sharpen non-technical skills too, including a study by Dorothy Breen applying PBP metrics to ICU patient handover using the ISBAR system, which made the process more efficient and filtered out unhelpful information. [See Breen et al. below.] 12. From proficiency to wisdom, and the role of Surgquest (52:01) Proficiency means performing a standard procedure safely. Wisdom is the further step: the volume of experience needed to keep improving, the ability to manage the unexpected, and access to experienced support when a case turns difficult. Surgquest, the 4Health digital learning platform, curates global experts demonstrating not just standard procedures but genuinely challenging cases, helping trainers give fellows more console time in the knowledge that a mistake can be repaired. Publications and Evidence Cited De Groote, R., Puliatti, S., Amato, M., Mazzone, E., Rosiello, G., Farinha, R., Paludo, A., Desender, L., Van Cleynenbreugel, B., Bunting, B.P., Mottrie, A., Gallagher, A.G. (2022). Proficiency-based progression training for robotic surgery skills training: a randomized clinical trial. BJU International. DOI: 10.1111/bju.15811. https://doi.org/10.1111/bju.15811 De Groote, R., Puliatti, S., Amato, M., et al. (2025). Does surg...