Professionals' Perception and Trust in AI Predictions in High-Risk Contexts

Helder Rodrigo Pinto is a Professor and Researcher in Computer Engineering at ISTEC Porto, specializing in Software Engineering, Programming, and Data Science. He supervises student R&D projects in the Computer Engineering program and serves as Director of CITECA (Center for Research in Advanced Technologies) as well as President of the Technical-Scientific Council. With a strong background in Python and .NET, UML, and Project Management, he is known for a practical, project-based teaching approach grounded in agile Scrum methodologies, continuous feedback, and critical thinking. Beyond academia, Helder also coordinates R&D at MEDCEI, a startup focused on delivering healthcare services at home, fostering strong industry partnerships that create real-world opportunities for students and applied research. Reference: This presentation examines how professionals perceive, calibrate, and operationalize trust in AI-driven predictions when decisions carry high stakes and low tolerance for error. We explore the psychological and organizational drivers of trust—such as perceived competence, transparency, accountability, and prior experience—as well as factors that undermine it, including model opacity, uncertainty miscommunication, and automation bias. Building on real-world high-risk scenarios (e.g., healthcare, critical infrastructure, and safety-relevant operations), we discuss how explanation quality, confidence/uncertainty reporting, and human-in-the-loop workflows affect decision quality and responsibility allocation. The talk proposes practical design and governance recommendations to improve appropriate reliance, including uncertainty-aware interfaces, auditability, training for judgment calibration, and socio-technical safeguards that align AI outputs with professional standards and regulatory constraints. The goal is to move beyond “trust vs. distrust” toward measurable, context-sensitive trust calibration that improves outcomes without eroding accountability. Key words: AI trust calibration, High-stakes decision-making, Uncertainty communication, Human-AI collaboration, Explainable AI.