Privacy

The European Data Protection Supervisor (EDPS) recently issued a TechDispatch on Automated Decision Making.

Here is what you need to know:

Part 1: 12 Myths About Automated Decision-Making (ADM) Systems

  1. ADM systems always operate within set boundaries. In reality, systems can exceed their intended function unless built-in safeguards prevent this.
  2. ADM systems reliably hand over control to humans in unusual situations. Most systems can’t accurately judge their own limits or real-world complexity, so they may not relinquish control when needed.
  3. Automation doesn’t influence human judgment. Automation bias is real — system outputs can nudge users toward certain decisions or reinforce existing biases.
  4. Human supervision means decisions aren’t automated. Even with human oversight, if users defer to system recommendations, the process can effectively become automated.
  5. Human operators always have the final say. Operators may lack the independence, expertise or confidence to override system outputs, especially when facing complex data.
  6. Human-machine teams always outperform either alone. Misunderstandings, mistrust or overreliance on automation can increase errors, as can machines’ lack of contextual awareness.
  7. Operators can easily override system decisions. Poor interface design and lack of timely, clear information often make meaningful intervention difficult or impossible.
  8. Transparency and explainable AI guarantee better oversight. Explanations can legitimize flawed models and don’t always help users spot when systems are out of their depth.
  9. Human oversight can fix system shortcomings. Oversight is no substitute for robust, reliable systems. Expecting humans to catch every error is unrealistic and risky.
  10. Operators know exactly when and how to intervene. Unclear protocols and insufficient training often leave operators uncertain about their responsibilities and authority.
  11. Operators are always ready and able to take over. Stress, cognitive overload and poor interface design can prevent operators from effectively assuming control when needed.
  12. Operators will always act with the right intentions. Ethical oversight requires clear roles, strong training and inclusive design that involves all stakeholders in decision-making.