The European Data Protection Supervisor (EDPS) recently issued a TechDispatch on Automated Decision Making.

Here is what you need to know:

Part 1: 12 Myths About Automated Decision-Making (ADM) Systems

Part 2: Effective Human Oversight

• Operator Intervention and Agency.
  • Systems must allow operators to intervene and override decisions when necessary.
  • Operators should have the authority and ability to act independently without fear of repercussions.
• Access to Relevant Information.
  • Operators need access to all necessary information to understand and evaluate system decisions.
  • System decisions should be explainable.
  • The system interface should have a clear, intuitive, and accessible design.
• Operator Intentions and Expertise.
  • Operators should have intentions aligned with fairness, lack of bias, and respect for fundamental rights.
  • Operators must possess the necessary expertise and training for their oversight role.
• Organizational Support and Culture.
  • Organizations must provide stable, healthy, and fair working conditions.
  • Sufficient time must be allocated for review and oversight tasks.
  • Adequate training and support should be provided, covering system functioning, failure cases, operational procedures, available tools, task objectives, and assessment criteria.
  • Human oversight should be valued as a key safeguard, not a fallback for system failures or a means to assign individual blame.
• System Interface Design.
  • Interfaces should be clear, intuitive, and accessible, allowing operators enough time to react and engage critically.
  • Information presented should be concise and easy to understand, avoiding overload.
• Auditing and Continuous Improvement.
  • Regular external audits should assess:
    • Operator dependence on automated decisions and the authenticity of human judgment.
    • Timing and presentation of system outputs.
    • Usability and cognitive ergonomics of interfaces.
    • Effectiveness of training programs and organizational culture in supporting critical thinking and encouraging challenge not blind compliance.
  • Oversight should include ongoing mechanisms for feedback from those affected by system decisions, not just periodic audits.