Web crawling and data protection: CNIL has issued a 180,000 EUR fine against a provider of automobile insurance policies for failure to adequately protect data in violation of GDPR, specifically citing disallowing web crawling as a way to protect personal data from wrongful access.
In particular the company :
- sent usernames and passwords in cleartext
- allowed users to access other users accounts
- allowed users’ accounts to be accessible by the general public when entering a URL or changing the last numbers in a URL
The compromised information included copies of driver’s licenses, registration cards, bank identification records and documents to determine whether a person had been subject to a license withdrawal or hit-and-run.
Key takeaways:
- Don’t send passwords in cleartext.
- Adopt a strong password policy.
- Ensure access controls to information are limited and accurate.
- Use a “robot.txt” or other means to disallow SEO and crawling by search engines of internal web pages containing sensitive information.