How to become (and stay) GDPR compliant in AI & ML models

It’s one thing to get to GDPR compliance, and it’s another thing to stay in compliance, especially when your data is big, unstructured, and changing by the minute. This is a common challenge for data science teams around the world, and without systems and strategies in place, you could become subject to a hefty GDPR fine.  

This is a real challenge for AI and machine learning teams because they can use terabytes of data, in various formats to create AI and machine learning models. And in many organisations, data scientists desire the autonomy to create datasets on an as-needed basis without oversight. 

The only problem here is that Personally Identifiable Information (PII) lurks within all that self-managed, ever-changing data, and this keeps teams from being in compliance with GDPR. 

Asking data scientists to self-manage PII is unfeasible, as is the notion of limiting them from creating datasets and models. 

This solution features three phases that can be scheduled and requires minimal intervention from your data science team. It also keeps your team in compliance, even as you create new datasets which potentially contain PII.

1. Scan all datasets

The first step in the process is to crawl through and compile a list of every single dataset (this could be a table or any type of file) that exists in your data science team’s world. 

The benefit of this compilation is that it gives you a starting point for a data registry. Not only is this generally useful for your team to know all the data, sizes, and types of datasets that you work with — but when additional compliance laws or policies crop up, you now have a central place to look. 

Often, teams don’t even know how much or what types of data (parquet, hive, JSON, CSV, etc.) they have.

2. Flag all datasets that contain PII

Once you have your lists, the next step is to comb through the datasets and identify which of them contain PII. Once identified, you’ll need to save the metadata around the datasets, which you should log and store for audit purposes.

3. Remove or redact PII from flagged datasets

Now you can remove your PII. This entails taking the list of flagged datasets containing PII, and removing all users from the dataset who need to be removed according to GDPR guidelines. You’ll also need to log the removal process for audit purposes.

As a note, there are techniques where you can be in compliance without completely removing PII, such as obfuscation. Whether you completely remove or obfuscate depends on your team’s use case and how you use your data. 

Putting into place this automated system gives you several advantages, all without sacrificing a change of workflow or adding onerous restrictions to your way of working.

In the short term, you can sail through an audit and legal review process, instead focusing on the actual work of machine learning and creating models while your data automatically stays in compliance.

Second, this framework puts you in a good position to pass future audits. Audits occur regularly, and rather than having to scramble for every legal review, this framework makes it easy to document and provide evidence of compliance for subsequent audits.

Finally, the system also puts them in a strong position for any future laws or even changes to current laws. The automated system serves as a foundation, and any changes or additional compliance can easily be built on top of the current solution.