Updated: Sep 14, 2020
The data subject shall have the right to obtain from the controller the erasure of personal data concerning him or her without undue delay and the controller shall have the obligation to erase personal data without undue delay...
GDPR: Chapter III, Article 17
GDPR and its right to be forgotten is reshaping how we approach system design and data architecture. The right to be forgotten gives individuals legal protection over their data such that they may request an organisation to erase all data stored about them.
Data subject refers to the end user (i.e. your customer), and the controller refers to the organisation holding personal data (i.e. your organisation).
The impact of GDPR is wide-reaching, but even if it is not currently applicable to your organisation, it is likely that similar legislation will develop across the world in the coming years.
In this write-up, I will introduce you to the concept of crypto-shredding and how it can help you from a technical perspective, comply with the right to be forgotten.
Crypto-shredding: How it can solve modern data retention challenges
Crypto-shredding is the concept of destroying data through the destruction of the cryptographic keys protecting the data. Without the decryption keys, the encrypted data is unusable - like a safe without the safe key or combination.
The relevance of crypto-shredding stems from advancement in technology and changes to the political environment.
Distributed technology (such as blockchain) depends on data being immutable, which introduces challenges for data destruction - doing so would conflict with the append-only requirement and violate the integrity of the chain. Unlike traditional databases, blockchain is designed to be an "append-only" data structure. That means you can only append new data in the form of additional blocks, which will then be chained together with previous blocks of data. All data is permanently stored across more nodes and cannot be altered.
From a political perspective, new regulations define specifics for data retention and consumer rights over their personal information.
On the surface, the technology and political landscapes are incompatible - this incompatibility is what crypto-shredding aims to solve.
What if your customer asks you to remove their personal data from your system:
Does your system allow this?
What is the impact on backup and archiving systems?
The technical implementation of this may seem straight-forward. If we have a database of customers, simply delete the customer's personal data, thereby allowing the customer to exercise their right to be forgotten (assuming your legal environment allows this).
The challenge of backups and archives
If Mary Jane wishes for her personal data to be removed, we delete her record from the database . A challenge arises in that the record hasn't be deleted from our backups.
For most back solutions, it is not feasible or possible to remove a specific record or records from a backup.
Even if it were possible, it would hinder data integrity checks, and increase storage/archiving costs because mutable storage would be required.
In large-scale environments, the time to simply read and modify backup media may be greater than the influx of deletion requests.
In the example above, two backups are taken before Mary's record is deleted. While the primary store and the latest backup (Backup 03) do not contain her personal data, historic backups (Backup 01, and Backup 02 ) retain a copy of since-deleted data for an extended period of time.
This is where crypto-shredding can help.
Crypto-shredding - Making use of the Encryption Key
We now know three things. We know that
We need to destroy certain personal data from our primary data store and all historic backups;
We cannot feasibly alter backup media or dispose of them; and
We must destroy the data without undue delay
If each customer record is encrypted with their own key, the key for that specific customer can be destroyed.
This will render their data effectively destroyed in both the primary store and all historical backups.
Without the key of the customer, their encrypted data cannot be decrypted or processed.
Keys must be stored separately from the customer data and should not form part of the customer backups. However, it is important to still back up the key store - without it, all personal data of customers are unrecoverable (Backup A, Backup B).
To achieve the right to be forgotten
The backup retention period of the key store needs to be significantly reduced.This ensures that deleted keys are removed promptly. From a transactional data retention perspective, we can now maintain long-term backups of customer data.
When deleting a customer record, both the record and the key should be deleted at the same time. After the last backup containing the customer key is removed, the customer data is unrecoverable.
Crypto-shredding is a relatively new concept driven by changing cloud computing and political landscapes. A crypto-shredding solution to data retention addresses several technically difficult requirements of GDPR. It also brings with it a broad range of challenges in how an application consumes and distributes personal information.
Each technical solution will have its own merits and will meet regulatory compliance differently. Crypto-shredding should be balanced between the legal requirements and technical capabilities, as it brings significant technical complexity. As implementations begin to exist and evolve, crypto-shredding and similar solutions will likely become commonplace.
To learn how Microsoft Azure implements crypto-shredding, read our article on Storage Encryption and Key Management with Azure Key Vault.
Are you preparing for your CCSP certification exam? Use our Q&A resources to master concepts related to Crypto-shredding.