I just finished reading a paper titled “Machine Unlearning: its nature, scope, and importance for a ‘delete culture’” by Luciano Floridi. It explores the cultural shift from recording to deleting information in the digital age and its implications on privacy, intellectual property (IP), and Large Language Models like ChatGPT.
I thought that the author did an excellent job addressing what is a multifaceted and sprawling topic. In fact it combines elements of data privacy and ethics that we have been wrestling with over a number of years, with a new set of technical challenges around practicality and feasibility given the way that machine learning systems work as opposed to more traditional programming models.
The following is my attempt to synthesize some of the key concepts and questions the paper raises, if only to help me better process and understand it myself. If I have misrepresented anything, please do let me know.
Key Concepts and Questions
- Unavailable in Principle – The paper begins by defining a delete culture where information, in principle legal, is made unavailable or inaccessible because unacceptable or undesirable, especially but not only due to its potential to infringe on privacy or IP.
- Delete vs Block: It focuses on two strategies in this context: deleting, to make information unavailable; and blocking, to make it inaccessible. The article argues that both strategies have significant implications, particularly for machine learning (ML) models where information is not easily made unavailable.
- Machine Unlearning (MU): The developing research area of Machine Unlearning (MU), still in its infancy, seeks to remove specific data points from ML models, effectively making them ‘forget’ completely specific information. If successful, MU could provide a feasible means to manage the overabundance of information and ensure a better protection of privacy and IP.
- This particular concept gets very complicated quickly, because machine learning models and other ‘downstream’ processing may still have knowledge of a concept even after the originating facts have been deleted.
- I found one of the references in the paper, Coded Machine Unlearning, to be very helpful exploring some of the detailed work happening here. There is an excellent list of papers on Machine Unlearning curated on Github.
- Ethical Risks of MU: Introducing a system of MU, inherently creates another ethical risk, that is the potential for misuse, overuse, and underuse of MU. For example, information “blocking” has been a tactic often deployed in data privacy cases where remediating software is simply not feasible and providers have simplify geofenced access to their systems. As Floridi points out, we will likely have mixed legislative models emerge where “unlearning” will be more appropriate, and others where a mix of both blocking and unlearning approaches may be used. Appropriate policies for handling these risks need to be considered before rolling out systematic approaches to MU.
Exploring this topic has made me very interested in improving my understanding of the field of machine learning and how it differs from the forms of computer programming I learned earlier in my career. I have already started diving into lots of other research to understand what I need to “unlearn”.
I plan to follow this article with another piece exploring some practical use cases around machine unlearning and the challenges as I see them based on how machine learning models work.
Note: The robot in the cover art was created in MidJourney using the following prompt, “A sentient robot unlearning what it knows by removing its brain circuits, shot on Agfa Vista 200, side-angle view, 4k –ar 16:9 –stylize 1000 –v 5 – Image #3 @adammonago”