Machine Unlearning

Preface

Chapter 1

1. Introduction to Machine Unlearning

1.1. Potential Approaches

1.2. Motivation for Unlearning

1.3. Unlearning Framework

1.4. Unlearning requests

1.5. Design Requirements

1.6. Machine unlearning applications

Chapter 2

2. Technological Approaches to Unlearning

2.1. Algorithmic Strategies for Unlearning

2.2. Data Deletion Techniques

2.3. Model Retraining vs. Unlearning

2.4. Na¯ve Unlearning Approaches

2.4.1. Retraining from Scratch

2.4.2. Model Reset and Data Exclusion

2.5. Approximate Unlearning Methods

2.5.1. Fine-tuning and Gradient Reversal

2.5.2. Knowledge Distillation and Substitution

2.6. Exact Unlearning Methods

Chapter 3

3. Machine Unlearning in GenAI and LLM

3.1 Overview of unlearning challenges in Generative AI and LLMs.

3.2. Unlearning in Generative AI

3.2.1. Retraining from Scratch

3.2.2. Knowledge Distillation

3.2.3. Gradient reversal and model editing

3.3. Unlearning in Large Language Models (LLMs)

3.3.1. Parameter-tuning Approaches

3.3.2. Parameter-Agnostic Methods

3.3.3. LLM Unlearning Taxonomy

3.4. Comparative Insights

Chapter 4

4. Benchmark Datasets and Experimental Frameworks

4.1. Benchmark datasets and experimental frameworks

4.1.1. TOFU

4.1.2. WMDP

4.1.3. CIFAR-10

4.1.4. MNIST

4.1.5. Fashion MNIST

4.1.6. UTKFace

4.1.7. Machine Unlearning for Facial Age Classifier (MUFAC)

4.1.8. Machine Unlearning for Celebrity Attribute Classifier (MUCAC)

4.2. Unlearning-Specific Benchmarks

4.2.1. Machine Unlearning Six-way Evaluation (MUSE)

4.2.2. Real-World Knowledge Unlearning (RWKU)

4.3. Unlearning Efficiency

4.4. Model Utility

4.5. Machine Unlearning Tools and Frameworks

4.5.1. Design requirements

4.5.2. Validation of machine unlearning

4.5.3. Metrics

4.6. Unlearning verification

4.6.1. Feature Injection Test

4.6.2. Forgetting Measuring

4.6.3. Information leakage

4.6.4. Membership inference attacks

4.6.5. Backdoor attacks

4.6.6. Slow-down attacks

4.6.7. Interclass confusion test

4.6.8. Federated verification

4.6.9. Cryptographic proofs

Chapter 5

5. Case Studies in Machine Unlearning

5.1. Unlearning in Healthcare

5.1.1. Unlearning Versus Deskilling

5.1.2. Technical Challenges

5.1.3. Limitations of Machine Unlearning in Healthcare

5.1.4. Unlearning in Digital Healthcare

5.2. Unlearning in Financial Services and Data Retention

5.3. Machine unlearning in social media and user data control

5.3.1. Challenges and Future Directions

5.4. Social Bias Mitigation in Language Models via Machine Unlearning

5.4.1. Machine unlearning techniques

Chapter 6

6. Data Privacy Ethical Implications

6.1. Right to be Forgotten (RTBF)

6.1.1. Reasons behind the increase in RTBF

6.1.2. Issues of LLMs related to Personal Data

6.1.3. The origin and development of the right to be forgotten

6.2. Toward a Multifaceted Strategy

6.3. Recommendations to maintain data privacy

Chapter 7

7. Challenges in Applying RTBF to AI Systems

7.1. Technical challenges

7.2. Conceptual challenges

7.3. Ethical Concerns Related to AI Inferences

7.4. Jurisdictional and cross-border data flows

7.5. Commercial Interests and Data Monetization

7.6. Societal and Cultural Implications

7.7. Stochasticity of training

7.8. Incrementality of training

7.9. Catastrophic unlearning.

7.10. Challenges of Applying RTBF to LLMs

7.11. Computational expense

7.12. Loss of important information

7.13. Lack interpretability

7.14. Enhancing Public Understanding of RTBF and AI

7.15. Balancing Privacy Rights with the Benefits of AI

7.16. Hyperparameter search

7.17. Uncertainty

Chapter 8

8. Conclusions and Future Research Directions

8.1. Future research trajectories

8.1.1. Influence functions are the dominant methods

8.1.2. Reachability of the model parameters

8.1.3. Unlearning verification (Data auditing)

8.1.4. Federated unlearning

8.1.5. Model repair by unlearning

8.1.6. Proposed Changes to the GDPR and CCPA to Better Address the AI Challenge

8.1.7. Unlearning for Diverse Data Structures

8.1.8. Unlearning for Nonconvex Models

8.1.9. User-Specified Granularity of Unlearning

8.1.10. Privacy Assurance for Unlearning

8.1.11. Quantitative evaluation metrics

8.1.12. Adversarial Machine Unlearning

8.1.13. Interpretable Machine Unlearning

8.1.14. Causality in Machine Unlearning

8.1.15. Verifiable Machine Unlearning

8.2. Summary