Holistic Unlearning Benchmark:
A Multi-Faceted Evaluation for
Text-to-Image Diffusion Model Unlearning

Saemi Moon*,1, Minjong Lee*,1, Sangdon Park1,2, Dongwoo Kim1,2
CSE, POSTECH1, GSAI, POSTECH2
*Indicates Equal Contribution
Paper Code arXiv
[03.10.2025] 🛠 The code is currently being updated. Please stay tuned!

HUB: Holistic Unlearning Benchmark

HUB systematically evaluates unlearning methods across six key perspectives, covering 33 target concepts categorized into four dimensions: Celebrity, Style, IP, and NSFW. HUB provides an extensive set of 16,000 prompts per concept.

No single method outperforms in all perspectives

comparison table

For each method and task, we compute the average performance across all concept categories as shown in the table below, and we then use the averages to rank the methods.

Leaderboard for unlearning methods

We report the average performance across concepts for each category. Overall represents the average performance across all categories.

Comparison with previous methods and benchmarks

comparison table

We use the abbreviations I, S, C, N, and O to denote Intellectual Property (IP), artist style (Style), Celebrity, NSFW, and Object, respectively. ✓ indicates that the method quantitatively evaluates the corresponding task.

Citation


        @article{moon2024holistic,
          title={Holistic Unlearning Benchmark: A Multi-Faceted Evaluation for Text-to-Image Diffusion Model Unlearning},
          author={Moon, Saemi and Lee, Minjong and Park, Sangdon and Kim, Dongwoo},
          journal={arXiv preprint arXiv:2410.05664},
          year={2024}
        }