Dataset Size Recovery from Fine-Tuned Model Weights

The weights of a neural network are fully determined by its training data and optimization process. Metanetworks aim to recover information about the training process and data by taking as input the weights of other neural networks and predicting specific attributes. For instance, predicting the type of training data the network was trained on or the accuracy it achieved. In this paper, we introduce a new weight-space learning task: dataset size recovery, which seeks to identify the number of samples a given model was fine-tuned on, using only its weights. To assess the feasibility of this task, we conduct a preliminary analysis of the norm and spectrum of fine-tuned weight matrices, we find they are closely linked to the fine-tuning dataset size. Based on this insight, we propose DSiRe, a simple yet effective method that takes the weights of fine-tuned models as inputs, extracts spectral features, and uses a nearest-neighbor classifier to predict the fine-tuning dataset size. Despite its simplicity, DSiRe is effective across diverse data modalities, architectures, and fine-tuning paradigms. By focusing on the spectra features, DSiRe can scale to models with hundreds of millions of parameters while remaining highly effective, unlike most metanetworks which are limited to small architectures. For instance, DSiRe can predict the number of images used to fine-tune a Stable Diffusion model with a mean absolute error of $0.36$ images. Moreover, training DSiRe on large models is also very efficient, and takes under five minutes. Finally, to encourage further research, we introduce the WiSE Bench, a comprehensive benchmark featuring over $3,600$ fine-tuned models.