Scalable Fingerprinting of Large Language Models

Model fingerprinting has emerged as a powerful tool for model owners to identify their shared model given API access. However, to lower false discovery rate, fight fingerprint leakage, and defend against coalitions of model users attempting to bypass detection, we argue that scaling up the number of fingerprints one can embed into a model is critical. Hence, we pose Scalability as a crucial requirement for good fingerprinting schemes. We experiment with fingerprint design at larger scales than previously considered, and propose a new method, dubbed Perinucleus sampling, to generate scalable, persistent, and harmless fingerprints. We demonstrate that this scheme can add 24,576 fingerprints to a Llama-3.1-8B model --- two orders of magnitude more than existing schemes --- without degrading the model's utility. Our inserted fingerprints persist even after supervised fine-tuning on other data. We further describe security risks for fingerprinting, and theoretically and empirically show how a scalable fingerprinting scheme like ours can help mitigate these risks.