ProtoSnap: Prototype Alignment For Cuneiform Signs

The cuneiform writing system served as the medium for transmitting knowledgein the ancient Near East for a period of over three thousand years. Cuneiformsigns have a complex internal structure which is the subject of expert paleographicanalysis, as variations in sign shapes bear witness to historical developments andtransmission of writing and culture over time. However, prior automated techniquesmostly treat sign types as categorical and do not explicitly model their highly variedinternal configurations. In this work, we present an unsupervised approach forrecovering the fine-grained internal configuration of cuneiform signs by leveragingpowerful generative models and the appearance and structure of prototype fontimages as priors. Our approach, ProtoSnap, enforces structural consistency onmatches found with deep image features to estimate the diverse configurationsof cuneiform characters, snapping a skeleton-based template to photographedcuneiform signs. We provide a new benchmark of expert annotations and evaluateour method on this task. Our evaluation shows that our approach succeeds inaligning prototype skeletons to a wide variety of cuneiform signs. Moreover, weshow that conditioning on structures produced by our method allows for generatingsynthetic data with correct structural configurations, significantly boosting theperformance of cuneiform sign recognition beyond existing techniques, in particularover rare signs. Our code, data, and trained models are available at the project page:https://tau-vailab.github.io/ProtoSnap/