An Unsupervised Model of Orthographic Variation for Historical Document Transcription

摘要

Historical documents frequently exhibit extensive orthographic variation, including ar-chaic spellings and obsolete shorthand. OCR tools typically seek to produce so-called diplo-matic transcriptions that preserve these variants, but many end tasks require transcriptions with normalized orthography. In this paper, we present a novel joint transcription model that learns, unsupervised, a probabilistic mapping between modern orthography and that used in the document. Our system thus produces dual diplomatic and normalized transcriptions simultaneously, and achieves a 35% relative error reduction over a state-of-the-art OCR model on diplomatic transcription, and a 46% reduction on normalized transcription.