11 November 2024 in IEEE Transactions on Image Processing
The open-set text recognition task is a generalized form of the (close-set) text recognition task, where the model is further challenged to spot and incrementally recognize novel characters in new languages, which also indicate a shift in the language model. In this work, we propose to alleviate the confounding effect such biases under an open-set setup by learning individual character representations out of their context.We propose a Character-First Open-Set Text Recognition framework that treats individual characters as first-class citizens by cotraining the feature extractor with two context-free learning tasks. First, a Context Isolation Learning task is proposed to wipe the context for each character, utilizing a character mask learned in a weak supervision manner. Second, the framework adopts an Individual Character Learning task, which is a single-character classification task with synthetic samples. Our framework can reliably spot unseen characters in Japanese with F1-score over 64%, and can adapt to recognize unseen characters in Japanese, Korean, Greek, and other languages without retraining. The framework also shows decent many-shot performance on close-set text benchmarks with 91.5% line accuracy on IIIT5k and a speed of over 69 FPS single-batched, making it a feasible universal lightweight OCR solution.