Information
Authors
Brian Tang, Kang G. Shin
Conference
Demo
Blog
Intro
Modern Vision-Language Models (VLMs) are remarkably good at general scene-text recognition, but they consistently fall apart on the long tail of text that is degraded by motion blur, low resolution, occlusion, or glare. This is exactly the kind of text captured in egocentric settings such as smart glasses, dash cameras, and phones held at arm’s length — text that remains readable to a human but becomes unintelligible to OCR models and off-the-shelf VLMs.
Almost every STR and textual-VQA dataset sidesteps this problem by placing text at the focus of the image and in high resolution, where ground truth is easy to label. We instead target the long-tail distribution of noisy, illegible text head-on. We introduce Hawkeye4B, a VLM fine-tuned to read and reason about degraded text by leveraging deep contextual priors and pixel-level visual alignment, and BSTRB (Blurry Scene Text Recognition Benchmark), a real-world dataset and challenge benchmark built to train and evaluate this capability. A system that robustly reads illegible text can extend the range of readable text for smart-glasses users and serve as assistive technology for the 253 million people worldwide with visual impairments.
Examples of degraded text that stays legible to humans but defeats off-the-shelf OCR.
Design Overview
BSTRB is built from a novel data-collection pipeline that turns real-world video into low-quality image / high-fidelity text pairs. Starting from 1,699 real-world video sequences, we process the footage into 9,936 noisy image samples containing 649,861 annotated words, each captured with full-image context rather than as isolated cropped words. Text annotations are produced from the clearest early frames using a state-of-the-art VLM (GPT-5), which outperforms OCR on this content, while per-word bounding boxes and pixel sizes come from Google Vision OCR. Pairing accurate transcriptions with fine-grained scale annotations lets us plot model performance as a continuous function of text pixel size — something prior datasets, which provide either scale or spatial annotations but not both, cannot support.
The BSTRB pipeline turns real-world video into low-quality image / high-fidelity text pairs.
Hawkeye4B is a fine-tuned VLM built on small open-source backbones (Qwen3-VL-4B, with a Qwen2.5-VL-3B variant). It is trained on an aggregate of 13 STR and text-image datasets — including BSTRB — that have been reformatted into single-turn transcription tasks spanning diverse aspect ratios, text sizes, positions, and print-vs-handwriting types. In deployment and data collection we additionally use a SAM3 segmentation layer to isolate text-bearing regions of interest, though this is excluded from evaluation to keep comparisons fair.
Evaluation
Despite its small size, Hawkeye4B performs on-par with — and in some cases better than — much larger closed-weight proprietary VLMs (e.g., GPT, Grok, Claude), and closes much of the gap to Gemini 3, a model estimated to be roughly 100× larger and trained on massive-scale proprietary vision data. This near-Gemini-3 text reading from a small, open, fine-tunable model is the core result beyond the dataset itself, making Hawkeye4B the second-best performing model in our comparison.
Blurry scene-text reading performance across models.
Word inclusion plotted as a continuous function of text height (px).
An ablation that retrains the model without BSTRB degrades performance by 10–20% across metrics on multiple text datasets while staying roughly even on the rest, evidence that BSTRB carries a strong, distinct learning signal. Attention probing shows the fine-tuned models have fewer blind spots than the base Qwen models — attending to text-bearing regions the base models miss entirely — and out-of-distribution samples collected in the wild confirm the models are learning to decode degraded text rather than overfitting.
Removing BSTRB from training reduces performance across nearly all datasets and metrics.
Attention maps (base vs. fine-tuned): Hawkeye has fewer blind spots; circles mark regions the base Qwen misses.
Finally, we use this setup to formalize a visual acuity limit for VLMs. Performance on in-the-wild text degrades around 20px, nearly 3× earlier than the ~7px threshold observed for clean digital text — a concrete characterization that matters for deployments where a region of interest lands between roughly 200×200 and 500×500 pixels, as is typical for smart glasses.
A Snellen-chart experiment formalizing a visual-acuity limit for VLMs.
Discussion
Hawkeye4B’s accuracy and speed suggest that on-device deployment in wearables like smart glasses is practical, enabling assistive reading of signs, medicine bottles, flight boards, nutrition labels, papers, and screens. The model also exhibits emergent multi-object transcription despite being trained only on single-object samples. Current limitations, relative to the base Qwen model, are in multilingual coverage and output formatting — natural directions for future work.
Citation
@inproceedings{tang2026hawkeye,
title={Hawkeye4B: Reading Illegible Text with Vision Language Models},
author={Tang, Brian and Shin, Kang G.},
booktitle={},
year={2026}
}