A Transformer-Based Online Urdu Handwritten Text Recognition Using Spatial-Temporal Stroke Data and Real-Time Learning API

Transformer-Based Online Urdu Handwritten Text Recognition

Abstract

Online handwritten text recognition (HTR) is a challenging task, especially for script-rich languages like Urdu and Arabic. In contrast to offline recognition, online HTR works with real-time spatial-temporal information recorded through digital pen movement, such as the order of strokes, direction, and speed. The current study presents a novel transformer-based framework that uses spatial-temporal stroke data to recognize handwritten Urdu text in real-time. In order collect real-time handwriting samples from users, a mobile application was developed that records pen coordinates, direction, and velocity. The data were collected, pre-processed, and then used to train several models covering typical machine learning, deep learning (CNN-LSTM, BiLSTM) and transformer architectures. Experiments demonstrate that transformer-based?techniques are superior to other methods, obtaining 3.2% CER and 7.8% WER, respectively. Additionally, a RESTful Flask API is created for real-time training and prediction. For low-resource cursive languages like Urdu, the work presents a deployment-ready API, a distinctive dataset, and strong recognition models.