Direct and multi-level machine translation: Korean-Uzbek and Korean-English-Uzbek language pairs

Abstract

This pilot study compares direct Korean–Uzbek machine translation with a pivot-based Korean–English–Uzbek workflow for a low-resource language pair. Two Korean public-information texts, divided into ten long segments, were translated using Google Translate and ChatGPT. The outputs were examined manually using selected Multidimensional Quality Metrics categories accuracy, fluency and style and were then compared using the automatic metric BLEU. Manual annotation identified fifteen errors in the direct output and four errors in the pivot-based output. However, the reported BLEU scores were higher for the direct workflow in both texts (41.21 versus 22.98; 46.53 versus 8.91). Manual and automatic evaluation therefore produced opposite rankings. The discrepancy shows that surface n-gram overlap and human judgments of meaning and naturalness measure different aspects of translation quality, and that a single metric is insufficient for this language pair. Because the dataset is small and the evaluation settings are incompletely documented, the findings are exploratory rather than generalizable.