Evaluating Naturalness in Machine Translation: A Case Study of DeepL on Philosophical Texts

Abstract

The rapid development of Neural Machine Translation (NMT) has significantly transformed translation practices, particularly in terms of speed and accessibility. Among these systems, DeepL Translator is widely recognized for producing fluent and natural-sounding translations. However, its performance in translating complex academic discourse remains underexplored, particularly in terms of naturalness. This study aims to analyze the naturalness of Indonesian translations generated by DeepL in translating “What is a Speech Act?” by John Searle. This research employs a qualitative descriptive design. The data consist of words, phrases, clauses, and sentences selected from ten randomly sampled paragraphs of the source text and their corresponding Indonesian translations. The analysis is guided by the concept of naturalness and supported by Translation Quality Assessment frameworks. A three-point scale—natural, less natural, and unnatural—was used to evaluate the data based on semantic accuracy, lexical choice, and syntactic structure. The findings reveal that the majority of translations fall into the less natural category, indicating partial equivalence. Natural translations occur when semantic accuracy, appropriate terminology, and flexible sentence structure are achieved. In contrast, less natural and unnatural translations are characterized by literal rendering, inappropriate lexical choices, structural rigidity, and, in some cases, semantic distortion. These issues are particularly evident in segments involving abstract concepts and complex sentence structures. The study concludes that while DeepL demonstrates effectiveness in translating structurally simple and terminologically stable segments, it remains limited in handling context-dependent meaning and conceptual complexity in academic texts. Therefore, human post-editing is necessary to ensure both accuracy and naturalness. This study contributes to Translation Quality Assessment by providing empirical insights into the performance of machine translation in academic discourse and highlighting the importance of balancing semantic accuracy, lexical appropriateness, and syntactic adaptation.