Present Giant Language Fashions (LLMs) are predominantly designed with English as the first language, and even the few which are multilingual are inclined to exhibit sturdy English-centric biases. Very similar to audio system who may produce awkward expressions when studying a second language, LLMs typically generate unnatural outputs in non-English languages, reflecting English-centric patterns in each vocabulary and grammar. Regardless of the significance of this situation, the naturalness of multilingual LLM outputs has obtained restricted consideration. On this paper, we tackle this hole by introducing novel computerized corpus-level metrics to evaluate the lexical and syntactic naturalness of LLM outputs in a multilingual context. Utilizing our new metrics, we consider state-of-the-art LLMs on a curated benchmark in French and Chinese language, revealing an inclination in direction of English-influenced patterns. To mitigate this situation, we additionally suggest a easy and efficient alignment technique to enhance the naturalness of an LLM in a goal language and area, reaching constant enhancements in naturalness with out compromising the efficiency on general-purpose benchmarks. Our work highlights the significance of growing multilingual metrics, sources and strategies for the brand new wave of multilingual LLMs.
† Sapienza College of Rome
‡‡ Work partially finished throughout Apple internship
