Why Translated Preference Data Isn't Enough for Japanese LLMs

If you search for Japanese preference or RLHF data today, the public results fall into two groups: English datasets translated into Japanese, and datasets whose responses were generated by another model. Both are useful for experiments. Neither is a substitute for judgments written by native speakers about Japanese output.

What is publicly available

A widely used example is a Japanese version of an English helpfulness-and-harmlessness dataset, produced by machine translation of a sampled subset. Others reformat chatbot-arena style comparisons whose text was produced by a language model. Their dataset cards say as much. That is not a criticism of the maintainers, who documented their method clearly. It is a description of what the data can and cannot teach.

Where translation breaks down

• Register: an English reply that is 'friendly' can become over-familiar or oddly stiff in Japanese. Preference labels made on the English source do not transfer.
• Cultural fit: what counts as an appropriate refusal, apology or recommendation differs by context and relationship.
• Naturalness: translated phrasing has recognizable patterns. A reward model trained on it can learn to prefer translationese.
• Safety: harm categories and sensitivities are local. Translating an English red-team list misses Japan-specific risks.

Why model-generated responses are limited

When both candidate answers come from the same family of models, preference labels mostly rank differences within that family's style. They say little about how a Japanese professional would judge the answer.

What to ask a vendor

• Who wrote the prompts, and who judged the responses? Native Japanese speakers, and with what domain background?
• Were prompts authored in Japanese or translated?
• What is the rubric, and how is annotator agreement measured and reported?
• How is calibration handled across annotators over time?
• What rights do you have to use the judgments and prompts for training?

A practical approach

Start small. A pilot of a few hundred carefully designed prompts, judged by a vetted pool against a written rubric, often reveals more than a large set of unclear provenance. Use it to test whether human-native labels change your reward model's behavior before scaling.

This article is general guidance. Descriptions of public datasets are based on their published dataset cards at the time of writing.