Japan is building its own versions of ChatGPT — the artificial intelligence (AI) chatbot made by US firm OpenAI that became a worldwide sensation after it was unveiled just under a year ago.
The Japanese government and big technology firms such as NEC, Fujitsu and SoftBank are sinking hundreds of millions of dollars into creating AI systems that are based on the same underlying technology, known as large language models (LLMs), but that use the Japanese language, rather than translations of the English version.
“Current public LLMs, such as GPT, excel in English, but often fall short in Japanese due to differences in the alphabet system, limited data and other factors,” says Daisuke Yamaguchi, a researcher at Took, thou University in Japan who specializes in natural language processing.
In Japan, there’s a concern that AI systems trained on data sets in other languages cannot grasp the intricacies of Japan’s language and culture. The structure of sentences in Japanese is completely different from English. ChatGPT must therefore translate a Japanese query into English, find the answer, and then translate the response back into Japanese.
Whereas English has just 26 letters, written Japanese consists of two sets of 48 basic characters, plus 2,136 regularly used Chinese characters, or kanji. Most kanji have two or more pronunciations, and a further 50,000 or so rarely used kanji exist. Given that complexity, it is not surprising that ChatGPT can stumble with the language.
In Japanese, ChatGPT “sometimes generates extremely rare characters that most people have never seen before, and weird unknown words result”, says Yamaguchi.
One large effort to create a Japanese LLM is using the Japanese supercomputer Fugal, one of the world’s fastest, training it mainly on Japanese-language input. Backed by the Tokyo Institute of Technology, Took University, Fujitsu and the government-funded RI KEN group of research centers, the resulting LLM is expected to be released next year. It will join other open-source LLM in making its code available to all users, unlike GPT-4 and other proprietary models. According to Yamaguchi, who is involved in the project, the team hopes to give it at least 30 billion parameters, which are values that influence its output and can serve as a yardstick for its size.
However, the Fugal LLM might be succeeded by an even larger one. Japan’s Ministry of Education, Culture, Sports, Science and Technology is funding the creation of a Japanese AI program tuned to scientific needs that will generate scientific hypotheses by learning from published research, speeding up identification of targets for inquiry. The model could start off at 100 billion parameters, which would be just over half the size of GPT-3, and would be expanded over time.
Meanwhile, Japanese researchers hope that a precise, effective and made-in-Japan AI chatbot could help to accelerate science and bridge the gap between Japan and the rest of the world.
“If a Japanese version of ChatGPT can be made accurate, it is expected to bring better results for people who want to learn Japanese or conduct research on Japan.”
You must be logged in to post a comment.