Aiming to send more manga titles out into the world in a shorter time, AI research graduates from the University of Tokyo have developed a machine translation system that has not only significantly streamlined the translation process, but also enabled several unexpected innovations.
Mantra Engine (above) can swiftly recognize text placed in the manga panel and identify its order, automatically turning the words on the page into a translation that considers the situation and the personalities of the characters. Leveraging their respective areas of specialty, the members of Mantra (left) have striven to create a system that is easy to use. MITSUKI KUCHITAKA
Mantra Engine (top) can swiftly recognize text placed in the manga panel and identify its order, automatically turning the words on the page into a translation that considers the situation and the personalities of the characters. Leveraging their respective areas of specialty, the members of Mantra (bottom) have striven to create a system that is easy to use. MITSUKI KUCHITAKA
Japanese manga, such as Demon Slayer, One Piece, Slam Dunk, and Dragon Ball, have a huge following around the world. Although manga are now translated into various languages, an endless amount of pirated translations is still being produced. Among the many reasons, two motivations are the time lag between translating and releasing the latest titles and the limited variety of titles existing in translation. Wouldn’t it be wonderful if translations of a wide range of the latest titles were available immediately and anywhere in the world? Mantra Inc., a startup founded in 2020 by AI-research graduates from the University of Tokyo, is taking on the challenge of realizing such a future. Using innovative AI technology, the firm has developed a translation system known as Mantra Engine, which not only pursues highly accurate, natural-sounding and nuanced machine translations—a feat considered difficult with manga—but has also succeeded in halving the time conventionally required for the process. “This system has already been adopted by more than 10 companies in Japan and overseas, where it supports the translation of 40,000 to 50,000 pages (equivalent to about 250 titles) a month,” says ISHIWATARI Shonosuke, Mantra’s co-founder and CEO.
In 2019, Mantra participated in SXSW (South by Southwest), a creative conference that covers a broad range of fields from technology to culture. The company plans to also focus on translating manga from countries other than Japan.
Ishiwatari, who came up with the idea of Mantra Engine in 2018, had been pursuing his doctorate at the University of Tokyo, researching how computers process the language used in everyday life. He chose to focus on manga translations for the medium’s unique characteristics—a diverse range of fonts, distinctive forms of speech dependent on individual characters’ personalities and emotions, and multiple speech bubbles in a single panel—that make machine translation especially complicated. Ishiwatari says, “It was an area that promised technological challenges.”
Together with HINAMI Ryota, the company’s CTO, who was also conducting research on image recognition technology in the same doctoral program, he developed a dedicated engine that enabled AI to learn from massive amounts of data, focusing on manga graphics and translation. It succeeded in accurately reading the location and content of the text in an image and in translating the words in colloquial form, while taking their order and the context into account. The text in the image can be translated and replaced just by uploading the manga’s data into the system and selecting the language, with the entire process taking as little as a few seconds per page. “If we heighten the efficiency and release translated versions without any time lag, we can prevent pirated manga translations from emerging. One pirate translation group has already announced that they will no longer produce titles that have been translated and released through our system,” notes Ishiwatari.
Meanwhile, barriers remain that current technologies cannot surmount. One is the fact that polishing up a translation to convey the nuances of a manga in a natural and enjoyable way is a highly creative process that still requires human skill. Since Ishiwatari and his team do not feel compelled to have translations automated 100%, they have instead focused on building a system to support the overall translation process. As the translation is carried out in the cloud through ordinary web browsers, anyone can work on a project without using any special applications. Additionally, multiple individuals can work remotely at the same time, from anywhere in the world, with feedback and confirmation given through the system itself.
ISHIWATARI Shonosuke co-founded Mantra after acquiring his doctorate from the University of Tokyo’s Graduate School of Information Science and Technology. He first became conscious of the power of manga during his childhood spent in China. “Japanese anime and manga were the center of conversation among the local children. That gave me a sense of how powerful this culture could be, going beyond national borders to bring people together simply for the enjoyment it provided.”
In 2021, the system was utilized in the translation of a major Japanese publisher’s popular manga series, carried out by enthusiastic overseas fans for simultaneous release in English and Japanese. According to Ishiwatari, “The cloud-based system turned out to be perfect for cross-border collaboration. In the future, it may even be possible for fans worldwide to get involved in the translation of various titles.” This is the future that he has dreamed of. “In addition to being a technological pursuit, we originally launched Mantra because we also wanted to use AI to go beyond national borders and deliver a wonderful culture across the globe. We believe that the world will become a little more peaceful if manga can encourage further international exchange. That is how we want to contribute to the global society.”
