Sony Patents In-Game Sign Language Translator

Sony is taking a significant step towards enhancing accessibility for deaf gamers with a newly filed patent aimed at providing real-time in-game sign language translation. The patent, titled "TRANSLATION OF SIGN LANGUAGE IN A VIRTUAL ENVIRONMENT," details a system that can translate American Sign Language (ASL) into Japanese Sign Language (JSL) and vice versa, enhancing communication within video games.

Sony Patents ASL to JSL Translator for Video Games

Proposed to Use VR Devices and Work Over Cloud Gaming

Sony's patent introduces a groundbreaking approach to assist deaf gamers by enabling real-time translations of sign languages during in-game interactions. The technology described in the patent involves virtual indicators or avatars that appear on-screen to perform sign language in real-time. The process begins with translating the sign gestures of one language into text, followed by converting that text into another specified language, and finally transforming the data into sign gestures of the target language.

"Implementations of the present disclosure relate to methods and systems for capturing sign language of one user (e.g., Japanese), and translating the sign language to another user (e.g., English)," Sony explained in the patent. "Because sign languages vary depending on geographical origins, sign language is not universal. This provides a need for appropriately capturing the sign language of one user, understanding the native language, and generating new sign language as output for another user in their native sign language."

Sony envisions the system being implemented using VR-type devices or head-mounted displays (HMDs). "In some implementations, the HMD connects via a wired or wireless connection to a user device, such as a personal computer, game console, or other computing device," Sony detailed. "In some implementations, the user device renders graphics for display through the HMD that provide immersive viewing of the virtual environment for the user."

The company further proposes that user devices can seamlessly communicate with each other over a network, facilitated by a game server. "In some implementations, the game server executes a shared session of a video game, maintaining the canonical state of the video game and its virtual environment," Sony said, "and to which the user devices are synchronized regarding the state of the virtual environment."

This setup allows users to interact within the same virtual environment, or game, over a shared network or server. Sony also mentioned that in some implementations, the game server could be part of a cloud gaming system, which "renders and streams video" between each user device, enhancing the overall gaming experience for deaf players by ensuring smooth and real-time communication.