With most of us holed up in our homes and coordinating work over video calls due to the COVID-19 pandemic, you might have become well-acquainted with the variety of video conferencing software. A great feature of these video calling apps is automatic switching between video feeds of the person talking in real-time. This, however, doesn’t work with sign language users and they could feel left out of the conversation.
Now, if you are wondering how this sign language detection engine works then Google has explained it all in detail. First, the video passes through PoseNet, which estimates the key points of the body such as eyes, nose, shoulders, and more. It helps the engine create a stick figure of the person and then compare its movements to a model trained with the German Sign Language corpus.
