What is signspeak?

SignSpeak is a web application that turns hand gestures into spoken English in real time. It was built to help people who are non-verbal or who have speech impairments communicate more easily, using a webcam, computer vision, and AI language models rather than requiring specialized hardware. The app watches a user's hand through their device's webcam directly in the browser, tracking its position and shape to recognize individual gestures as words. As gestures are recognized, they build up in a buffer on screen. When the user presses the spacebar, that buffer of words is sent to a backend server, which uses an AI language model to turn the raw sequence of words into a natural, grammatically correct sentence. That sentence is then converted into an audio file and played back out loud. Under the hood, the project is split into two parts. The frontend, built with React and TypeScript, handles the webcam and hand tracking entirely on the user's device using a library for real-time hand landmark detection, so gesture recognition happens instantly without needing to send video over the network. The backend, built with Python's FastAPI framework, receives the recognized words and sends them to two different AI providers at the same time, using whichever one responds first, with a short timeout as a safety net. It then converts the resulting sentence into speech using a text-to-speech tool. The author describes several real situations this could help with: giving a voice to people with conditions such as ALS, cerebral palsy, or vocal cord damage, allowing communication in very loud environments where speaking isn't practical, and helping patients in intensive care express basic needs like food, water, or pain relief to their caregivers. Running it locally requires both Python and Node.js, along with API keys for the AI services it depends on.