AI Virtual Mouse

A real time motion capture and analysis system designed as foundational infrastructure for sign language translation and accessibility technology. Originally built for gesture-based cursor control, this project evolved into a full motion interpretation pipeline with 12 AI features including continuous CSLR, fingerspelling A-Z, HID output, and a Vision Transformer architecture. 142 tests passing, deployed on Render.

Challenges

• 1Building a real time pipeline that processes 32 hand gesture combinations at 15 FPS over WebSocket
• 2Implementing dual classification: rule based (32 combos) and ML based (RandomForest with Platt calibration)
• 3Extending gesture recognition into sign language infrastructure with CTC/attention decoding and DTW template matching

Outcomes

• ✓12 AI features integrated: handedness, two-hand detection, continuous CSLR, fingerspelling, HID output, NLP translator
• ✓142 tests passing with per module coverage
• ✓Live demo deployed on Render with browser webcam support