Real-time biomechanical pose estimation and rep analysis. An interactive, browser-based system that pairs Google MediaPipe Pose with finite-state joint trigonometry for skeletal tracking, form validation and automated repetition counting.
Real-time landmark inference via WebGL GPU acceleration
3D coordinate mapping across every major joint axis
No video streaming or server dependencies, for privacy
Runs purely on a commodity browser webcam feed
Investigating how client-side edge computer vision can remove the barrier to injury-free resistance training, without specialised sensors.
The challenge
Improper exercise kinematics in resistance training frequently lead to musculoskeletal injury and inefficient neuromuscular recruitment. Beginners often lack access to qualified supervision or immediate feedback on their form.
33 full-body 3D landmarks at 30+ frames per second. Inter-joint vectors are computed in real time to detect eccentric and concentric phases and evaluate depth and range of motion.
Inference runs client-side on WebAssembly and WebGL. No raw video frames or biometric telemetry are ever sent to a remote server.
No app to install and no account to create. Open a model and lift.
Prop a phone or laptop a few steps away, roughly at hip height.
Keep the shoulders, hips, knees and ankles the model measures in view.
Joint angles stream live. A rep only counts once you reach full range of motion.
A deterministic four-stage pipeline that turns raw webcam video into quantified repetition counts and posture cues.
Captures the camera stream through the HTML5 WebRTC API and buffers live frames into a hidden video element at the device’s native refresh rate.
A lightweight convolutional pose landmarker infers 33 key landmarks per frame: normalised X, Y, Z coordinates plus a visibility confidence score.
Joint angles between adjacent segments (shoulder–elbow–wrist, hip–knee–ankle) come from the two-argument arctangent, mapped to degrees and smoothed.
Each rep must pass through every phase in order. A repetition is counted only once its anatomical thresholds have all been met.
Open any model to start the live camera, watch the landmark overlay and test rep detection in real time.
Hip–knee–ankle flexion verifies full depth and a complete standing lockout.
Bilateral elbow extension enforces a full touch-and-press range of motion.
Hip-hinge mechanics count reps, while a spine-angle check flags a rounding back.
Vertical head-to-wrist clearance confirms a dead hang and a chin-over-bar finish.
Left and right elbows are tracked independently for strict alternating curls.
Both arms are averaged to follow the press from shoulder height to overhead.
Open the full Exercise Vision Lab for every tracker, plus the 1RM calculator and strength standards.