LiftTrack: A 3D CNN-Based Mobile Solution for Monitoring and Improving Weightlifting Techniques
3D CNN Mobile Form Correction + Flutter
Overview
Role: Flutter Frontend & Architecture Lead
Stack: Flutter, Dart, BLoC, 3D CNN, Python
Key Result: 100% Deadlift Form Accuracy
Duration: 8 months (July 2024 - February 2025)
LiftTrack is a research-backed mobile application designed to democratize professional weightlifting coaching. By integrating advanced computer vision models into a Flutter application, the system provides real-time biomechanical analysis and corrective feedback to prevent injuries. This 8-month capstone project focused on developing a high-performance solution for real-time weightlifting form monitoring using 3D CNN and Pose Estimation.
The Problem
Challenge: High injury risk in lifting due to poor form and lack of professional coaching.
Audience: Solo gym-goers and athletes who lack access to expensive personal trainers.
Constraints: Real-time inference on mobile devices, handling complex temporal movements versus static poses, and maintaining high frame rates for immediate feedback.
Solo lifters often struggle with maintaining proper form, leading to chronic injuries or suboptimal gains. Traditional pose estimation often fails to capture the "rhythm" and temporal sequence of a lift, which is critical for identifying technical flaws in movements like the deadlift or squat. The lack of an affordable, portable solution for form analysis creates a significant barrier to safe and effective training.
Key Decisions: Combined 3D CNN (Temporal) for movement phase identification with MoveNet (Pose) for skeletal analysis and YOLO-NAS (Equipment) for bar path tracking.
State Management: Applied Clean Architecture with BLoC to manage 3 parallel data streams without UI lag.
Trade-offs: Remote cloud inference was chosen to maintain 30+ FPS on mid-range devices while processing heavy 3D CNN tensors.
The system utilizes a custom data flow: Camera -> 30-Frame Buffer -> Parallel Inference -> Feature Aggregation. This ensures that the application doesn't just see a "snapshot" of a pose, but understands the entire movement arc. By decoupling the presentation layer from the heavy inference logic, we achieved a smooth user experience even during intensive computation.
Key Results & Impact
Outcome: Validated 95% general accuracy across core lifts, reaching 100% on specific exercises like the Deadlift.
Impact: Created a low-cost, accessible alternative to personal training, providing professional-grade form correction to anyone with a smartphone.
Performance: Achieved a sub-100ms inference feedback loop, maintaining 30+ FPS on recommended devices.
Exercise
Accuracy
Deadlift
100.00%
Romanian Deadlift
83.33%
Bench Press
80.00%
Overall Average
~95%
Technical Deep Dive
Parallel Multi-Modular Inference Pipeline
3D CNN (Temporal Classification): A 3-dimensional convolutional neural network processes a buffer of 30 frames to identify exercise type and movement stage.
Architecture: Input (30x128x128x3) → 2x Conv3D (32, 64 filters) → 3D Max Pooling → Global Average Pooling → Softmax.
MoveNet (Human Pose Estimation): Extracts 17 body keypoints per frame to calculate joint angles and body alignment.
YOLO-NAS (Equipment Tracking): Detects and tracks barbells/dumbbells to calculate displacement, speed, and bar path.
Key Optimizations
Normalization: Body keypoints are normalized using the user's torso length as a reference vector, ensuring accuracy across different body types and camera distances.
Heuristic Evaluation: Once the 3D CNN identifies the exercise (e.g., Deadlift), the system applies specific biomechanical thresholds (e.g., back angle > 150°) to the MoveNet/YOLO data.
Feedback Loop: Audio and visual feedback are generated within milliseconds, outputting specific suggestions like "Keep your back straight" or "Lower your hips."
Reflections
Success: The clean BLoC architecture successfully managed complex state and real-time data streams, proving that Flutter can handle intensive ML-integrated applications.
Future Directions: Exploring AR overlays for more intuitive spatial guidance and expanding the training set to 50,000+ images for broader inclusivity.
Challenges: Navigating iOS development barriers and cross-platform library limitations required custom implementations. The project faced hardware failures and curriculum delays, which we navigated through agile adjustments and infrastructure improvements.
My Contributions
Flutter Frontend (iOS/Android): Built the cross-platform mobile interface using Dart and Flutter.
Architecture: Applied Clean Architecture with BLoC for scalable, maintainable code.
Core Features: Engineered core functionality for workout logging, progress tracking, and authentication.
UI/UX Design: Designed interactive interfaces with emphasis on clarity and feedback.
iOS Compatibility: Resolved platform-specific issues and implemented alternatives to Android-only Flutter packages.
Refactoring: Led the architectural refactor of the Profile module to improve separation of concerns.
Cloud Infrastructure: Assisted in deploying the 3D CNN model to a remote cloud server due to high compute requirements.
DevOps Setup: Integrated frontend with backend ML services using a custom deployment pipeline.
Project Architecture
This app follows Clean Architecture with modular layers and feature-based organization.
Layered Structure: Presentation, Domain, and Data layers are strictly separated. Read about it
Feature Modules: Each major feature (e.g., Profile, Progress, Workout Logging) is independently structured.
BLoC State Management: Ensures predictable state transitions and testable business logic.
Recent Improvements: Profile module refactored to comply with architectural goals. Refactoring Notes
Demo Preview
While the live version is unavailable, here are core views from LiftTrack:
Workout logging interface on Android and iOS
Progress tracking and dashboard
Cloud ML pipeline (architecture diagram available)
Hosting Note:
This project was deployed on a cloud server managed by the project manager. Due to expired hosting and access limitations, the live app is currently offline. A full demo and recorded walkthrough are available upon request.
