[PDF]

Intelligent Fitness System: Integrating Advanced Action Recognition and Natural Language Processing for Personalized Guidance and Supervision


Aohan Li

04/12/2023

Supervised by Sylwia Polberg; Moderated by Víctor Gutiérrez Basulto

In today's society, with increasing awareness of health, more and more people are focusing on fitness exercises. Especially for beginners, scientific guidance and reasonable supervision are crucial due to a lack of fitness foundation. To meet this demand, we have developed an intelligent fitness system aimed at providing users with comprehensive fitness assistance services. The main functions of the intelligent fitness system are divided into two parts: human motion recognition and fitness training plan generation: In terms of human motion recognition, we built a rich dataset of fitness action videos. By training Faster R-CNN, HRnet, and PoseC3D models, we use the combination of these three models for action recognition. Specifically, we first use Faster R-CNN for target detection, then extract human keypoints with HRnet, and finally, based on the outputs of the previous models, use PoseC3D to accurately recognize human movements. Experimental results show that the accuracy of this combined model reaches 91.06%. For fitness training plan generation, we leverage chatGPT3.5-turbo and, based on the LangChain framework, generate personalized fitness training plans according to user requirements. By invoking chatGPT3.5-turbo, we achieve intelligent matching between user needs and fitness training plans, providing users with personalized and scientific exercise recommendations. And utilize the trained action recognition model to identify user actions, recording them in the generated plan, thereby completing supervision of the user. In summary, our intelligent fitness system combines advanced action recognition technology with natural language processing capabilities to provide users with comprehensive and personalized fitness guidance and supervision. The system aims to enhance users' health experience and exercise effectiveness. In the future, we will continue to expand the dataset, optimize models, and introduce more intelligent features to continuously improve the system's performance and user experience.


Final Report (04/12/2023) [Zip Archive]

Publication Form