Application of AI in the analysis of human movement
17 December 2025.
Autor/es: Mario Lamas Rodríguez, Juan López Pascual, Eduardo Parrilla Bernabé, Francisco Parra González, Ignacio Bermejo Bosch, Beatriz Fernández Gallo.
Instituto de Biomecánica (IBV)
Human motion capture is a fundamental tool in medicine, rehabilitation, ergonomics and sport, as it makes it possible to describe quantitatively how people move. Markerless technology is increasingly being proposed as an alternative for motion analysis due to its shorter set-up time and lower computational cost. However, the available pose-detection algorithms do not provide sufficient information to carry out a biomechanical analysis, preventing these advantages from being fully exploited. In this context, there is a need to develop a tool that can leverage the strengths of this technology within the field of biomechanics.
The aim of this article is twofold: first, to explain how artificial intelligence (AI), and in particular deep-learning algorithms, are being used in markerless systems to record movement in a simpler and more user-friendly way; and second, to present Markerless IBV, a tool capable of performing biomechanical analysis using this technology, thereby maximising its benefits.
INTRODUCTION
Markerless systems estimate body position and orientation using cameras and pose-detection algorithms. Unlike conventional photogrammetry, which requires reflective spheres to be placed on the skin, a markerless system uses neural networks trained on thousands of labelled images to automatically detect anatomical landmarks.
These systems offer a number of advantages over photogrammetry, positioning themselves as a genuine alternative for human movement analysis. First, they remove the need to instrument the subject and the time associated with placing markers on specific anatomical landmarks. This streamlines session preparation and improves participant comfort, while also eliminating the need for an experienced assessor to locate specific anatomical regions and the associated error. Second, capture is performed in real time, with no need for manual marker identification, and at lower cost. As they rely solely on cameras and algorithms, hardware requirements are reduced and the system can run on mid-range computers. In addition, because they do not depend on markers, it is possible to analyse movements in everyday clothing, in settings other than the laboratory, or outdoors, opening the door to assessments in more realistic environments.
However, these systems depend heavily on how the neural networks have been trained. Training is carried out by providing labelled examples (see Figure 1) from which the networks can define patterns that can then be used to label new images. This “example-based learning” enables the algorithm to automatically label future postures [1]. Therefore, the performance of pose detection by these networks depends largely on the datasets used for training. Many available image datasets, and the point models they include, were designed for animation or computer vision rather than biomechanical applications. These datasets label only a subset of joints (hip, knee, ankle, shoulder) and treat whole segments as rigid structures, which leads to errors when estimating joint centres and segment lengths. In addition, labelling is often performed by personnel without anatomical training, is not subject to a second verification, and requires points to be marked even when they are not visible, resulting in inconsistencies and systematic errors [1].
Figure 1. Examples of labelled images from the databases.
Examples of pose-detection algorithms include OpenPose and MediaPipe (see Figure 2). OpenPose [2] was one of the first open-source algorithms capable of detecting multiple people from a single image and estimating 18 or more body points per individual. This convolutional neural network uses part affinity fields to associate the joints of each person. OpenPose’s accuracy has enabled its adoption in sport and animation, but the network has a high computational cost. In addition, including hand and face detection increases memory consumption and reduces processing speed.
Figure 2. Detection of anatomical landmarks using the OpenPose detection algorithm (left) [2] and MediaPipe (right).
MediaPipe [3] is a Google library designed to run pose-detection models on mobile devices. It uses a lighter network than OpenPose that directly predicts the coordinates of 33 keypoints. MediaPipe is optimised for devices with limited CPU and GPU resources and is open source, which facilitates its implementation across a wider range of applications.
Both algorithms enable 2D pose detection, but they can be extended to 3D, enhancing their capabilities. By arranging two or more cameras within a calibrated space and applying a pose-detection algorithm, it is possible to obtain 3D points, which can then be used for biomechanical analysis of complete movement tasks.
However, one limitation of these algorithms is that they are not designed for detailed biomechanical analyses. In the specific case of OpenPose and MediaPipe, they detect between 18 and 33 points, which is insufficient to define complete anatomical axes, and their datasets are not intended for biomechanics [1]. Consequently, to obtain joint angles across all three axes it is necessary to apply a marker augmentation process [4], i.e. increasing the number of markers by combining the detected points with anthropometric models (see Figure 3). Therefore, there is a need to develop a tool that enables this information to be extracted in order to perform the required analysis.
Figure 3. Marker augmentation based on the points obtained by the pose-detection algorithm [4].
DEVELOPMENT
MARKERLESS IBV
To overcome the above limitations, Instituto de Biomecánica (IBV) has developed Markerless IBV, a tool capable of performing biomechanical analysis using markerless technology. This tool is based on Move4D, a dynamic scanner comprising 12 to 16 photogrammetric modules with infrared cameras and integrated lighting, capable of generating sequences of 3D meshes without markers with accuracy comparable to photogrammetry [5], [6].
This tool is based on the definition of a parametric human model [7] that makes it possible to assign each subject an avatar. This avatar (see Figure 4) consists of a 50,000-point mesh linked to an internal skeleton of 63 joints, defining the subject’s shape (body dimensions and volume) and pose (the position and orientation of each segment).
Figure 4. Schematic of the parametric human model [7]. It consist of a mesh of 49,530 vertices and an internal skeleton with 63 points.
Defining this avatar involves two stages: defining the shape and then fitting the pose and shape to the performed movement. First, the pose is determined from an A-pose of the measured subject, from which MediaPipe points are extracted. Using principal component analysis (PCA) and multivariate regression techniques, an approximate shape is calculated from basic anthropometric variables (age, sex, weight and height), based on a database of more than 20,000 subjects in A-poses measured with Move4D (see Figure 5). The estimated shape is then fitted to the 3D markers obtained with MediaPipe, refining the model’s shape and fitting it to the A-pose (see Figure 6).
Figure 5. Application of PCA to obtain the model shape that best matches the provided anthropometric measurements (weight, height, sex and age) [7].
Figure 6. Model fitting based on the 3D marker sequence from the 3D pose and the patient’s anthropometric data.
Once the subject’s shape has been defined, the next step is to fit the pose and refine the shape. The initial avatar is deformed to match the 3D markers obtained with MediaPipe, supported by two datasets containing measurements from men and women performing a series of movements. Joint constraints are imposed to maintain anatomical consistency (see Figure 7). This yields the final mesh and internal skeleton that describe the exact posture of the measured movement (see Figure 8).
Figure 7. Unrealistic deformation of the avatars after fitting them to the performed movement (left), and correction of this deformation through the application of these constraints.
Figure 8. Capture of a subject’s gait and inference of the 33 MediaPipe points (top). Inference of the 33 3D points de una marcha del sujeto y su inferencia de los 33 puntos de mediapipe (arriba). Inferencia de los 33 puntos 3D a partir de las 12 cámaras.
Once the parametric mesh has been generated, each vertex can be associated with one of the markers of a previously defined biomechanical model, obtaining the spatial distribution of the markers throughout the movement (see Figure 9). This makes it possible to calculate joint angles, segment trajectories and other biomechanical variables of interest for the analysis.
Figura 9. Integración de los modelos biomecánicos en el avatar.
CURRENT APPLICATIONS OF THE TECHNOLOGY
The implementation of Markerless in Kinemov has enabled exploration of this measurement system in areas such as sport, ergonomics and bioengineering. The greater versatility of this technology allows for a wider range of clothing, making it possible to recreate more realistic scenarios.
In the context of sport, a biomechanical analysis has been carried out on a cyclist using an indoor trainer. Using this technology, the aim has been to analyse the cyclist’s technique, efficiency, performance and injury risk.
Figure 10. Biomechanical analysis of a cyclist on an indoor trainer.
In the field of ergonomics, the system enables the simulation of movements performed during work activities and the assessment of the associated risks. This technology makes it possible to conduct ergonomic analyses in more realistic scenarios, allowing potential risk factors present in workplaces to be identified.
Figure 11. Ergonomic analysis of a worker at their workstation.
Another application of this technology is the functional assessment of patients. By analysing the gait of a patient with hemiparesis using a markerless approach, differences have been detected between the unaffected side and the pathological side, highlighting the potential of this technology as a diagnostic tool in functional assessment.
Figure 12. Biomechanical analysis of a gait study in a patient with hemiparesis.
If the accuracy of this technology can be improved through more robust neural networks, it could be used as a diagnostic tool and for treatment follow-up in the field of functional assessment. In addition, its faster analysis compared with the traditional method would enable more immediate evaluation of treatment, allowing a more responsive approach and accelerating the patient’s recovery. Therefore, provided it complies with the Medical Device Regulation, a clinical medical device for functional assessment could be developed using this technology.
One of the advantages of this technology is the possibility of carrying out measurements outside the laboratory, enabling the democratisation of biomechanical analysis.
One of the studies conducted using the Markerless solution in Kinemov took place at a spa, where users were assessed with the aim of analysing their functional status. Unlike traditional methodology, the speed of data acquisition and greater flexibility improves adherence to the technology.
Figure 13. Biomechanical analysis of gait in a non-laboratory environment.
NEXT STEPS
The current system for pose detection and definition of the HO-Model (Human-Object Model) in the markerless system uses the MediaPipe network. This network stands out for being free for commercial use and very lightweight, enabling its use on mobile devices. However, it is not a particularly robust or reliable network, which limits the development of sophisticated applications in the field of biomechanics.
To overcome the limitations of third-party solutions, a proprietary, specialised pose-detection neural network is being developed. This network aims to surpass the robustness and reliability performance of alternatives such as MediaPipe and OpenPose. This new network will enable automatic and optimised labelling using the Move4D database.
Implementing this pose-detection network will achieve significantly higher accuracy in pose estimation, as it is designed and trained specifically for the project’s objectives, eliminating dependence on generic third-party networks. In addition, it will provide greater versatility by supporting different types of input data, including, in addition to RGB images, LiDAR depth maps and thermal camera information.
CONCLUSIONS
AI is emerging as an alternative to photogrammetry for the analysis of human movement. Markerless systems make it possible to record movement without instrumenting the subject, reducing costs and improving comfort. However, the most widely used pose-detection algorithms, such as OpenPose and MediaPipe, do not provide enough points to enable detailed biomechanical analyses. In addition, their performance depends on datasets that are not designed to reflect the diversity of movements and populations.
To address these limitations, Markerless IBV combines the Move4D scanner, dense meshes and parametric models trained on anthropometric data. This solution generates an avatar comprising dozens of joints and enables the calculation of kinematic variables comparable to those obtained with a marker-based system.
IBV is working on implementing this technology across a range of applications and on certifying them as a medical device, which will open the door to its use in rehabilitation, research and sports training. In the not-too-distant future, health and sports professionals are expected to have access to this technology to obtain faster, more objective and more comfortable assessments.
BIBLIOGRAPHY
[1] L. Wade, L. Needham, P. McGuigan, and J. Bilzon, ‘Applications and limitations of current markerless motion capture methods for clinical gait biomechanics’, PeerJ, vol. 10, p. e12995, Feb. 2022, doi: 10.7717/peerj.12995.
[2] Z. Cao, G. Hidalgo, T. Simon, S.-E. Wei, and Y. Sheikh, ‘OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields’, May 30, 2019, arXiv: arXiv:1812.08008. doi: 10.48550/arXiv.1812.08008.
[3] C. Lugaresi et al., ‘MediaPipe: A Framework for Building Perception Pipelines’, Jun. 14, 2019, arXiv: arXiv:1906.08172. doi: 10.48550/arXiv.1906.08172.
[4] S. D. Uhlrich et al., ‘OpenCap: 3D human movement dynamics from smartphone videos’, Jul. 10, 2022, bioRxiv. doi: 10.1101/2022.07.07.499061.
[5] A. V. Ruescas Nicolau, H. De Rosario, F. Basso Della-Vedova, E. Parrilla Bernabé, M.-C. Juan, and J. López-Pascual, ‘Accuracy of a 3D temporal scanning system for gait analysis: Comparative with a marker-based photogrammetry system’, Gait Posture, vol. 97, pp. 28–34, Sep. 2022, doi: 10.1016/j.gaitpost.2022.07.001.
[6] A. V. Ruescas-Nicolau, H. De Rosario, E. P. Bernabé, and M.-C. Juan, ‘Positioning errors of anatomical landmarks identified by fixed vertices in homologous meshes’, Gait Posture, vol. 108, pp. 215–221, Feb. 2024, doi: 10.1016/j.gaitpost.2023.11.024.
[7] E. Parrilla, A. Ballester, J. Uriel, A. V. Ruescas-Nicolau, and S. Alemany, Capture and Automatic Production of Digital Humans in Real Motion with a Temporal 3D Scanner. The Eurographics Association, 2024. Accessed: Oct. 15, 2025. [Online]. Available: https://doi.org/10.2312/cl.20241048
AUTHOR’S AFFILIATION
Instituto de Biomecánica de Valencia
Universitat Politècnica de València
Edificio 9C. Camino de Vera s/n
(46022) Valencia. Spain
HOW TO CITE THIS ARTICLE
Author/s: Mario Lamas Rodríguez, Juan López Pascual, Eduardo Parrilla Bernabé, Francisco Parra González, Ignacio Bermejo Bosch, Beatriz Fernández Gallo. (17 December 2025). «Application of AI in the analysis of human movement”. Revista de Biomecánica nº 72. https://www.ibv.org/actualidad/move4d-la-tecnologia-de-escaneado-dinamico-que-cuantifica-el-efecto-de-las-prendas-de-compresion-en-corredores/
