Automatic repetitive action counting for construction worker ergonomic assessment

Work-related musculoskeletal disorders (WMSDs) are a significant cause of nonfatal occupational injuries in the construction industry [1]. The prevalence of WMSDs is very high among construction workers worldwide. For example, in the United States, the prevalence of musculoskeletal symptoms among construction workers is approximately 34 % [2]. In Hong Kong, it is 41 % [3], in Ethiopia, it is 36 % [4], and in Malaysia, it is 76 % [5]. In addition, challenges such as an aging workforce, economic compensation and wage burden, and productivity loss have also become new challenges facing the construction industry [6]. Evaluating the ergonomics of construction workers can reduce the risk of WMSDs, protect worker health, improve productivity, and reduce economic and labor pressures [7,8]. Therefore, it is necessary to supervise and evaluate the ergonomic status to achieve the goal of sustainable development of construction labor.

Worker ergonomic assessment and interventions can effectively prevent work-related injuries. These assessment methods, such as Rapid Entire Body Assessment (REBA) [9], Rapid Upper Limb Assessment (RULA) [10], and occupational repetitive actions (OCRA) [11], evaluate the risk of WMSDs by considering factors such as worker posture, external load, and action frequency. Existing research focuses primarily on automated posture and load assessment. For example, visual methods or Inertial Measurement Units (IMUs) sensors are used to estimate workers' 3D posture data for awkward posture analysis, and smart pressure insoles are used to assess load [2,12]. Research on the incidence of musculoskeletal disorders indicates that repetitive and rapid workloads are one of the primary causes of musculoskeletal injuries [13], and many ergonomic evaluation methods have specific criteria for evaluating repetitive action. For example, REBA and RULA use the frequency of repetitive actions as a key standalone criterion to evaluate ergonomic risks to workers, while OCRA conducts ergonomic assessments based on the frequency of repetitive actions. The lack of repetitive action data can result in incomplete assessments for REBA and RULA and make OCRA assessments impossible to conduct. However, in existing studies, the repetitive count component does not have an automated approach and still relies on traditional manual observation or is not evaluated at all [14,15]. Manual observation results are subjective, and different observers may provide different evaluations for the same repetitive motion segment. Previous studies have shown that the error rate in manual identification of repetitive motions and movement frequency exceeds 30 %, leading to inaccurate estimation of WMSD risk levels for workers [13]. Therefore, in order to achieve automated and accurate WMSD risk assessment, an automated method for repetitive motion counting should be established.

Traditional methods for analyzing repetitive motion used fixed features to model specific actions [16,17]. However, these methods were not effective for different repetitive motion patterns. Other studies transformed image features into one-dimensional time-domain curve models and used time-domain peak detection repeat action or calculated self-similarity in fixed frames to compute repetitive actions [[18], [19], [20], [21]]. However, these methods failed when dealing with motion features affected by interference or dynamic backgrounds, and they performed poorly on sequences of different lengths (different repetition frequencies). In conclusion, the model based on fixed features is ineffective for various types of repetitive construction tasks, and repetition counting methods based on images, and time-domain signals cannot be reliably applied to construction scenes with dynamic backgrounds. In addition, these methods cannot solve the problem of repetitive counting for sequences of different lengths. These challenges have prompted us to seek a motion repetition counting approach that can be applied to various motion patterns of construction workers.

For accurate and automated ergonomic evaluation of repetitive movements, this study developed an automated method for Counting the Repetitive Actions of construction workers (CRA). This method includes a 3D posture estimation module, a similarity comparator, and a transform-based repetitive action counting model. Firstly, we employ a 3D pose estimation model to extract the 3D joint coordinates from the videos of construction workers. Subsequently, we use a self-similarity matrix to encode 3D pose. By calculating the self-similarity matrix of each pose feature in the entire sequence, we can distinguish if an unstructured action is repetitive. Finally, we use a classifier to determine whether the action is repetitive. This study performs repetitive action counts using 3D human data to avoid the interference of a dynamically chaotic construction background. Also, a transform layer and a fully connected layer are used to make regression predictions on the length of the action features. The transform model is used to handle actions with variable frequencies in the action features, improving the prediction accuracy of the length of variable frequency repetitive actions. By counting the repetitiveness of workers' actions based on 3D coordinates in the video, it can automatically and accurately evaluate the part about repetitive actions in ergonomics. The code and datasets can be accessed at the following GitHub repository: https://github.com/Chenxy875/Repetitive-Actions-Counting-Method-for-Construction-Workers-Ergonomic-Assessments.git.

Worker ergonomic assessment in construction

Worker ergonomic assessment methods, such as Rapid Upper Limb Assessment (RULA), Occupational Repetitive Action (OCRA), and Rapid Entire Body Assessment (REBA), evaluate WMSDs risk levels of construction workers based on working postures, external load, repetitive action, and duration. Researchers initially analyzed and evaluated worker ergonomic assessment by manually observing videos, such as using OCRA to assess the upper limb risk of painters and mortar workers using different tools [22,23

Methodology

To overcome these challenges, a repetitive action discrimination and prediction model is proposed, leveraging self-similarity degree and skeletal model to judge repetitive actions, and using a transform model to predict the no-fixed length of each repetitive action. By combining the cycle length prediction results with the action judgment results, we can obtain the final repetitive cycle.

We propose an automated method for Counting the Repetitive Actions of construction workers (CRA). This

Experimental setup

The 3D worker posture estimator uses a trained model from an existing study to estimate posture data [46]. Therefore, this experiment discusses only the training of the embedding on self-similar matrix comparator and repetitive counter. The following sections will detail experimental data, the environment, parameters, and evaluation metrics.

Specific case of ergonomics assessment for construction worker

To demonstrate the specific application of our method in ergonomics assessment, we selected a set of on-site data for evaluation using RULA, OCRA, and REBA, as shown in Fig. 12. In these cases, the first row shows the input video at various moments, the second row presents the 3D posture data estimated by our method, and the third and fourth rows show the ergonomic risk assessment scores and levels in RULA, OCRA, and REBA with/without our method. During the evaluation, posture scores based on

Conclusion

Ergonomic evaluation methods can effectively prevent fatigue injuries related to work. Repetitive work is also a major cause of musculoskeletal injuries, and many ergonomic evaluation methods have specific criteria for evaluating repetitive action. However, existing research mostly concentrates on the automatic evaluation of posture and load in evaluation indicators. The existing automatic ergonomic evaluation methods lack research on repetitive actions, and the recognition of repetitive

CRediT authorship contribution statement

Xinyu Chen: Writing – original draft, Visualization, Validation, Formal analysis. Yantao Yu: Writing – review & editing, Supervision, Resources, Project administration, Methodology, Investigation, Funding acquisition, Data curation, Conceptualization.

Declaration of competing interest

YU, Yantao reports financial support was provided by National Natural Science Foundation of China. YU, Yantao reports financial support was provided by University Grants Committee Research Grants Council. YU, Yantao reports financial support was provided by Hong Kong Chief Executive's Policy Unit. If there are other authors, they declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This work was supported by the National Natural Science Foundation of China [grant number 72201226], the Early Career Scheme from the Research Grants Council (Hong Kong) [grant number 26208323], and the Public Policy Research Funding Scheme of The Government of the Hong Kong Special Administrative Region [grant number 2023.A7.030.23C].