Abstract Ergonomics studies encompass a lot of researches aimed to the evaluation of the discomfort and of the suitability of a tractor cabin at different working conditions. In the present study for driver movements analysis we propose implementation of the Kinect sensor, a low-cost infrared depth camera allowing markerless skeletal tracking, both in standing or seating position. Such sensor allows generation of a stick skeleton mode, extracting and tracking skeletal joint locations (two for each hand, two for the head, four for shoulders and arms, etc.) at a sampling rate of 10 positions per second. The tracking system was developed by means of the Microsoft software development kit (SDK), and allowed tracking of body movements of a tractor driving person for several minutes. In order to test the applicability of the sensor, it was applied for monitoring two different drivers (an expert and a beginner) on two different paths taking advantage of a 81 kW tractor allowing assisted steering. Tests highlights how the sensor can be successfully implemented for quantification of drivers body movements

Andrea Pezzuolo

Francesco Marinello

Franco Gasparini

Luigi Sartori

CiteSeerX

 
 
 
Proceedings International Conference of Agricultural Engineering, Zurich, 06-10.07.2014 – www.eurageng.eu    1/5 
 
 
Ref: C0253 
 
Ergonomics analyses through motion capture in a vehicle 
cabin by means of Kinect sensor 
Francesco Marinello, Andrea Pezzuolo, Franco Gasparini, Luigi Sartori, University of Padova 
- Department TESAF, Viale dell'Università 16, I-35020 Legnaro (Padova) 
Abstract 
Ergonomics studies encompass a lot of researches aimed to the evaluation of the discomfort 
and of the suitability of a tractor cabin at different working conditions.  
In the present study for driver movements analysis we propose implementation of the Kinect 
sensor, a low-cost infrared depth camera allowing markerless skeletal tracking, both in 
standing or seating position. Such sensor allows generation of a stick skeleton mode, ex-
tracting and tracking skeletal joint locations (two for each hand, two for the head, four for 
shoulders and arms, etc.) at a sampling rate of 10 positions per second. 
The tracking system was developed by means of the Microsoft software development kit 
(SDK), and allowed tracking of body movements of a tractor driving person for several 
minutes.  
In order to test the applicability of the sensor, it was applied for monitoring two different driv-
ers (an expert and a beginner) on two different paths taking advantage of a 81 kW tractor 
allowing assisted steering. Tests highlights how the sensor can be successfully implemented 
for quantification of drivers body movements.  
Keywords: ergonomics, tractor, real time, sensor, Kinect 
1. Introduction  
Agricultural operations are typically characterized by an inherent variability, consequently 
causing a lot of physical and mental stress upon operators. In particular when driving trac-
tors, the body undergoes to severe physiological demands due to frequent movements 
needed by steering (arms and shoulders), looking forward, backward and sideways to control 
implements and monitor operations (head, neck, shoulders and back), operating clutch, 
brake and other control levers (arms), etc. Furthermore driver fatigue is extremely influenced 
by the specific cabin layout, being minimized in those vehicles featuring an optimal dimen-
sioning and ergonomic disposition of seat and commands and implementing driving aids. As 
a consequence, stress cannot be merely quantified in terms of working time, and more accu-
rate systems have to be implemented in order to evaluate or compare driving quality of dif-
ferent machines.  
Excluding qualitative self-reporting (very much influenced by driver assessing attitude), two 
different approaches are typically proposed as a solution for fatigue monitoring.  
The first approach deals with implementation of sensors which allow real time analysis of 
physiological parameters. Physiological signals have the advantage of providing a quantita-
tive feedback on driver’s state with almost no interference with the driver’s task performance.  
Among these, electrocardiogram (for heart activity monitoring), electromyogram (to measure 
the electrical activity of muscles), chest cavity expansion (for respiration) or skin conductivity 
 
 
Proceedings International Conference of A
(to estimate sweat activity) are certainly the most common and interesting (
Healey and Picard 2005; Zhang et al. 2013)
tation of such instruments lie in the need for high competence 
pretation) and in the low sensibility to long term phenomena (sensors are responsive to short 
time variations). 
The second approach deals with integration of sensors which allow real time analysis of dri
er movements. Motion analysis is very interesting due to the fact th
term effects can be observed
but also the average motion rate and the cumulative angle or distance run by different body 
parts and joints can be monitored. Different approaches have been proposed in the past, 
mainly dealing with implementation of 
2010) or with image processing (
ters have the merit to allow accurate 
are needed for three-dimensional analysis of different b
capture cameras allow real time acquisition of positions of given markers on a body but are 
expensive and, requiring multiple point of view, are difficultly installed inside vehicle cabins. 
Motion capture devices costs 
vices from videogame industry. 
veloped by Microsoft for Xbox videogame console, for ergonomics analyses in vehicle inter
ors.   
2.  Materials and methods
2.1 The Kinect depth sensing camera technology
The present work takes advantage of depth sensing cameras technologies. Such technology 
is not new, but in the last years has raised a great interest and widespread diffusion mainly 
thanks to recent applications to the videogames industries. As a consequence
has rapidly reached a mature level, and at present depth cameras 
less are available on the market. 
Sony, Kinect developed by Microsoft
For the present study a Microsoft Kinect™ RGB
2012), which is already raising significant interest and growing attention in agr
cations for its high quality and low cost (Chéné et al., 2012; Tanke et al., 2012
al., 2013).  
This sensor includes an infrared laser emitter, an infrared camera and an RGB camera. 
construction of depth information is achieved 
tion grating split the infrared laser into a 
ed pattern is then collected by the infrared camera: 
where larger shifts values correspond to farer positions and, conversely, lower values corr
spond to positions closer to the sensor. 
A schematic view of the sensor is 
Figure 1: Schematic view of the 
analyses.  
 
gricultural Engineering, Zurich, 06-10.07.2014 – www.eurag
. The main limitations connected with impleme
(for physiological data inte
at both short and long 
. Indeed not only instantaneous movements can be qua
gyroscopes and accelerometers
Tran and Trivedi, 2010). Rotational and linear accelerom
monitoring in real time, however often multiple sensors 
ody parts. On the other hand, motion 
can be significantly reduced taking advantage of hardware d
The present study exploits motion sensing input devices d
 
 
costing as low as 100 
Some examples are the  PlayStation 4 Eye 
, or Asus Xtion PRO LIVE distributed by Asus
-depth camera was implemented (Zhang, 
by means of a triangulation process: a diffra
grid which is projected onto the scene. 
detected shifts generate a disp
 
reported in Figure 1. 
Microsoft Kinect™ RGB-depth camera implemented for e
eng.eu    2/5 
Egelund 1982; 
n-
r-
v-
ntified, 
 (Young and Hsu, 
e-
 
e-
e-
i-
, technology 
€ or 
distributed by 
.  
iculture appli-
; Marinello et 
Re-
c-
Such project-
arity map, 
e-
 
rgonomics 
 
 
Proceedings International Conference of Agricultural Engineering, Zurich, 06-10.07.2014 – www.eurageng.eu    3/5 
Using a projected and reflected image, the Kinect sensor suffers of all  the problems typical 
of optical sensors. In particular the sensor has a limited view angle (about 57 degrees) and 
does not allow tracking of hidden parts: therefore parts which exit from the field of view or 
which are laying behind some obstacle (as is the case of legs which typically lie below the 
steering wheel) are not monitored. However an interpolation algorithm can be used for hid-
den joints comprised between to visible joints or for those joints which occasionally exit the 
field of view. A second limit of the sensor is the high sensitivity to excessive sun light exposi-
tion: however it is worth noting that typical shadowing in any vehicle cabin is fully adequate 
for satisfactory motion capture. A third limit is connected to the occurrence of vibrations and 
uncontrolled displacements between the sensor and the monitored body: in fact uncontrolled 
relative shifts are read as body motions, which can distort and abnormally increase detected 
movements.  
2.2 Software analysis 
Beside Kinect, Microsoft also released a Software Development Kit (SDK) which included a 
set of robust algorithms for real time extraction of three-dimensional surfaces and silhou-
ettes. For the present work we took advantage of Kinect’s SDK to develop and code a soft-
ware tool for three-dimensional extraction and tracking of skeletal joint locations. 
Joints positions and names are summarized in Table 1.  
 
Table 1: Modelling of a driver’s body implemented into monitoring software. 
Driver Body model Human body joints 
   
0) Head 
1) Shoulder center 
2-3) Right and left shoulders 
4-5) Right and left elbows 
6-7) Right and left wrists  
8-9) Right and left hands 
10) Hip center 
11-12) Left and right hips 
15-16) Left and right ankles 
13-14) Left and right knees 
17-18) Left and right feet 
 
Each joint position is described by three coordinates x, y, z: two of them lie on a plane paral-
lel to the Kinect sensor, while the third one is perpendicular to it. Positions can be monitored 
with a sampling interval ∆t which can be arbitrarily set between 0.1 s and 10 s (i.e. with a 
frequency in the range 0.1-10 Hz). For a given joint j, movements dj(t) and speed vj(t) at a 
given time t are therefore extracted according to the following equations (1) and (2):  
 = 	 + ∆ − 	 +  + ∆ −  +  + ∆ −   (1) 
vt = ∆            (2) 
where vj(t) clearly represents the first derivative of the movements dj(t). 
Relative positions of joints can be also monitored. In particular for a given joint j comprised 
between two adjacent joints i and k, the relative angle can be quantified as follows (3):   =
   !" ∙$ !" %& !&" ∙&$ !&" %' !'" ∙'$(!'" ) !" *%& !&" *%' !'" *∙)$ !" *%&$ !&" *%'$ !'" *+ (3) 
 
 
Proceedings International Conference of Agricultural Engineering, Zurich, 06-10.07.2014 – www.eurageng.eu    4/5 
Monitoring such parameters is very interesting in order to define not only instantaneous but 
also cumulative stress. Indeed, while accelerometers can difficultly provide quantitative in-
formation on total displacements or relative movements (such as angles between arms and 
body), by means of a motion capture sensor cumulative stress in a total time T for a given 
joint can be estimated through the total distance Dj or the total angle Aj, according respec-
tively to equation (4) and (5): 
, = - ./           (4) 0 = - ./           (5) 
2.3 Measuring procedures 
The Kinect sensor together with the developed software tool can be implemented for simul-
taneous tracking of different body joints during some typical driving operations.  
In order to allow extrapolation of quantitative data, the Kinect sensor has to be installed in 
front of the driver, preferably in the vehicle cabin on the top of the front windshield. The posi-
tion has to be chosen to give the sensor the widest optical access to the driver movements 
and to avoid reduction of visibility to the driver himself. The Kinect sensor has to be tightly 
fixed, in order to enhance stability and minimize vibrations.  
A specific software was developed to collect data from the sensor and in particular to extract 
x, y and z coordinates of different joints (Table 1) with a frequency of up to 10 Hz. In particu-
lar during tractor driving positions of joints numbered from 0 to 10 could be revealed and 
monitored. Conversely, joints numbered from 11 to 18 were not revealed by the sensor, be-
cause of the presence of the steering wheel and of the dashboard hiding the corresponding 
body parts.  
3. Results and discussion 
The Kinect sensor was applied for monitoring two different drivers: an expert and a beginner 
driver. They were monitored in two different steering conditions: 
- field straight driving, where the driver had the task of keeping the tractor wheels over a 
straight reference line 
- road driving, where the driver had the task of steering the tractor along a path with curves  
Tests were repeated on a Fendt 211v tractor equipped with continuously variable transmis-
sion (nominal power 81 kW), considering both manual and assisted steering system (EZ-
Steer by Trimble). In all conditions, a standard speed condition of 3 km/h was defined. 
For the sake of brevity, here results are reported only for the right and left wrists (positions 6 
and 7) and for the head (position 0). It is interesting no note how implementation of the sen-
sor, allow estimation of quantitative data (Table 2).  
 
Table 2: Monitored movements for wrists and head. 
Driver Path Steering 
Left wrist  
[cm/s] 
Right wrist 
[cm/s] 
Head 
[cm/s] 
Expert 
Straight 
Manual 13.7 9.5 3.2 
Assisted 0.9 1.1 2.3 
Road 
Manual 8.5 7.4 3.7 
Assisted 1.0 1.2 3.0 
Beginner 
Straight 
Manual 5.4 5.8 2.2 
Assisted 1.0 1.2 2.9 
Road 
Manual 9.1 9.2 4.2 
Assisted 0.8 0.7 3.1 
 
 
Proceedings International Conference of Agricultural Engineering, Zurich, 06-10.07.2014 – www.eurageng.eu    5/5 
For our tests, the beginner tractor driver exhibits a more rigid behavior with movements 
which are in general almost the 20% lower than the expert driver, which, on the other hand 
moves arms and head frequently to put the tractor in the “perfect” position. Quantitative data 
also allow measuring average arms and head movements, which are about 1.6 cm/s in the 
case of assisted steering system and quadruplicates in the case of manual steering (with an 
average of 6.8 cm/s).  
Such preliminary tests demonstrate how the sensor can be efficiently implemented for quan-
tifying the impact of driving operations on drivers body and to understand how new technolo-
gy (such as assisted steering, continuously variable transmission, etc.) and adequate cab 
design can help reducing body fatigue.  
4. Conclusions 
In the present study we propose implementation of the Kinect sensor for driver movements 
analysis. Such low-cost infrared depth camera allows markerless skeletal tracking, both in 
standing or seating position: as a results, average movements, relative positions, speeds and 
angles of different joints can be monitored at a relatively high sampling rate (up to 10 Hz) 
providing relevant feedbacks for optimization of tractor drivers working conditions. 
Acknowledgements  
Dr. Massimiliano Rinaldo (massimiliano_rinaldo@libero.it) and Diego Gattolin 
(gattoxy89@gmail.com) are acknowledged for their assistance in the development and cod-
ing of the software implemented for tracking and analysis of the Kinect sensor data. 
References 
Chéné, Y., Rousseau, D., Lucidarme, P., Bertheloot, J., Caffier, V., Morel, P., Belin, E., Cha-
peau-Blondeau, F. (2012). On the use of depth camera for 3D phenotyping of entire plants. 
Computers and Electronics in Agriculture, 82, 122-127.  
 
Egelund, N. (1982). Spectral analysis of heart rate variability as an indicator of driver fatigue. 
Ergonomics, 25 (7), 663-672.  
 
Healey, J.A., Picard, R.W. (2005). Detecting Stress During Real-World Driving Tasks Using 
Physiological Sensors. IEEE Transactions on intelligent transportation systems, 6 (2), 156-
166. 
 
Marinello, F., Pezzuolo, A., Gasparini, F., Sartori, L. (2013). Three-dimensional sensor for 
dynamic characterization of soil microrelief. Precision agriculture ’13, J.V. Stafford ed., 71-78. 
 
Tanke, N., Long, G.A., Agrawal, D., Valada, A., Kantor, G.A. (2012). Automation of Hydro-
ponic Installations using a Robot with Position Based Visual Feedback. Proceedings of the 
International Conference of Agricultural Engineering CIGR-Ageng 2012, C1429.  
 
Tran, C., Trivedi, M.M. (2010). Towards a Vision-based System Exploring 3D Driver Posture 
Dynamics for Driver Assistance: Issues and Possibilities. Proceedings of the IEEE Intelligent 
Vehicles Symposium, 179-184.  
 
Yang, C.C., Hsu, Y.L. (2010). A review of accelerometry-based wearable motion detectors 
for physical activity monitoring. Sensors, 10, 7772-7788. 
 
Zhang, J.Y., Qiu, W.W., Fu, H.J., Zhang, M.T., Ma, Q.G. (2013). Review of techniques for 
driver fatigue detection. Applied Mechanics and Materials, 433-435, 928-931.  
 
Zhang, Z. (2012). Microsoft kinect sensor and its effect. IEEE Multimedia, 19 (2), 6190806 4-
10.  


Ergonomics analyses through motion capture in a vehicle cabin by means of Kinect sensor

http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=86044427D32546165DCA2DA7DC20ADCD?doi=10.1.1.1054.6685&rep=rep1&type=pdf

Ergonomics analyses through motion capture in a vehicle cabin by means of Kinect sensor

Abstract

Similar works

Full text

Available Versions

CiteSeerX