Automatic Workflow Monitoring in Industrial Environments

Robust automatic workflow monitoring using visual sensors in industrial environments is still an unsolved problem. This is mainly due to the difficulties of recording data in work settings and the environmental conditions (large occlusions, similar background/foreground) which do not allow object detection/tracking algorithms to perform robustly. Hence approaches analysing trajectories are limited in such environments. However, workflow monitoring is especially needed due to quality and safety requirements. In this paper we propose a robust approach for workflow classification in industrial environments. The proposed approach consists of a robust scene descriptor and an efficient time series analysis method. Experimental results on a challenging car manufacturing dataset showed that the proposed scene descriptor is able to detect both human and machinery related motion robustly and the used time series analysis method can classify tasks in a given workflow automatically

Ein modulares Konzept von Klassifikatoren für Aktivitätserkennung auf Mobiltelefonen

In this thesis a modular activity recognition using accelerometer sensors on mobile phones is presented, which includes solutions to five challenges: 1.Flexibility: The conditions of the mobile phone usage and therefore for the activity recognition can always change. An activity recognition needs to flexibly adapt to this changes. 2.Extensibility: Different users have different demands of activities to be recognized. Only a small set of activities are performed by nearly every user. Therefore, the recognition needs to be extensible to the individual needs. 3.Robustness: The device is typically not firmly attached to any position, which results in noisy sensor data. A robust recognition is needed, which is able to detect the activities with high accuracy. 4.Resources: The resources on mobile phones are limited (processor and battery capacity), therrfore the activity recognition needs not to have a high impact on these. 5.Conditionality: The user and her phone can be situated in various different conditions. Each of these conditionalities implies different sensor patterns, which need representation in the activity recognition algorithm. The modularity of the proposed approach enables the individual adaption of parts of the activity recognition to offer flexibility. A modular recognition is extensible by new modules which detect new activities. The recurrence of the classification process stabilizes the recognition and enables the derivation of a reliability measure. Only one module and not the whole activity recognition is active at each point in time, which decreases the calculation effort and therefore the energy consumption. Each module can be suited for dealing with one conditionality, through which neither the complexity of the recognition is increased nor the accuracy is significantly lowered. All these solutions to the challenges of activity recognition on mobile phones are rounded by a service, which supports the novel system on the common user's phone.In dieser Dissertation wird eine modulare Aktivitätserkennung mit Beschleunigungssensoren auf Mobiltelefonen vorgestellt, die Lösungen für folgende fünf Herausforderungen bereitstellt: 1.Flexibilität: Die Bedingungen der Nutzung eines Mobiltelefons und damit auch für die Aktivitätserkennung können sich jederzeit ändern. Eine Aktivitätserkennung muss flexibel auf diese Veränderungen reagieren können. 2.Erweiterbarkeit: Unterschiedliche Anwender haben unterschiedliche Anforderungen welche Aktivitäten erkannt werden sollen. Daher muss die Erkennung erweiterbar sein, um die individuellen Bedürfnisse befriedigen zu können. 3.Robustheit: Das Gerät ist typischerweise nicht fest an einer Position angebracht, woraus verrauschten Sensordaten resultieren. Desswegen ist eine robuste Erkennung erforderlich, welche in der Lage ist die Aktivitäten trotzdem mit hoher Genauigkeit zu detektieren. 4.Resources: Die Ressourcen (Prozessor und Akku-Kapazität) auf Handys sind beschränkt, weshalb die Aktivitätserkennung diese nicht noch zusätzlich übermäßig einschränken sollte. 5.Konditionalität: Der Benutzer und sein Telefon können in verschiedensten Gegebenheiten situiert sein. Jede dieser Konditionen impliziert andere Muster der Sensoren, welche jeweils durch die Aktivitätserkennung repräsentiert sein müssen. Durch die Modularität, welche in dieser Dissertation zur Bewältigung der Herausforderungen vorgeschlagen wird, wird ermöglicht, dass Flexibilität bereitgestellt werden kann. Eine modulare Erkennung ist erweiterbar durch neue Module, welche neue Aktivitäten erkennen. Die Rekurrenz des Klassifikationsprozesses stabilisiert die Erkennung. Nur ein Modul ist zu einem Zeitpunkt aktiv, was Ressourcen schont. Jedes Modul kann passend sein, um mit einer Konditionalität umzugehen, wobei die Komplexität weder erheblich erhöht noch die Genauigkeit stark erniedrigt wird. Alle diese Lösungen für die Herausforderungen der Aktivitätserkennung werden durch einen speziellen Service abgerundet

Automatic workflow monitoring in industrial environments

Robust automatic workflow monitoring using visual sensors in industrial environments is still an unsolved problem. This is mainly due to the difficulties of recording data in work settings and the environmental conditions (large occlusions, similar background/foreground) which do not allow object detection/tracking algorithms to perform robustly. Hence approaches analysing trajectories are limited in such environments. However, workflow monitoring is especially needed due to quality and safety requirements. In this paper we propose a robust approach for workflow classification in industrial environments. The proposed approach consists of a robust scene descriptor and an efficient time series analysis method. Experimental results on a challenging car manufacturing dataset showed that the proposed scene descriptor is able to detect both human and machinery related motion robustly and the used time series analysis method can classify tasks in a given workflow automatically. © 2011 Springer-Verlag Berlin Heidelberg.Veres G., Grabner H., Middleton L., Van Gool L., ''Automatic workflow monitoring in industrial environments'', Lecture notes in computer science, vol. 6492, pp. 200-213, 2011 (10th Asian conference on computer vision - ACCV 2010, November 8-12, 2010, Queenstown, New Zealand).status: publishe