9 research outputs found
Industrial Internet of Things, Big Data, and Artificial Intelligence in the Smart Factory: a survey and perspective
International audienceThanks to the rapid development and applications of advanced technologies, we are experiencing the fourth industrial revolution, or Industry 4.0, which is a revolution towards smart manufacturing. The wide use of cyber physical systems and Internet of Things leads to the era of Big Data in industrial manufacturing. Artificial Intelligence algorithms emerge as powerful analytics tools to process and analyze the Big Data. These advanced technologies result in the introduction of a new concept in the Industry 4.0: the smart Factory. In order to fully understand this new concept in the context of the Industry 4.0, this paper provides a survey on the key components of a smart factory and the link between them, including the Industrial Internet of Things, Big Data and Artificial Intelligence. Several studies and techniques that are used to enable smart manufacturing are reviewed. Finally, we discuss some perspectives for further researches
Simpler learning of robotic manipulation of clothing by utilizing DIY smart textile technology
Deformable objects such as ropes, wires, and clothing are omnipresent in society and industry but are little researched in robotics research. This is due to the infinite amount of possible state configurations caused by the deformations of the deformable object. Engineered approaches try to cope with this by implementing highly complex operations in order to estimate the state of the deformable object. This complexity can be circumvented by utilizing learning-based approaches, such as reinforcement learning, which can deal with the intrinsic high-dimensional state space of deformable objects. However, the reward function in reinforcement learning needs to measure the state configuration of the highly deformable object. Vision-based reward functions are difficult to implement, given the high dimensionality of the state and complex dynamic behavior. In this work, we propose the consideration of concepts beyond vision and incorporate other modalities which can be extracted from deformable objects. By integrating tactile sensor cells into a textile piece, proprioceptive capabilities are gained that are valuable as they provide a reward function to a reinforcement learning agent. We demonstrate on a low-cost dual robotic arm setup that a physical agent can learn on a single CPU core to fold a rectangular patch of textile in the real world based on a learned reward function from tactile information
Addressing Action Oscillations through Learning Policy Inertia
Deep reinforcement learning (DRL) algorithms have been demonstrated to be
effective in a wide range of challenging decision making and control tasks.
However, these methods typically suffer from severe action oscillations in
particular in discrete action setting, which means that agents select different
actions within consecutive steps even though states only slightly differ. This
issue is often neglected since the policy is usually evaluated by its
cumulative rewards only. Action oscillation strongly affects the user
experience and can even cause serious potential security menace especially in
real-world domains with the main concern of safety, such as autonomous driving.
To this end, we introduce Policy Inertia Controller (PIC) which serves as a
generic plug-in framework to off-the-shelf DRL algorithms, to enables adaptive
trade-off between the optimality and smoothness of the learned policy in a
formal way. We propose Nested Policy Iteration as a general training algorithm
for PIC-augmented policy which ensures monotonically non-decreasing updates
under some mild conditions. Further, we derive a practical DRL algorithm,
namely Nested Soft Actor-Critic. Experiments on a collection of autonomous
driving tasks and several Atari games suggest that our approach demonstrates
substantial oscillation reduction in comparison to a range of commonly adopted
baselines with almost no performance degradation.Comment: Accepted paper on AAAI 202