Smart Perception for Situation Awareness in Robotic Manipulation Tasks

Abstract

Robotic manipulation in semi-structured environments require perception, planning and execution capabilities to be robust to deviations and adaptive to changes, and knowledge representation and reasoning may play a role in this direction in order to make robots aware of the situations, of the planning domains and of their own execution structures. In this line, this paper proposes a smart perception module intended to enhance the ability of the robot to perceive and interpret its environment by combining visual sensor data, implementing object detection and pose estimation (using fiducial markers and deep learning-based methods), and introducing reasoning capabilities (using ontologies). The result is a robust and smart perception system capable of handling both simulated and real-world complex scenarios and providing the required functionalities to allow the robot to understand its surroundings, with a primary focus on robotic manipulation tasks. The discussion on the tools used and the key implementation details are included, as well as the results in some simulated and real scenarios that validate the proposal as a module that provides situation awareness to allow a manipulation framework to adapt the robot actions to uncertain and changing scenarios.

Keywords: Perception, situation awareness, robotic manipulation, reasoning, ontologies.