With the rise of robotic arms in both industrial and research applications, a growing
need is observed for autonomous robotic arm applications. This thesis aims to provide
an example case of this need and also to showcase the possibility and limitations of
vision-based solutions, specifically in automating chess. The focus is on developing a
modular system that is able to autonomously recognize chessboard, detect and manipulate
chess pieces. The modular design allows for further exploration into autonomous mobile
manipulators. The key components include chessboard recognition using fiducial markers
to facilitate accurate chessboard recognition and utilizing image processing techniques
like segmentation, absolute difference matching, and perspective warping to analyze and
extract meaningful information. By mounting a camera above the chessboard, it enables
the detection algorithm to accurately capture and analyze the most important information
about the environment to determine the current state of the game. Using this information,
human move detection is enabled. Then, a custom protocol is utilized to communicate
between the detection algorithm and the chess engine, encapsulating information about
the game state changes within the system. The chess engine serves the purpose of game
analysis and provides legal moves for the robot manipulator to execute. Manipulation
happens through careful motion planning and execution, ensuring the safety of the robot
and its environment. Extensive evaluation proves that the system demonstrates high
accuracy and success rates for piece manipulation and move detection
