Leveraging RGB‑D Data and SAM Segmentation for Object Segmentation in Industrial Bin Picking : Master’s thesis

Abstract

Robotic bin picking requires a robotic arm to identify and extract objects from a container for placement on a production line. While this task can be broken down to two main aspects, the object selection and path planning and picking of the object. This thesis will focus on the latter. The task would include the selection of an object that can reliably be identified and picked in the least amount of time. The challenges faced here will include cluttered bins, occlusions and an ever-changing selection of objects that need to be identified by the segmentation model with the least amount of training time to identify novel objects. Traditional segmentation models struggle in these environments especially when they have not been trained on the specific ob jects being picked. This thesis investigates how the Segment Anything Model (SAM), a modern zero-shot segmentation model can be adapted in the simplest and effective way possible to in dustrial bin picking. The goal is to refine SAM’s mask selection process so that a suitable candi date object can be selected from a bin with the least amount of training or tweaking of the segmentation pipeline. A modular scoring pipeline was designed and integrated into SAM. The experiments to test out the pipeline were conducted on both a synthetic dataset and real RGB D images captured by a depth camera. Initial tests showed that a simple least depth-based mask performed well but suffered in the selection of masks that covered the whole object and in se lecting objects that were away from the border of the image. By including all the aspects of the segmentation pipeline the quality of the object picked increased dramatically by avoiding issues such as partial masks, selection of objects too close to the bin walls and selection of the bin wall as an object. The findings highlight both the strengths and weaknesses of SAM in industrial con text. More importantly, it shows that a carefully designed scoring system can negate those weak nesses and be used as an adaptable and effective segmentation solution for robotic bin picking offering a practical solution without retraining or complex hardware.