Image-based onsite object recognition for automatic crane lifting tasks

The construction industry is suffering from aging workers and frequent accidents, as well as low productivity. Automation and robotics is regarded as a promising approach for enhancing the development of the industry, and the automatic operation of cranes, as an important aspect of construction, is attracting increasing attention. However, due to the complexity and dynamics of construction sites, it is difficult for cranes to automatically recognize and locate lifting objects (e.g., precast facades and partitions) on site. To solve this problem, an image-based automated onsite object recognition approach for the automatic operation of cranes is developed in this study. This is a fusion of Faster-R-CNN (Region-based Convolutional Neural Network), Canny, Hough Transformation, Endpoint clustering analysis and Vertex-based Determining Model, to uniquely locate a lifting object with exact pose and extract its features (e.g., centroid coordinates, size, and color). Based on the extracted features, the lifting object can be retrieved in the database with the IFC (Industry Foundation Classes) format of BIM (Building Information Modeling) to obtain more features for the automatic operation of a crane. It is shown from a field experiment that the developed approach is workable and has the potential to support the automatic operation of cranes. This contributes a basic approach to the automatic operation of cranes and promotes the rapid development of construction automation and robotics.