Modeling yard crane operators as reinforcement learning agents

Due to the importance of drayage operations, operators at marine container terminals are increasingly looking to reduce the time a truck spends at the terminal to complete a transaction. This study introduces an agent-based approach to model yard cranes for the analysis of truck turn time. The objective of the model is to solve the yard crane scheduling problem (i.e. determining the sequence of drayage trucks to serve to minimize their waiting time). It is accomplished by modeling the yard crane operators as agents that employ reinforcement learning; specifically, q-learning. The proposed agent-based, q-learning model is developed using Netlogo. Experimental results show that the q-learning model is very effective in assisting the yard crane operator to select the next best move. Thus, the proposed q-learning model could potentially be integrated into existing yard management systems to automate the truck selection process and thereby improve yard operations.