Innovative Urban Design Simulation: Utilizing Agent-Based Modelling through Reinforcement Learning

Data-driven design for cities is improving the quality of everyday life of citizens and optimizes the usage of resources. A new aspect is artificial intelligence, which Smart Cities could greatly benefit from. A central problem for urban designers is the unavailability of data to make relevant decisions. Agent-based simulations enable a view of the dynamic properties of the urban system, generating data in its course. However, the simulation must remain sufficiently simple to remain in the realm of computability. The research question of this paper is: How can we make agents behave more realistically to analyze citizens' mobility behavior? To solve this problem, we first created a simulated virtual environment, where agents can move freely in a small part of a city, the harbor area in Hamburg, Germany. We assumed that happiness is a crucial motivating factor for the movement of citizens. A survey of 130 citizens provided the weights that govern the simulated environment and the happiness score assignation of places. As an AI method, we then used Reinforcement Learning as a general model and Q-learning as an algorithm to generate a baseline. Through randomly traversing the model environment a baseline was created. We are in the process of enhancing Reinforcement Learning with a Deep Q-Network to make the actors learn. Early experiments show a significant improvement over a tabular Q-learning approach. This paper contributes to the literature of urban planning, and data-driven architectural design. The main contribution is replacing the inefficient search for a global maximum of the happiness function, with an efficient local solution global maximum. This has implications for further research in the generation of synthetic data through simulations.