HomeWebAndroid & PCPublicationsCVContact

Energy- and Cost-Efficient Pumping Station Control

Description The goal of my master thesis was to investigate how artificial intelligence can make operating pumping stations in polder areas more energy- and cost-efficient. This problem is made difficult by the many different pumping stations to control, great uncertainty in energy prices and the long-term effects of actions. During this project, I developed a water system simulation to predict the flow of water and energy price developments, as well as decision making algorithms that are able to utilise this simulation to determine the most promising actions. As the experimental results show a significant improvement over current techniques and the work was well received by both the industry and the University of Amsterdam, we decided to publish a compact paper about the work at the Computational Sustainability track of the 2016 AAAI conference. The paper is accompanied by the thesis report which describes the used methods in further detail and investigates a more scalable adaptation.
Abstract With renewable energy becoming more common, energy prices fluctuate more depending on environmental factors such as the weather. Consuming energy without taking volatile prices into consideration can not only become expensive, but may also increase the peak load, which requires energy providers to generate additional energy using less environment-friendly methods. In the Netherlands, pumping stations that maintain the water levels of polder canals are large energy consumers, but the controller software currently used in the industry does not take real-time energy availability into account. We investigate if existing AI planning techniques have the potential to improve upon the current solutions. In particular, we propose a light weight but realistic simulator and investigate if an online planning method (UCT) can utilise this simulator to improve the cost-efficiency of pumping station control policies. An empirical comparison with the current control algorithms indicates that substantial cost, and thus peak load, reduction can be attained.
Citation Timon V. Kanters, Frans A. Oliehoek, Michael Kaisers, Stan R. van den Bosch, Joep Grispen, and Jeroen Hermans
Energy- and Cost-Efficient Pumping Station Control
In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, Arizona, USA, 2016
Link PDF
Master thesis PDF
Year 2015

Quality Assessment of MORL Algorithms: A Utility-Based Approach

One of the projects that I did with two fellow students at the University of Amsterdam focused on evaluating decision making algorithms with multiple objectives. Through a division of labour based on our backgrounds, one student developed mathematical proof for some important properties of a quality metric while the other and I designed and programmed a benchmark for evaluating the algorithms of other researchers. Using the RL-Glue library, our open-source benchmark can be hooked up algorithms written in most common programming languages. In order to promote the use of both our benchmark and metric, we published our work as a paper at the 2015 Benelearn conference. Description
Sequential decision-making problems with multiple objectives occur often in practice. In such settings, the utility of a policy depends on how the user values different trade-offs between the objectives. Such valuations can be expressed by a so-called scalarisation function. However, the exact scalarisation function can be unknown when the agents should learn or plan. Therefore, instead of a single solution, the agents aim to produce a solution set that contains an optimal solution for all possible scalarisations. Because it is often not possible to produce an exact solution set, many algorithms have been proposed that produce approximate solution sets instead. We argue that when comparing these algorithms we should do so on the basis of user utility, and on a wide range of problems. In practice however, comparison of the quality of these algorithms have typically been done with only a few limited benchmarks and metrics that do not directly express the utility for the user. In this paper, we propose two metrics that express either the expected utility, or the maximal utility loss with respect to the optimal solution set. Furthermore, we propose a generalised benchmark in order to compare algorithms more reliably. Abstract
Luisa M. Zintgraf, Timon V. Kanters, Diederik M. Roijers, Frans A. Oliehoek, and Philipp Beau
Quality Assessment of MORL Algorithms: A Utility-Based Approach
In Benelearn, 2015
PDF Link
2014 Year