Recently, the operators of power exchanges in North-Western European countries (in cooperation with the transmission grid operators) have begun to couple their day-ahead trade using one majestic algorithm. Also, roughly the same set of players just signed an agreement to integrate their intraday trade with each other in a similar manner. I have some comments about the level of dynamics going into this process and how little we will be able to understand single effects.

First of all, this development is in line with the rough guideline which the European Commission set out in their recent multi-year plans. The goal is to reduce price differences between regions. Specifically, local markets will keep running, so nothing much seems to change for bidders, but the markets run "implicit auctions" between each other to determine how to make best use of available cross-border transmission capacity. In general, the efficiency of a market increases if there is more buyers and more sellers and the number of possibilities to make agreements increases (i.e. regularly-held auctions find agreements and new prices for them much more often than long-term service and import/export contracts). This is especially important if the good is perishable (storing electricity is not the best economical option) and the local conditions are different (i.e. different countries have different generation portfolios and different weather conditions). Several cross-country connections have already been installed in recent years, so the physical means to interchange exist.

Besides such general economic considerations, I wouldn't be too sure that all citizens in Europe will be better off, short- or medium-term. But I won't go into this discussion here. I have a deal with the complexity involved. European Commisioner Guenther Oettinger says that this will happen:

"Fragmented European energy markets will soon be history."

But the local markets will exist for the foreseeable future. And they are all quite different in their way of working. The bid formats are different, e.g. constant price/quantity blocks, piecewise linear functions, or non-linear functions. Not to speak of the many different ways to specify reserve capacity. The method of clearing bids also differs of course, even if the bid formats are comparable. The timing of bidding and clearing is very often different.

I have talked to someone who was told first-hand about the making of the algorithm. It finds price-coupling solutions for the integration of these electricity markets. The problem it has to solve and the way it goes about doing it boggle the mind. I won't go into much details here. Suffice it to say, the number of constraints that have to be satisfied is high, such that normal solvers of linear programs will not suffice (prices for the day-ahead case are supposed to be computable within 10 minutes). Very modern computer science techniques are used to approximate a solution, in order to be fairly sure that the solution is close enough to the optimum.

I believe the people involved did a decent job figuring this out. But I cannot take their word that we have a good idea of what happens under the hood here.

We're not completely sure how this coupling algorithm performs. We can have a rough idea, but if there is a weird effect, we probably will find, more often than not, that we have no means of finding out what caused it. Because, let this be said, the local markets which are coupled together here, are not very well understood themselves. I concur that they are fairly well-understood. Debacles like the electricity crisis in California 2001still might happen, but most markets run in a stable manner. Probably this mega-market will also run mostly stable. However, for markets on the country level, debates are still ongoing which market clearing rules and bid formats are better and why that would be. No real agreement between economists so far, as I learned in my literature reviews. Mostly markets are set up to see if they will do well, and are analysed later.

Thus, we are having a hard time in these local markets to explain weird effects. Plus, there are very difficult novel problems only on the local level, like the interplay between day-ahead and intraday activity.

With this stochastic algorithm added on top of this bundle of markets, where does this leave us in terms of ability to go down the layers of abstractions, should we need to do that, and find out what is causing effects? Will we need to pay expensive consultants to tell us what they believe happened, without ever being sure?

As we do in other corners of our civilisation, we are creating massively complex systems. Here, the inability to understand what is going on is clearly built into the approach, as we build the next layer.