Why so much excitement about electricity markets? Ed Miliband's pledge for a 20 month price freeze has met been siezed upon gleefully by the right-wing press as an opportunity to tar him a 'socialist'. 

And yet only a few months ago David Cameron himself was pledging to force the electricity companies to offer the same fixed tarriff to all - apparently flattening electricity market competition.

This is clearly one of those issues where each poitical party is chasing the other in attempts to capture the popular support on the issue of rising energy bills. But the media coverage gets nowhere near the real issues - and it is not clear that either party's policy does either.

Behind Labour's "price freeze" sound-bite is a pledge to break up the big players and "reset" the electricity market. "The electricity market is not functioning competitively" we are told, and "the big six are price-gouging". There is certainly plenty of evidence that the market does not function well - as regulator Ofgem said in June 2013, "liquidity in the electricity wholesale market remains insufficient... poor liquidity acts as a barrier to entry and competition."

But I fear that the market changes currently being discussed will achieve little. Although many experts accept that more and more government intervention is likely, I wonder in contrast whether we need more market activity, not less - but in the context of a different market structure.  My research is suggesting that the way to achieve a dynamic electricity market which can deliver innovation and cost efficiencies might be almost the polar opposite of what is being suggested.

There are two parts to the problem: where we get energy from, and how we use it. The truth is that there is little any government can do to change the global trends in fossil fuel energy prices. However we can act to continue to bring the cost of clean energy down - through continued incentives for deployment and R+D in clean energy.  And critically, we can act to reduce the total need for primary energy as well as to reduce system peak demands, as both will have a major impact on the cost of the clean energy system.

It comes down to two main tractable factors: on the generation side, changes to reduce the cost of capital, and on the end-use side, changes to increase the productivity of energy as well making demand smarter and more responsive.  But current wholesale and retail electricity markets are failing to deliver these things.

On the wholesale, generation side, the cost of capital has been driven up by weak and flip-flopping government policy (and is of course already greatly exacerbated by the increased risk-aversity of capital following the financial crisis).  On the retail side, the current incentive for suppliers is to sell more kilowatt-hours, not less, rather than to innovate to overcome the transaction costs of measures to achieve higher energy efficiency. Radical changes to both sides are needed.

The market for provision of 'wholesale' electricity is rightly a market in capacity and security, not in energy - most zero-carbon energy has zero marginal cost, distorting the current marginal-cost market. A new market in the commissioning and availability of generation capacity needs somehow to be managed at arms length from government, with long-term objectives for decarbonisation and energy security set firmly in statute - which will reduce investor risk and the cost of capital.

Meanwhile the retail side of the market needs to be opened up to price variation, and to a different class of utilities altogether. Current electricity retailers add little value other than smoothing wholesale price variation into the single unit rate that customers see. But smart grid technology now allows the price to vary, incentivising demand response. The new utilities would follow a new business model based on the provision of energy services, aggregating efficiency measures while also using the 'internet of things' to maximise automatic demand response, and to integrate local generation with energy storage in the most cost-effective way - for example that from photovoltaics, whose price plunge promises to continue.

Such a market structure could drive a huge flowering of innovation in the smart grid and energy efficiency, including both products and services, and probably surprising combinations of the two. Such a market would tap the ocean of increased energy productivity which technology is making available, while on the generation side it would create the demand certainty and large-scale, long-term contracts required to drive down the cost of capital and the cost of technology.

For more formal and detailed explorations of what these ideas entail, watch this space.

At last week's conference of doctoral students in energy, hosted at Imperial College, "Artist in residence" Eleanor Beer did some brilliant sketches of the speakers.  Here is her sketch of my talk:
I presented some initial results of a simulation of a zero-carbon electricity system, which I am using to ask how the market might best be structured to incentivise energy storage... Some of the conclusions are surprising - more detail to follow.