Most energy can't be used where it's produced.
A distribution system is needed to transport energy from the site of production to the site of consumption.
There are 2 major energy distribution systems used today: the logistics networks and pipelines that deliver fossil fuels to customers, and the electrical grid.
We'll start by understanding oil and gas pipelines, which made up the first large scale energy distribution networks.
Then, we'll go deep into the function of grids.
We'll see how grids drive electricity markets, and how electricity market incentives are one of the primary drivers of energy infrastructure innovation and adoption.
This will give us context on the obstacles preventing renewables from replacing all electricity generation.
Pipelines
Fossil fuels are primarily delivered to consumers through shipping, trucking, rail, and pipelines.
The transportation methods are self-explanatory, and are also inefficient for oil and gas delivery, so we won't cover them here.
Pipelines are the most economical way to transport oil and gas, and make up 70% of global oil delivery, with 23% by ship, 4% by truck, and 3% by rail. Let's look at the structure and function of these pipelines.
Oil Pipelines
Crude oil extracted from a variety of oil reserves is initially collected into upstream gathering pipelines and sent to separation and treatment facilities.
From here, it enters larger transmission pipelines that travel across countries and continents. Pipes are much larger in these pipelines to accommodate greater flows of product, usually around 10-48 inch diameter, with oil pumped at high pressure traveling 3-8 mph.
the basic structure of fossil fuel supply chains
Pumping stations along the route, spaced out between 20-100 miles apart depending on terrain, maintain high pressures in the pipes.
The pipe materials have to be able to withstand these high pressures as well as corrosion and temperature changes, so they're usually made of steel.
Crude oil is then collected in storage tanks and sent into refineries where they're separated into products through the distillation process we discussed in Part III.
Final products are pumped into smaller distribution pipelines and travel to distribution terminals across the country.
Finally, trucks carry out the last mile of transit, like delivering gas from terminals to gas stations.
Oil pipelines are monitored using supervisory control and data acquisition systems (SCADA) that collect data on pressure, flow, and temperature at points through the network and detect leaks. When leaks are detected, local valves are shut to isolate the issue.
Natural Gas Pipelines
Similar natural gas pipelines deliver pressurized natural gas to consumers. However, due to the fact that natural gas has a high volume compared with oil, it's difficult to transport it efficiently through pipelines over long distances.
Originally, no intercontinental natural gas networks existed, so natural gas use was isolated to countries with access nearby gas sources.
Then, in the late 1960s, the first liquefied natural gas (LNG) distribution network was built, enabling the overseas delivery of natural gas.
These networks use liquefaction facilities to cool natural gas to -162 °C, condensing it into a liquid form that's 600x denser.
These liquids are then transported to specialized tankers and cryogenic tanks that can distribute LNG globally.
basic LNG setup for global shipping
At its destination, LNG goes through regasification facilities to convert it back to standard natural gas. From here, it can then be distributed through ordinary natural gas pipelines.
The development of LNG technology enabled the global scale natural gas usage that we see today.
Overall
There are over 220,000 miles of crude oil and refined petroleum product transmission pipelines, and over 300,000 miles of natural gas transmission pipelines in the US.
Meanwhile, there are over 2.2 million miles of natural gas distribution pipelines, due to the common use of direct natural gas lines to consumer residences and other buildings.
map of oil and gas pipelines in the US[1]
By looking at the function of pipelines, we can extract a few requirements for the function of all energy distribution systems:
- Energy must be in a form that's easy to transport.
- Transmission networks with high capacity are used for efficient long-distance transportation.
- Distribution networks using lower capacity local systems are used to deliver energy directly to consumers.
- Consistent monitoring is required to ensure that the system is meeting demand
- There should be a way to turn off specific modules of the network to isolate damages and accidents.
We'll see that the electrical grid follows the same principles.
Grids
Along with inventing the light bulb, Thomas Edison also created his own electricity production and distribution system in 1882.
It served electricity from Pearl Street Station in Manhattan to 59 customers through an underground network of copper wires, lighting homes within a 1-mile radius.
This system was the first electrical grid.
Edison's Pearl Street Station and the area of Manhattan that it served
A grid is a network of power producers and consumers, connected by transmission and distribution lines, and operated by control centers.
Electricity is treated as a constant service, meaning that customers should always be able to access it on-demand without notifying the grid in advance.
This means that each grid has to constantly adjust power generation in real time to meet with changes in demand. This is known as grid balancing, and is a challenging problem.
We'll see that generators are built to operate within a narrow band of constraints, meaning that any small imbalances in generation and demand can cause a cascading chain of events that can shut down entire grids.
These blackouts can be catastrophic. On the extreme end, there are people on life-support in hospitals and residences for whom an electrical outage would mean death.
This is why grid balancing is taken so seriously.
Because of the high economic and social cost of failure, grids require consistent, reliable, and predictable resources.
This is an obstacle for renewables like solar and wind, which are highly unpredictable.
To understand the function of grids, we have to go down to the fundamentals of how grids distribute energy.
Let's start by looking at the energy transfer that occurs in each step of the grid.
Units
Energy in the grid is held and transported in the form of electricity, which is really the flow of electrons.
Let's look at a few units that will allow us to quantify the energy exchanges in wires.
Charge
The most basic unit of electricity is charge. The charge of a particle is measured in coulombs (C), where the charge of an electron is around
Voltage
Charges that are near each other experience a tension. Opposing charges want to move toward each other, and like charges want to move apart.
We can express this tension charges feel to move as electric potential energy.
Electric potential measures the strength of attraction or repulsion a charge would feel at a specific point. It's measured in volts (V), where 1 volt really is 1 joule per coulomb.
This means that there is a potential of 1 V at a point if a charge of 1 C would have 1 J of potential energy.
Current
Current measures the amount of charge flowing through a point over a certain amount of time.
Current is measured in amperes (A), where 1 A = 1 C / s. This means that a point with 1 A current would see 1 coulomb of charge flowing through it every second.
Resistance
Finally, resistance is the measure of a material's opposition to the flow of current, and is measured in ohms ().
Different materials have different innate oppositions to current based on their chemical compositions. This property is known as resistivity.
The total resistance of a device can be calculated as a function of the resistivity () of the material it's made of, it's length (), and the cross-sectional area of the device () as follows:
Most importantly, we see that wires with a larger diameter (and thus larger cross-sectional area ) have lower resistance, and long wires (high ) have a higher resistance.
This means that as the length of power lines increases, their resistance will increase proportionally.
Power
Our electrical infrastructure is built to maximize the efficiency of energy transfer through wires.
Let's see what the relationships between these units tell us about how to accomplish this.
Ohm's Law
Ohm's law tells us how the voltage across a wire changes as a function of its resistance.
Specifically, the voltage across a wire drops more with higher current and higher resistance.
In our case, the resistance of a wire is going to remain constant since electrical wires have constant length, cross-sectional area, and material, so the only thing that will vary here is the current.
Ohm's law tells us that as we increase the current in the wire, the voltage drop across it also increases.
Power Delivered
Now we can calculate how this affects the actual energy transferred in a wire.
Specifically, we will refer to the power delivered by a wire, which is the energy delivered by the wire to a load (the thing consuming electricity) every second.
The power delivered by the line is given by:
Note that we have used here, rather than . The power delivered by the line increases as the current increases, and as the voltage at the load increases.
Power Dissipated
We know that there's a voltage drop across any wire determined by its length, area, and resistance.
So a load connected to a long line will receive less power, since the voltage will have dropped across the wire before reaching the load.
This means that the total power delivered by a line decreases as electricity travels farther.
This is due to power being dissipated in the form of heat.
As the electrons in the wire crash into the atoms of the wire material, they cause tiny vibrations in the atoms, transferring part of their energy.
The total power dissipated due to heating in a wire is given by the following:
The dissipated power increases as the voltage drop across the wire increases. If we plug in the calculation of from Ohm's law, we get this new form:
The power dissipated due to heating scales with the square of the current, meaning that transporting electricity at high currents will lead to massive power losses due to heating.
This was initially the biggest problem limiting the distribution capabilities of power lines.
In fact, Thomas Edison originally thought that it would never be possible to economically transfer electricity more than 5 miles away from a power plant.
Transformers
The invention of the transformer solved these power dissipation problems.
The secret behind the transformer can be found in the two equations for power we discussed above.
The power dissipated in a wire scales with the square of the current. In order to minimize power dissipation, we have to keep current very low.
Looking at the equation for power delivery , we have to compensate for low current by using a very high voltage in order to deliver the same power. This is exactly what transformers are for.
The transformer allows us to transfer electricity through wires at very high voltages and low currents, minimizing the power loss due to heat.
Aside: How does a transformer work?
The transformer uses two adjacent coils that carry electrical current and interact with each other through magnetic fields.
As the currents in one side of the coil change, they induce a current in the nearby coil, causing current to flow despite the lack of connection.
The ratio of loops between the 2 coils determines the voltage increase and current decrease between the coils.
By using a coil with fewer loops on the side of an electricity generator, and a coil with more loops on the side of a power line, the transformer can "step up" the voltage of the wire and decrease the current.
It can then "step down" the voltage and increase the current with the reverse setup.
the basic structure of a power transformer
This allows us to take in the power output of a power plant, which may be coming in around 10 kV, and then use a transformer to step it up to 500 kV for transportation.
Then, a low current could be used in transportation to minimize heating losses and still deliver the same power due to high voltage.
This enabled the development of long distance power grids.
Note that a transformer requires a changing current through the coils to work.
If the current stays constant, the magnetic field produced by one coil won't induce a current in the next coil, and the transformer won't function. This will become important soon.
Transmission & Distribution
The transformer enabled the construction of large grids with the ability to deliver power across thousands of miles.
These grids are divided into transmission & distribution systems, just like oil & gas pipelines.
The transmission system transmits electricity over long distances, often >100 miles. Like transmission pipelines, these transmission lines have a high capacity.
They accomplish this by using transformers to step up transmission lines to very high voltages, usually around 60-500 kV. This has a "higher capacity" since it can transport power longer distances with lower losses.
simple transmission and distribution system
Distribution systems carry power from transmission systems to end consumers. They use transformers to step down the voltage from transmission lines, usually to ~60-70 kV for sub-transmission systems, and to <10 kV for local distribution.
There are also circuit breakers at different points that can be used to turn off any section of the grid.
This allows operators to isolate problems caused by downed wires and other anomalies. These match the function of valves in oil and gas pipelines.
Alternating Current & Generators
We saw that transformers require a constantly changing current.
This is why all modern electrical grids use alternating current (AC). Using AC allows us to use voltage transformers, which enable the long distance transportation of electricity.
In Part III, we saw that all the major electricity production methods aside from solar work by spinning a turbine. This setup is known as a generator.
Generators use a spinning magnet surrounded by wires (equivalent to spinning wires surrounded by magnets) to induce a current in the surrounding wires.
As the poles of the magnet rotate back and forth, the induced current oscillates in correspondence with this, creating an alternating current.
A simple generator using spinning wires inside magnets
A load connected to this system will receive power in periodic spikes corresponding with the peaks in the alternating current. This means that there isn't actually a consistent source of power but rather a rapid on-and-off pulsing of power through wires.
Intuitively, it doesn't seem like this is the type of power we rely on today, since it doesn't look like our appliances are quickly turning on-and-off.
In fact, light bulbs are actually flickering on and off very quickly, right in lock-step with the peaks of AC current. This flickering is just fast enough that it's imperceptible to us. Other electrical devices also draw power in this periodic fashion.
Aside: Three-phase power
Modern generators use three-phase power, which uses 3 different coils around a rotating magnet rather than just one to capture multiple different oscillating waves of power that are slightly shifted from each other.
This allows loads connected to these 3 different phases to rapidly switch their connections between 2 or 3 phases, providing a more consistent power output than a single phase.
We can see below that the aggregate power output of a three-phase power system is more consistent than a single oscillating phase.
3-phase power generator has a more steady power output[2]
This oscillating pulse of voltage in AC power systems can be described with a frequency.
Modern grids operate on either 50 Hz or 60 Hz, referring to the fact that the voltage in these grids oscillates back and forth 50-60x every second (implying that the generators turbines rotate 50-60x per second).
Individual loads have the ability to cause slight shifts in the grid's AC frequency due to their inductive and capacitive reactance (we won't cover the details of how this works, for the sake of simplicity).
As many loads connect and disconnect to the grid at the same time, they can cause large aggregate changes that shift the overall frequency of the network.
When the frequency through the wires in a generator shift, its internal magnetic field changes, which can break the generator.
Because of this, the AC frequency in the wires of a generator needs to remain within a very narrow band.
Any shifts in frequency due to reactance from unexpected demand can cause generators to shift their frequency, further affecting other generators and causing a cascading effect that can shut down the entire grid.
This is why it's so critical to ensure that generation and consumption on the grid are perfectly balanced.
Inverters
The one exception to electricity generation we saw was solar photovoltaics, which create direct current (DC) by capturing the energy from photons.
In order to integrate these systems with the grid, we need a way to convert DC into AC. This is exactly what an inverter does.
An inverter uses switches to rapidly turn DC current on and off and periodically flip the direction of current flow. These oscillations can be timed such that the net current output of the circuit resembles AC, but at a low voltage.
The inverter then uses a transformer to step up the voltage to a level that's usable.
Inverters are necessary for solar panels to be used on the grid.
In Part V, we'll see that battery storage systems also depend on inverters to connect to the grid.
an inverter with switches (S1, S2, S3, S4) to convert DC to AC[3]
Grid Operators
Let's understand the various groups that operate modern grids and are tasked with balancing generation and consumption.
Electricity transmission and distribution is a natural monopoly.
Due to the high upfront capital expenditure required to build new power lines, and the fact that one set of power lines can naturally serve an area completely, it doesn't make sense for multiple companies to invest in infrastructure in the same areas.
Early electricity companies that were the first to build energy infrastructure in an area permanently dominated those markets.
They then used profits from this dominance to expand their grids, creating excess generation and reserve capacity that made their grid more reliable than smaller ones.
Through this process, the entire country was unified under just a few large grid operators.
The results of this unification are still visible in our modern grids in the fact that the US today has 3 major grid networks which connect all of the country's electrical infrastructure. Each grid is disconnected from the other and has to service its own consumers' demands independently.
the 3 grid networks that serve the US[4]
These few electricity companies had uncontested power in their markets.
The government saw this and decided to step in. They considered electricity to be a fundamental need and wanted to ensure that it was safe, cheap, and reliable.
The first step to accomplish this was to prevent electricity companies from abusing their monopoly pricing power.
The Public Utility Holding Company Act (PUHCA) of 1935 officially established electricity as a utility. This meant that electricity was a public good, and the government enacted price controls to cap the profits they were legally allowed to earn on infrastructure investments and electricity generation.
However, utilities still held control over all the electricity infrastructure in their regions of operation, preventing healthy competition that could lead to fair pricing and independent energy generators.
To address this, the government passed the Energy Policy Act of 1992, which forced utilities to give up their transmission infrastructure to independent organizations known as Independent Service Operators (ISOs), leaving utilities to maintain their production and distribution assets.
the 7 ISOs that operate US grids[5]
Transmission networks are the most central assets in grids, determining the flow of electricity from generators to distributors.
By separating out these assets into their own entities, ISOs could now manage competitive markets that enable fair pricing for consumers and enable independent generators to participate in electricity production.
Independent System Operators
ISOs often carry out 3 distinct roles:
- Maintaining the transmission system infrastructure.
- Ensuring that generation and consumption are exactly balanced at all times.
- Maintaining competitive electricity markets.
ISOs are responsible for purchasing electricity from generators and then reselling this electricity to large consumers and load-serving entities (LSEs) that deliver electricity to smaller consumers.
They also manage 3 different kinds of markets:
- Energy markets that allow for the sale and purchase of electricity.
- Capacity markets where they can incentivize generators to build new capacity.
- Reserve markets where they maintain reserves of excess energy to resolve last-minute imbalances and anomalies.
ISOs and regulatory bodies set the market rules that incentivize the development of new energy production facilities, grid-storage, and reserve capacity.
Grid Balancing
Electricity flows through power lines from the generation facility to consumers almost instantly.
Ensuring that electricity is produced exactly as it is consumed is the central challenge of grid balancing.
We know that the cost of failure in grid balancing is high since excess demand that is unmet by supply can shift the frequency of the grid, causing blackouts.
Because of this, balancing authorities must always ensure that they have excess capacity available at all times of the day to meet the changing demands of consumers.
Demand
Thankfully, fluctuations in demand on the grid are not entirely unpredictable. Electricity demand curves follow common patterns that vary with time of day and seasonality.
weekly electricity demand at different points in the year
We can see a few common patterns in this demand graph that are worth noting.
First, there's a drop in demand every night ~11pm as people go to sleep. Demand hits its lowest point ~4-5am. It starts to pickup again in the early hours of the morning.
We also see a small drop in demand in the late afternoon as most people are finishing work and commuting home, followed by peak daily demand in the late afternoon and night hours when people are back at home with free time.
There's a noticeable decline in demand toward the weekends where people are spending less time in their houses or indoors consuming electricity.
Finally, we see a strong correlation between seasonality and electricity consumption, with extreme temperatures causing much higher consumption.
We see that air conditioning demands in the summer (red line) put significant pressure on the grid compared with the heating needs during the winter (blue line) or the neutral needs during the spring (black line).
These types of consistencies in demand allow grid operators to forecast electricity consumption needs and ensure that sufficient generation capacity is available throughout the day.
Supply
Not all energy production methods are equal in their ability to meet grid demands.
From the perspective of the grid, there are only a few things that differentiate the various energy production methods:
- Startup time - The time it takes to start generating electricity. Some generation methods like coal can take up to a day to start producing at full capacity, whereas others like hydroelectric can start almost instantly.
- Minimum up-time and output - The minimum operating time and power output necessary to stay running. For example, nuclear plants cannot run economically at low output.
- Shutdown time - The time to shut down electricity generation. Combined-cycle turbines are much faster to stop than coal burning plants.
- Minimum down-time - Certain generation facilities have to be off for a minimum amount of time, like nuclear plants which require long rest times before they can safely start again.
- Cost - The operational costs of producing electricity including the costs of fuel and maintenance.
demand is fulfilled by the 3 different categories of suppliers
In addition to these factors, one new factor has emerged:
- Emissions - Recently, some ISOs like California's ISO mandate that grids deploy clean and renewable energy before other sources, regardless of price or intermittency concerns.
Using these determinants, we can derive the three different categories of power plants that supply electricity to the grid:
- Baseload plants provide consistent power for the part of demand that remains constant.
- Mid-range plants adjust their output based on the predictable changes in demand that we see throughout the day.
- Peaking plants provide capacity that can be deployed almost instantly to meet last minute demand fluctuations.
how the different categories of electricity suppliers fulfill demand
Baseload plants run on cheap electricity sources and are slow to get started but easy to keep going.
Coal is the most common baseload plant type because of it's relatively cheap cost compared with other fossil fuels, and the fact that coal plants take 12-24h to fully start.
Combined-cycle natural gas turbines can also augment base load when necessary, though this is rare.
The only renewable that can currently rival coal in its ability to meet baseload needs is nuclear.
Fission reactors offer similarly cheap power that can run almost indefinitely, whereas other renewables like wind and solar are far too intermittent to supply base demand.
Mid-range plants strike the balance between cheap to operate and quick to start.
Combined-cycle natural gas turbines, oil and diesel generators, and hydroelectric are the most common sources here.
Finally, peaking plants can start and stop instantly.
They can afford to use more expensive energy sources since they are used infrequently at times of high demand.
Current peaking plants are usually combined-cycle natural gas turbines or grid storage from pumped hydroelectric.
More recently, battery storage systems are also being used for this. We'll talk more about pumped hydro and batteries in Part V.
Grid operators constantly keep track of all the available power generation based on reliability, cost, startup time, etc.
They rank these resources in order of which power plants they want to deploy energy from first.
This is known as the merit order, and determines how power gets deployed to meet the constantly changing demand on the grid.
Renewables
Renewables make grid balancing challenging because they're intermittent and often highly unpredictable.
For example, wind energy can start and stop frequently throughout the day based on unpredictable changes in wind speeds.
When this happens, other electricity sources need to quickly turn on and off to pick up slack from wind, creating excess demand for more costly peaking plants.
Solar is also intermittent, though slightly more reliable since we can predict the presence of clouds over solar farms at least a few days in advance.
However, solar brings another challenge for the grid in addition to unpredictability.
Solar energy is only available during the day when demand is at its lowest point.
Distributed solar setups allow consumers to stop using energy from the grid and use their own local production capacity, which greatly reduces grid demand.
Then, when solar stops producing energy, demand quickly rises again, causing a large spike in electricity demand that needs to be met by a deployment of mid-range and peaking capacity. The shape of this steep demand curve caused by solar has become infamously known as the "duck-curve" because of its resemblance to the profile of a duck.
the size of the duck curve has increased with solar adoption[6]
Though renewables offer intermittency challenges, they have still gained adoption due to incentives created by regulations, like the government policies that force certain balancing authorities to deploy renewables first.
We'll see in Part VII that innovations in grid storage are addressing this renewable intermittency problem.
Markets
Finally, let's look into the incentives created by electricity markets. This will give us direct insight into some of the most important economic forces that drive the energy industry.
Aside: Maximizing global welfare with markets
Before we go into the specifics of electricity markets, let's quickly go over a few fundamental concepts that determine the purpose and function of markets in general.
We'll start with a simple definition of what a market really is:
A market is a place where buyers and sellers collect to see if deals can be made.
When markets are allowed to operate efficiently and price discovery can happen quickly, they converge to the prices that maximize global welfare and abundance for all parties involved.
An ideal efficient market has a few properties:
- Liquidity - It has enough buyers and sellers to facilitate transactions at all times.
- Convenience - It allows buyers and sellers to trade quickly and easily.
- Price Dissemination - It has good methods to spread information about market prices.
- Transparency - Market operations are transparent.
Since electricity markets serve most of the world's population, ensuring that they operate efficiently and enable fair price discovery has significant impacts on global welfare.
With this in mind, let's take a look at how modern electricity markets meet these ideals to arrive at fair prices that maximize benefits for generators and consumers.
Market Structure
First, let's go over the basic structure of these markets.
There are 4 major parties involved in electricity markets:
- Generators that supply the primary sources of electricity.
- Suppliers that buy electricity from generators in bulk and resell it to consumers.
- Flexibility providers that maintain available reserve capacity in case of unexpected issues on the grid.
- Consumers that use electricity to do stuff they care about.
Electricity markets are broadly made up of two different segments of transactions.
Wholesale markets are where suppliers and retailers buy electricity supply from generators in bulk. These markets allow for efficient electricity price discovery, but constant price fluctuations are undesirable for regular consumers.
Retail markets are where suppliers and retailers resell electricity to small consumers who want stable prices. Retailers then take on price risk and absorb the volatility of wholesale markets to give consumers price stability.
As we've discussed, ISOs typically maintain the wholesale electricity markets, from which large consumers, distribution companies, and retailers can purchase.
basic structure of wholesale and retail electricity markets[7]
Market Types
Spot MarketsISOs run the electricity markets of last resort.
These are known as spot markets and allow for the purchase and sale of electricity on the day-ahead market (which is where 95% of spot market transactions occur) and the hour-ahead and 5-minute-ahead markets (where the other 5% occur).
Though there are a variety of electricity markets, we'll see that all the other markets derive their prices from these spot markets.
Forward Markets
Electricity prices can often be highly volatile in spot markets due to last minute changes in supply and demand.
Large producers and consumers want to hedge against this price risk by buying and selling electricity far ahead of time.
For example, a large data center doesn't want to wait to buy a huge amount of electricity on the spot market since it exposes them to the risk that prices will spike around the time of their purchase.
Similarly, a big generator doesn't want to wait until the last minute to sell its electricity in case the price drops before it sells.
Both of these parties want a reliable way to settle electricity transactions well in advance of when spot markets are active.
This is what forward markets are for.
Forward markets for electricity allow producers and consumers to trade forward contracts well in advance that specify the amount of electricity to deliver, the price it will sell for, and the exact date it will be delivered.
These contracts derive their price from expectations about the spot market.
Generators will sell at a lower price if they think the spot market is going to be even lower at the time of generation.
Consumers will buy at a higher price if they think the spot market is going to be even higher at the time of purchase.
There are multiple types of forward markets for electricity.
Bilateral trading markets allow bids and asks to balance out to equilibrium on their own by letting buyers and sellers make their own pricing decisions. Privately negotiated contracts between a large consumer and an electricity generator are an example of bilateral trading.
Electronic trading marketplaces collect bids and asks in one place and allow buyers and sellers to make independent purchase decisions.
There are also centralized trading markets for electricity where bids and asks are aggregated by the market operator into a single price. In these markets, buyers and sellers can only choose to buy/sell at the current price or wait.
As we'll soon see, generators and producers often prefer to transact electricity on these forward markets, but they can't always accurately predict their real time output and consumption.
When differences emerge between contractual promises and reality, these parties are forced to buy/sell electricity from the spot markets to resolve them.
Futures Markets
The existence of publicly accessible forward markets naturally attracts speculators to enter.
To these people, forward markets are actually futures markets. They don't actually care to produce or consume electricity, but instead just hope for prices to appreciate, so they can sell their purchases in the future.
This is why it's known as trading futures: these buyers don't care for the underlying asset but instead are buying based on their expectations about future price increases.
These spectators increase market efficiency by accelerating price discovery and increasing liquidity, which contributes to increasing global welfare.
Resolution
Generators have unpredictable output, and consumers have unpredictable demand. As a result, contracts negotiated well in advance often don't resolve perfectly.
A generator may break down on the day of delivery and not fulfill the entire power output it promised. The consumer still needs to get access to the electricity they needed, so the grid operators themselves have to resolve this last minute.
Similarly, a consumer may actually use more electricity than negotiated in their contract last minute, and the grid has to resolve this for them as well.
These inconsistencies are handled by the market operator at gate closure, when the market prevents all further electricity transactions for a given time interval.
the function of a market operator after gate closure[8]
At this point, the market operator looks at all previous contractual promises for supply and demand. Then, during the time interval, it monitors the actual production and consumption of the parties involved.
If any differences emerge, the market operator buys the difference off reserve capacity providers and delivers it to consumers. Then, it forces the delinquent parties to buy this electricity off the spot-market.
This is the final step that explains why the spot-market determines all other prices, and why it makes sense for market operators to run this market of last resort.
Incentives
Finally, we can take a look at the incentives created by these markets for different parties and how they impact the energy industry broadly.
Generators
Generators are forced to pay a significant price on the spot-market if they fail to meet the generation that they promised to provide in advance.
For this reason, a primary focus of generators is to ensure reliability with regular maintenance to keep high uptimes.
Electricity markets also necessitate their usage of cheap fuels that can improve their operating margins compared with other alternatives.
Finally, generator companies may decide it's profitable to build new production capacity based on financial incentives from capacity markets, regulatory tailwinds, and access to cheap electricity production methods.
This is the force that drives the development of new renewable production infrastructure.
Retailers
Retailers act as intermediaries, purchasing electricity from wholesale markets with volatile prices and selling it to consumers at a fixed price.
Most residential and small business consumers prefer this model for its predictability and simplicity, even if it means paying a slight premium over wholesale rates.
To manage the risks associated with wholesale price volatility, retailers aim to aggregate large groups of consumers.
By pooling many customers, they spread their risk across a broader base, making it easier to absorb price fluctuations in the wholesale market.
They can also use hedging strategies to further protect themselves from volatility.
Retailers help to stabilize electricity prices for end consumers and increase market liquidity.
Consumers
As consumers, all we care about is that we get cheap, reliable, and available energy, which the system almost always fulfills like magic.
Then we get to run all our computers, phones, TVs, and appliances without this complex system that's working smoothly every hour of the day to give us the electricity we depend on.
Overall
The most important takeaway here is that the grid has to perfectly balance generation and consumption at every instant.
This goal drives grid operations, creating the need for base-load, mid-range, and peaking power plants that fulfill daily energy demand fluctuations.
Within this context, we see that renewables pose many challenges to the grid due to their unpredictability.
We also see the role fission may play in our future as the only renewable production source capable of replacing the base-load provided by coal.
Finally, electricity market incentives built around the goal of grid balancing drive long-term investment in new generation and storage capacity, determining how energy technology progresses.