[E14.1] Power Density: a catalyst for change
What happens when a Bitcoin miner acquires four power plants?
Dear Readers,
In this Series for Product | Strategy | Innovation I will discuss how increasing demand for highly specialized computing with many times the power density of conventional cloud data centers provides a catalyst for change. End uses for this “new” computing infrastructure will provide multiple downstream benefits to society. Upstream it will accelerate the build out of sustainable energy infrastructure in key “hubs” around the world to provide cheap, abundant, reliable energy. Economic growth results from all these upstream and downstream benefits.
When I first started reviewing this topic over 6 months ago, I was skeptical. It seemed like a niche strategy by a few ventures to raise capital or diversify their cyclical businesses. Conventional wisdom
regarding electricity focuses on energy efficiency and conservation on the demand-side to reduce power consumption. This also reduces the cost to generate the electricity needed and use of fossil fuels on the supply-side. Less demand on the supply-side limits carbon emissions from these fossil fuel energy sources.
What if we reimagine the problem we aim to solve?
Problem reimagined
How do we scale access to abundant, cheap, sustainable energy?
Solution yields multiple Benefits
Unlock new uses for energy when its price approaches zero outside Peak energy demand
Solutions are not just advances in technology. The transition to sustainable energy is capital intensive to build out the required infrastructure. This transition requires a more open, decentralized ecosystem of stakeholders who integrate their services and build on the value created by others to disrupt legacy systems. Ideally, a free market develops to maximize the return on the sustainable energy generated.
This will be a 3-Part Series, with the following Parts:
Power Density: a catalyst for change
Proof of Work: a proxy for energy
High Performance Computing: a growing opportunity
Companies providing High-Density Computing consume many times the energy per square foot of other businesses, so these companies are building new infrastructure in hubs where energy is abundant and cheap. This also happens to be where sustainable energy generation is stranded with production curtailed outside of Peak energy demand. That is why you often see wind turbines idle.
The immediate strategy is to co-locate High-Density Computing into markets with surplus sustainable energy like Texas (solar and wind), New York, Quebec, Ontario, Paraguay (hydroelectricity), United States, Indonesia, Philippines, New Zealand (geothermal), nuclear energy and biofuels. But as the demand side of load balancing scales with this computing infrastructure, this will also drive the need to build out more sustainable energy generation from these same sources like hydroelectricity, solar, wind and stationary battery storage.
A recent development last month that validates the need for cheaper, abundant energy to support co-located, High-Density Computing was a winning bid by Bitcoin miner, Hut 8, to take over operations for 4 power generation plants in North Bay, Ontario, Canada. This represents downstream energy demand swimming upstream (backwards integration) to generate and use their own electricity. A desirable outcome at scale would help accelerate the transition to sustainable energy.
Background
Sustainable energy sources like wind and solar vary in availability across a 24-hour period and day-to-day. These projects are often located in remote areas to reduce land costs where the local demand for energy is also usually much less than in more urban areas.
This leads to curtailment (turning off sustainable energy generation as shown above on the left side of Fig. E14.1-2) when the energy supply exceeds energy demand. Curtailment detracts from the profitability of these projects and leads to the need for incentives to make these projects financially viable. This limits the scalability for these sustainable energy projects.
Energy transmission to locations with Peak energy demand (as shown above on the right side of Fig. E14.1-2) and stationary battery storage are options to counter sustainable energy generation curtailment, but require additional investments to deploy and expand this infrastructure. Thus, there are multiple barriers limiting the transition to grid-scale sustainable energy.
A watt is a unit of electric power and represents the rate energy is used or produced. One watt is equivalent to electricity flowing at a rate of 1 joule (unit of energy) per second.
1 kilowatt = 1 kW = 1,000 watts
1 megawatt = 1 MW = 1,000 kW
1 gigawatt = 1 GW = 1,000 MW
1 terawatt = 1 TW = 1,000 GW
A kilowatt-hour is a unit of electric energy and represents the quantity of electricity used or produced over a certain period of time like hours ( example: 1 kW over 20 hours requires 20 kWh)
1 kilowatt-hour = 1 kWh = 1,000 Wh
1 megawatt-hour = 1 MWh = 1,000 kWh
1 gigawatt-hour = 1 GWh = 1,000 MWh
1 terawatt-hour = 1 TWh = 1,000 GWh
Unconventional wisdom
regarding the transition to sustainable energy co-locates high power-density loads (concentrated energy use) where sustainable energy sources are at the lowest cost and in abundant supply even if the infrastructure to generate this energy is at an early stage of development.
However, it does help if sustainable energy infrastructure already exists with adequate curtailment to help drive favorable terms for a Power Purchase Agreement (PPA). This lowers the key input cost for the business case to build out the high power-density computing center to take all of this surplus energy during Off-Peak electrical grid power demand.
High power-density loads are even better if they can be curtailed quickly when the demand for electricity on the electric utility grid is high to help balance energy supply and demand. A sustainable energy marketplace drives growth and acts as a catalyst to expand the supply of cost-competitive energy since co-located demand is sustainable.
So what are some examples of high power-density loads? And do they provide economic value outside the upside from accelerating the transition to sustainable energy? The best example of a high power-density load is an enterprise-scale Bitcoin mining facility designed to optimize the Proof of Work (POW) computing required for the Bitcoin protocol. The leading data centers specializing in this technology currently operate at about 70 kW of power per rack to house, power and cool the semiconductors.
A data center rack is a standard unit configuration for semiconductor equipment.
The standard horizontal dimension for a rack unit is 19 inches (480 mm) wide between vertical frames. The vertical metal frames have holes distributed vertically that allow equipment to be configured in vertical units of 1.75 inches (44.45 mm) for 1U. There are typically 42Us per rack. Some equipment may span 1, 2, 3 or 4U based on its vertical dimension. Multiple racks can be configured side-by-side to build out a data center in rows of racks separated by enough space to access and service the equipment as needed.
Multiple racks can push the total power load for the data center to 50-750 MW based on current designs and usable square footage for the facility. However, those loads could increase to 3-5 GW per facility within 5 years based on the pace of innovation and growth in the industry.
The power density for cloud computing just a decade ago ran in the range of 4-5 kW per rack. By 2020, 8-10 kW per rack was considered state of the art for cloud computing that consists of CPUs, data storage and network communication equipment.
State-of-the-art High-Performance Computing (HPC) using the latest GPUs for machine learning to train large language models (LLMs) for AI-based applications currently requires about 40 kW per rack, but this power density will also increase over time.
GPUs will likely keep lagging the power density per rack specifications for Application Specific Integrated Circuit (ASIC) semiconductors used for Bitcoin mining. These ASICs could reach 200 kW per rack within the next 5 years.
Estimated Data Center rack power density, chip technology & use cases in 3-5 years:
8-10 kW - CPU chips - Cloud Computing: data processing/storage/communication
40-100 kW - GPU chips - HPC for LLM training to advance AI applications
70-200 kW - ASIC chips - PoW computing for Bitcoin mining
Considering this optionality for data centers specializing in High-Density Computing, cloud computing is out of scope in these facilities except what is required to operate and connect High-Density Computing services to third parties. Curtailing the computing power becomes a high priority during periods of Peak electrical grid power demand to “sell” the energy that would be used for computing to the utility grid when the economics favor that option.
Curtailing High-Density Computing can be done quickly due to how process tasks are batched and can vary from reducing a portion of the computing power deployed all the way to total curtailment within the facility. When the economics are favorable, the backlog of batches can be delayed until the economics favor switching back to High-Density Computing. However, the optionality and deployments for PoW versus HPC are a function of capital allocation for the equipment between these options and will likely vary across multi-year cycles.
1. Proof of Work evolves rapidly to secure the Bitcoin blockchain
A desktop computer was adequate to run a node for the Bitcoin protocol when bitcoin was first introduced over a decade ago as a new digital asset. The “reward” for verifying Bitcoin transactions and writing a block to the blockchain was initially 50 bitcoin tokens. Writing a new block happens every 10 minutes. The computing power (or hash rate) across all nodes determines the difficulty level needed for the PoW built into the Bitcoin protocol.
The demand to win the PoW reward increases as bitcoin interest increases and the bitcoin price per token increases. Enhancing the hash rate of a node relative to the overall hash rate for the entire Bitcoin network improves the probability to win the reward. This feature drove the conversion from CPUs to GPUs to improve the performance of solving this very specific computing challenge.
ASIC semiconductor equipment designed for PoW eventually beat out the GPUs on graphic cards plugged into desktop computers. The ASIC specialization and optimization won. PoW, also called mining, started to transition away from home-based hobbyists to dedicated ventures with access to capital to build out the required infrastructure.
As this transition started to accelerate, the primary constraint was access to capital to build out the infrastructure using dedicated facilities with racks of Bitcoin mining equipment built specifically for PoW. Many of these ventures went public to access even more capital. But as the scale kept increasing, the cost to acquire the energy and the perception this was not the most environmentally sound use of that energy became growing constraints.
PPAs between Bitcoin miners and utility companies evolved to improve the cost structure of energy if the computing load could be curtailed when the demand for utility grid energy peaked. This is more likely during hot summer days when air-conditioning use peaks in the late afternoon and evening.
This Peak may only last for 4-6 hours, but the spot price for energy increases dramatically during this Peak. This is when PPAs increase access to more energy by curtailing Bitcoin mining until grid energy demand normalizes. And Bitcoin miners are paid a premium to curtail operations to redirect electricity to the grid. Outside of Peak demand, the PPA terms provide electricity at a low cost. It is a win-win for all parties involved.
2. High Performance Computing evolves rapidly for AI and other applications
The introduction of ChatGPT in late 2022 accelerated the adoption of Large Language Models (LLMs) primarily for Generative AI applications. Microsoft had already partnered with OpenAI to accelerate access to this AI to improve the value of MS Office and related products, but no one really imagined the impact prompt-based AI would have on society. This has created exponential demand across many software companies and end-users to improve existing LLMs and create entirely new LLMs for specific vertical use cases.
NVIDIA, AMD, Intel and other semiconductor companies are all working to advance and manufacture the semiconductors used to train and operate these LLMs. But the GPU chip supply cannot keep up with the pace of demand. This is the primary constraint limiting the scale of LLM training right now. But the energy requirements to operate these GPU chips will become the primary constraint in the near future. Data Centers designed for CPUs and cloud computing requirements are finding they cannot easily adapt to the higher power density loads associated with the LLM workloads and the newest GPUs designed for these tasks. These latest chips are so specialized they should really be called Neural Processing Units (NPUs).
Training LLMs requires urgency to reach key milestones, but also allows curtailment to be designed into project plans during Peak energy demand to manage cost. The capital requirements, business model and core technology are different between PoW for Bitcoin mining and training LLMs for AI applications, but the Data Center infrastructure requirements are very similar and curtailment is an option to further monetize this infrastructure during Peak energy demand.
GPU chips used to train LLMs can also be used for other applications like visual effects (VFX) and rendering where 3D scenes are converted into a series of 2D images. This technology is often used for computer generated scenes in movies and other commercial uses of video. These are also workloads that can be curtailed if required.
3. Current deployments of High-Density Computing
Bitcoin miners like BitFarms, CleanSpark, Hut 8, Iris Energy, Marathon Digital Holdings, Riot Platforms, and others are prioritizing access to existing cheap energy sources around the world to offset the growing input energy cost to run ever increasing computing capacity. These energy sources are currently hydroelectricity, flared gas at well sites and natural gas produced in remote areas without adequate gathering pipelines connected to larger pipeline infrastructure. Texas also has a growing solar and wind infrastructure capacity.
The current Bitcoin mining strategy prioritizes back-filling stranded energy supply using high power-density loads. This energy use for Bitcoin mining must also be curtailed when electricity grid demand Peaks. The economics will favor building out transmission to urban areas to sell this energy during electricity demand Peaks.
Texas operates a deregulated electricity grid and marketplace through an entity called the Electric Reliability Council of Texas (ERCOT). The geographical size and energy resources in Texas allow ERCOT to operate almost exclusively within Texas.
Colorado, New Mexico, Utah and some other states also offer abundant undeveloped land with optimum solar and/or wind capacity to build out and consolidate more sustainable energy infrastructure to compete with ERCOT. Southern Australia offers a similar solar and wind energy opportunity in the southern hemisphere.
Many of these Bitcoin miners are also diversifying into HPC to provide revenue streams other than mining to offset the wide volatility in the spot bitcoin price at times during its 4-year cycle. This provides multiple high power-density options to balance curtailing energy use during Peak energy. Bitcoin mining is likely either 100% deployed or 100% curtailed, but HPC capacity could be scaled back proportionally to the spot price for energy. This optionality helps optimize revenue and margins.
2024 may exhaust most of the available cheap energy capacity. BitFarms negotiated favorable terms in Argentina for gas-powered energy during non-winter months when the demand for natural gas is much lower than the winter when natural gas is also used to heat commercial buildings and homes.
Argentina’s natural gas reserves are somewhat trapped and limited to domestic use without a pipeline and LNG infrastructure to export natural gas when domestic demand is low. Thus, BitFarms provided a unique opportunity and was already located in Argentina for Bitcoin mining.
Iris Energy is a Sydney-based company in Australia, but its current High-Density Computing operations (load centers) are located in North America with most of the original infrastructure based in British Columbia, Canada. Favorable economics in Texas through ERCOT with abundant solar and wind infrastructure offered an opportunity to expand into Texas north of Dallas. Curtailment is required during Peak energy demand, but the negotiated Off-Peak energy price for surplus sustainable energy improves the economics their High-Density Computing for Bitcoin mining and HPC data centers.
Hut 8 negotiated a PPA with a power plant provider in North Bay, Ontario, Canada called Validus Power Corp (VPC) at the time. VPC failed to honor the supply of energy based on the agreed terms. VPC filed for bankruptcy mid-2023. Hut 8 relocated their Bitcoin mining equipment to help monetize those stranded assets, but also submitted a stalking horse bid for all 4 of the natural gas power plants tied up in VPC’s bankruptcy. Hut 8 won the bid for these assets mid-December 2023.
Hut 8 is vertically integrating into both electricity generation and High-Density Computing across Bitcoin mining and HPC data centers. Hut 8 also holds over 9,000 bitcoin tokens on their balance sheet and is a public company (NASDAQ: HUT). The company has multiple options to access and deploy capital to fund development projects and operate High-Density Computing through its growing infrastructure.
New York recently announced Tesla will build a $500 million “Dojo” supercomputer and data center at or near Tesla’s existing factory in Buffalo, New York. This will expand their compute infrastructure to support neural network and LLM training.
Buffalo is strategic for Tesla because it is close to abundant hydroelectric capacity. Elon Musk has stated in the near future, the supply constraint limiting the rate AI advances will transition from access to state-of-the-art semiconductors to access to adequate energy.
4. Future deployments of High-Density Computing
Remote locations with abundant sustainable energy sources in reasonable proximity to urban areas are prime locations to setup and expand High-Density Computing with the option to curtail energy use during periods of Peak grid electricity demand.
These projects should stage development while also accelerating the build out of solar and other means of sustainable energy as the low cost energy option to scale High-Density Computing capacity over time. As the unit economics for solar panels improve with scaled production, the unit economics also improve for commercial and residential buildings to transition to generating their own solar energy. This accelerates the conversion to even more solar energy capacity. Virtual power plants (VPPs) resulting from scaled solar capacity and stationary batteries can also sell excess solar energy to the electric utility grid to service those who need electricity.
Capital will develop to fund these projects when returns are attractive. Many companies with available capital can benefit downstream as the resulting computing operations drive more value through better technologies. This has a multiplying effect where a dollar invested to build out the required sustainable energy and High-Density Computing infrastructure generates many dollars downstream through secondary and tertiary effects.
Some inefficiencies may develop as VPPs ramp up within urban areas to reduce the need for electricity transmission from remote areas building out solar and wind farms primarily for High-Density Computing. This may drive an energy surplus in these remote areas, but that just helps attract more businesses where energy is a key input cost.
Stationary battery storage deployments will also grow to meet the needs for growing solar and wind farms. Solar in particular needs to generate and store about 5x the energy generated during daylight to meet the needs of constant energy demand over a 24-hour period. Otherwise, energy demand needs to be reduced when solar energy is not being generated overnight.
The rate and scale that High-Density Computing builds out its capacity for both sustainable energy and co-located computing will be driven primarily by the bitcoin price and the volatility of this price over the next 8-12 years and beyond. Speculation has been the primary driver for the bitcoin price so far, but recently launched bitcoin Exchange Traded Products (EDP) from companies like Blackrock and Fidelity Investments will help drive institutional investment and lower volatility.
Some Final Thoughts
The concept of scaling deployments of High-Density Computing and building out sustainable energy capacity with miles of solar and wind farms initially seems far-fetched. However, the current electricity use in the US could be generated with less than a 100 mile x 100 mile grid of solar panels in Texas.
Energy loss with electricity transmission over long distances and the growing demand for more electricity with EVs and economic growth would drive the need for the generation of even more solar energy. And sustainable energy generation really needs to be located everywhere and closer to where it is used as growth scales.
But this highlights there are abundant sustainable energy resources with undeveloped land. Co-locating the build out of sustainable energy capacity with High-Density Computing helps to return capital deployed into this energy infrastructure faster. These returns are even better when the sustainable energy can be sold into an energy market when the demand for energy Peaks. Texas is recognized as one of the best markets for this business model.
And to answer the question “What happens when a Bitcoin miner acquires four power plants?” The answer is an indicator for how strategic abundant, cheap, reliable energy is to the success for High-Density Computing. Hut 8 is backwards integrating into their key input cost (electricity) to mine Bitcoin with a unique opportunity in North Bay, Ontario. This is a proxy for what could scale next through a wider network of companies in Texas with a larger sustainable energy ecosystem and market.
In the next two Parts of this Series, I will go into more details on why there will likely be a growing demand for High-Density Computing with Bitcoin mining and HPC over many decades. This can also accelerate the transition to sustainable energy beyond scaling production and deployments for EVs, charging stations, and stationary battery storage.
Best,
Stephen
I’m long BTC, HUT, IREN and TSLA mentioned in this post. Nothing in this Update is intended to serve as financial advice. Do your own research. The opinions and views expressed in this newsletter are those of the author. They do not purport to reflect the opinions, views or policies of any other organization, company or employer.