[E14.3] High-Performance Computing: AI changes the game
Advances in artificial intelligence drive the need to scale computing power to train and operate neural networks and large language models
Dear Reader,
In this Post for Product | Strategy | Innovation I will discuss the growing opportunity for High-Performance Computing (HPC) in Part 3 of a 3-Part Series on Power Density as a catalyst for change. Generative AI applications, computer vision, visual effects rendering and molecular simulations are driving rapid innovation for HPC with Power Density at least 10x what is required for legacy cloud computing.
And the Power Density for high-end GPU semiconductors used to train Large Language Models (LLMs) for generative AI applications has more in common with the latest bitcoin mining technology than cloud computing data centers. This is based primarily on the energy requirements to operate these high-end GPUs and future chip designs from companies like Nvidia, AMD and Arm Holdings.
In this Post, I will first provide some Background on HPC as a business line for High-Density Computing. Then, I will cover HPC in 3 sections plus some Final Thoughts.
HPC applications: generative AI (training and inference), computer vision, visual effects rendering, modeling and simulations
HPC innovation: 3 waves of innovation over the next decade
HPC augments 3 Levers for Growth to drive High-Density Computing at scale
Sustainable energy is the future. We should use all available levers to accelerate the transition to sustainable energy. HPC provides another strategy for High-Density Computing to diversify and scale in-demand revenue-generating services. Energy is a primary input cost and low-cost sustainable energy is a key advantage. Reliable, growing revenue enables deployment of new capital to scale infrastructure to generate more co-located, low-cost sustainable energy.
Background
Even though this Post is focused on HPC, the journey to where we are today with High-Density Computing really started with the evolution of semiconductors used for bitcoin mining. The original bitcoin miners used desktop computers around 2009 with CPUs to run the Proof of Work (PoW) consensus to mine bitcoin. But as more nodes were added to the bitcoin network, the aggregate computing power increased significantly. This caused the Bitcoin protocol to increase the PoW difficulty.
So bitcoin miners started adding GPU-based graphic cards used for gaming to their computers in 2010 to gain an advantage and developed software to enhance computing specific to the PoW consensus using these GPUs. AMD and Nvidia were the preferred manufacturers for these graphics cards. Bitcoin mining became a windfall for Nvidia in particular and a new business opportunity beyond computer gaming using the same advanced graphics cards. These bitcoin mining operations were mostly run at home as hobbies or inside small-businesses.
But as the business opportunity improved for bitcoin mining with increasing prices for bitcoin, the mining transitioned into larger operations in dedicated facilities. Bitcoin mining equipment was also developed around ASIC semiconductors in 2013. As the ASIC equipment improved, thousands of these machines could be run under one node to scale the aggregate computing power behind the node. But soon the cost of energy to operate these machines became a constraint to scale the number of machines deployed and run continuously around the clock.
As mentioned in the prior Post, bitcoin mining with PoW consensus is the primary driver for High-Density Computing based on its Power Density and its feature that enables self-custody for a speculative digital asset (bitcoin) on the miner’s corporate balance sheet to help fund strategic investments into future mining capacity. Some miners liquate the bitcoin they earn immediately to enhance cash flow.
High-Density Computing has evolved into public corporations like Marathon Digital Holdings, Riot Platforms, Hut 8 and IREN that specialize in designing, building and operating infrastructure for Data Centers with high Power Density. The demand for energy is also driving the need to co-locate with abundant, low-cost, sustainable energy generation.
High-Density Computing infrastructure has 3 primary levers to hyperscale growth:
Semiconductor Technology
12-18 month strategy and deployment roadmap to allocate semiconductor capacity across the different technologies based on market dynamics
Core semiconductor infrastructure
HPC applications: GPUs and future chip designs
PoW consensus: ASICs for bitcoin mining
Advanced Innovation
New chip technologies
Single- and 2-phase immersion technology to cool semiconductors, reduce noise and improve reliability
Other technologies
Capital
12-18 month strategy and project plan to allocate and raise capital to grow and operate business lines
Balance sheet
Bitcoin
Self-custody of bitcoin through mining operations
Other opportunities to acquire bitcoin
Cash
Bitcoin sales to generate cash
HPC operations to generate cash
Curtailment of High-Density Computing operations to generate cash
Capital Markets to generate cash
Equity: IPO and follow-on offerings
Debt: bank loans, corporate bonds, and convertible notes
Corporate Development and Project-based Financing: joint ventures, partnerships and other business structures
Sustainable Energy
3-year strategy and deployment roadmap for sustainable energy generation to operate and scale High-Density Computing power
Hydroelectric (if available)
Solar farm
Wind farm
Stationary Battery Storage
Geothermal
Nuclear
Other forms of sustainable energy (onsite and new locations)
Purchase Power Agreements (PPAs)
Off-Peak electrical utility power demand: access to surplus, co-located, low-cost energy
Peak electrical utility power demand: curtail computing operations to sell energy to electrical utility grid
Vertical integration into sustainable energy generation & stationary battery energy storage for more control over the sustainable energy roadmap
These 3 levers to hyperscale growth result in 9 tools when each lever has 3 core attributes. And the biggest impact of diversifying High-Density Computing across HPC and PoW consensus is a robust, reliable and growing demand for energy. This allows use of the capital markets to fund the build out for more sustainable energy generation over time.
HPC usually includes 4 key attributes. First, computing is configured into a supercomputer or cluster of multiple discrete computing nodes connected by a high-speed network. Second, computing is divided into smaller, independent tasks that can be run in parallel (simultaneously) on different nodes in the cluster. Third, the data required for computing are distributed across these nodes, so each node has a portion of the data to process. Fourth, the outcome is the result or rendering of the computing distributed and completed across all the nodes in the cluster.
Tensor Processing Units (TPUs) are a semiconductor innovation aligned specifically with machine learning with HPC. As this technology evolves, it could challenge GPUs with greater speed and maybe better results. But GPUs have such scale and ongoing innovation across many applications, they will be hard to replace near-term.
1. HPC applications: machine learning/large language models (training and inference), computer vision, visual effects rendering, modeling and simulations
HPC gained early adoption in modeling and simulations used for computer gaming. PC computers could be equipped with GPU-based graphics cards to improve resolution and speed. This has evolved into more specialized HPC for visual effects rendering where a stream of 2D images can generate video from a 3D computer scene. GPU-based clusters provide dedicated on-premise or virtual render farms.
Neural networks have been used for decades to mimic many connected neurons in the human brain. Deep learning as demonstrated in the 2012 ImageNet competition with AlexNet takes neural networks even further with at least 8-10 network layers, inexpensive computer hardware optimized to process image data fast and over 1 million labelled images to train models. This research led Tesla to pursue computer vision with deep learning for autonomous navigation of its vehicles.
Computer/machine vision requires a very comprehensive technology stack that spans concurrent image acquisition with multiple high-resolution cameras, real-time image processing using inference based on a trained model, control of the vehicle or robot with the output of the trained model for autonomous navigation, and collection of exceptions (or edge cases) when a supervisor over-rides autonomous navigation.
Tesla centralized HPC to train its Full-Self Driving (FSD) models using both Nvidia GPU-based clusters and Tesla’s Dojo supercomputer developed specifically for computer/machine vision and autonomous navigation. This work at Tesla has now evolved to include computer vision for electric vehicles and humanoid robots.
The latest iteration of autonomous navigation at Tesla is FSD Beta v12.3. This version uses an end-to-end neural network to convert the real-time video streamed from the onboard cameras into real-time control of the speed, direction, braking, lighting and signals for the vehicle. FSD Beta v12.3 is limited to approved beta testers who must supervise FSD at all times and intervene if needed. These exceptions trigger uploading all the data before and after the intervention to Tesla to determine if the data should be included in future FSD model training.
The aim of Tesla’s Dojo supercomputer is to scale HPC capacity significantly as the Tesla EV fleet scales to accelerate the pace of innovation for FSD towards level 4 and level 5 autonomous navigation. The objective is to realize robotaxis using Tesla EVs equipped with FSD approved by regulators for autonomous navigation.
Generative AI entered the market in 2022 with ChatGPT based on a transformer model developed by Open-AI to enable prompt-based generation of text. Other models provided prompt-based image generation. This technology has evolved quickly from GPT-3 to GPT-4 to GPT-5 and beyond to include text or voice conversation with robots and generating text, images and video from only a text or voice-based prompt.
Applications of generative AI are being used in vertical industries but with more general purpose learning. The next wave of innovation is to add proprietary data in a specific vertical industry to optimize generative AI for that vertical industry. This means training these Large Language Models (LLMs) will continue and expand rapidly as general purpose and more specialized models require ongoing training.
The current choke-point limiting the HPC innovation are the supply constraints for the latest GPU semiconductors. Nvidia is now the 3rd largest company in the S&P 500 behind Microsoft and Apple with a market cap over $2 trillion as of March 15, 2024. But with the rapid deployment of these semiconductors with increasing Power Density, the choke-point will soon become the energy capacity to power HPC around the clock at scale.
2. HPC innovation: 3 waves over the next decade
Nvidia, AMD, Arm Holdings, Amazon, Microsoft, Alphabet, Meta, Tesla, IBM and other companies are all designing semiconductors to advance AI innovation for years to come. Semiconductors usually dominate the initial wave of a step change in computing technology. Once a chip design validates commercial assumptions with initial production, the choke-point becomes scaling production to meet demand in the market.
Wave 1
S-curves in adoption usually allow chip production to scale while the market is limited to 12-15% with early adopters in key markets. The diffusion of innovation takes year. But ChatGPT short-circuited adoption to the majority worldwide when consumers were able to test generative AI for free. Nvidia was already working with Open-AI to build their HPC capacity to support its generative AI innovation. Nvidia used its success with Open-AI to showcase its roadmap to support the required HPC across many industries. Microsoft was already on-board with its investment and partnership with Open-AI.
The race was on in 2023 to realize the first wave of innovation for generative AI with the semiconductors to eventually train LLMs across almost every industry in addition to more general purpose generative AI. This first wave will be the dominant wave through 2025 while semiconductors are still the primary choke-point for innovation. But as the production of GPUs and other chip architectures scale to realize the HPC infrastructure needed for generative AI, the second wave of innovation will be realized through the hyperscalers and High-Density Computing data centers to emerge and scale through 2030.
Wave 2
Hyperscalers like Amazon AWS and Microsoft Azure will retrofit GPU-based capacity within existing cloud computing locations until energy costs become a constraint to scale High-Density Computing further. The hyperscalers will likely bifurcate computing infrastructure into 2 primary models within 2-3 years. Legacy cloud computing will continue to grow within urban areas to support mission-critical, real-time computing at near the edge of demand. But High-Density Computing will chase low-cost, abundant sustainable energy wherever it can be found.
The unknown is what business model emerges to realize this need. Bitcoin mining will be a primary driver for the build out of sustainable energy based on the Power Density of its ASIC chips and the growing scale of these operations. But as AI-based training also scales, this will provide a competing need for sustainable energy. Unless the hyperscalers want to enter bitcoin mining, they may be best served by partnering or creating JVs with the larger bitcoin mining operations to consolidate and cooperate on the build out of sustainable energy at scale to support High-Density Computing. This could realize solar, wind and stationary battery deployments with many gigawatts of power. Or a modern day nuclear power plant with the latest technology.
This second wave of innovation will last for many years, but it will dominate the first wave within 2-3 years once the semiconductors to build out AI-based training infrastructure are readily available from multiple manufacturers including the hyperscalers themselves. Nvidia may continue into this 2nd wave by building out its own High-Density Computing infrastructure.
The second wave of innovation is dominated by batch-based training wherever energy is the lowest cost. This training can be curtailed during Peak electrical utility demand and then resume when Non-Peak demand returns. Or a high-priority batch of training can be diverted to another part of the world with Off-Peak electrical utility demand if the primary location is curtailed due to Peak demand.
This will likely be realized through AI-based “HPC pools” that aggregate specific High-Density Computing across multiple locations around the world with low-cost sustainable energy to manage periods of curtailed computing due to Peak electrical utility demand in a specific region. This is the business model used for bitcoin mining where nodes opt-in to a specific “mining pool” to aggregate their High-Density Computing capacity to compete against the total High-Density Computing capacity across all nodes making up the dynamic bitcoin network.
Wave 3
The largest and most sustainable wave is the 3rd wave of innovation. Here the semiconductors and High-Density Computing infrastructure are realized to unlock all the applications that can leverage the foundational AI-based technology. Companies like Microsoft, Salesforce, Palantir, Oracle, Adobe, and others will build out many enterprise, small-to-medium-sized business, and consumer applications on top of AI-based technologies.
This 3rd wave will also require inference to operate these AI-based technologies by end users. This is more efficient when handled by the smart device, computer, vehicle or robot using the AI-based technology. This will kick-off another wave of innovation for semiconductors and firmware focused on improving efficiency and lowering power requirements for inference on these devices and systems. Tesla is designing its own FSD semiconductor chips and FSD computers to provide inference onboard each Tesla EV.
3. HPC augments 3 Levers for Growth to deliver High-Density Computing at scale
I previously outlined 3 Levers for Growth for High-Density Computing operators in the Background of this Post. Fig. E14.3-2 also highlights attributes available to these operators in each lever. But a key requirement is the foundational expertise these operators have to design, build, operate and exploit the value these levers can unlock to scale the infrastructure over time.
And this infrastructure may need to scale across different geographies to position the right opportunity for the right conditions. Standardization across data center facilities allows the highest power density equipment to be deployed where energy costs are lowest. That will most likely be ASIC-based bitcoin mining equipment. HPC can backfill the vacated racks to monetize operations within the data center.
The operators expertise must include sourcing and acquiring land best suited for sustainable energy generation with existing surplus energy and available space to add more energy generation capacity over time. This land needs improvements with access to highways to bring in materials and staff from surrounding communities. Gravel and concrete pads need to be developed to house data centers. Electricity, fresh water, waste water, LED lighting, HVAC, communication networks and standard computer equipment racks also have to be integrated into these facilities.
Monetization strategies and business cases are needed for various HPC solutions based on the revenue generation possible and input costs required to fund specific semiconductor technology. Accuracy to estimate an opportunity within a few days is often more important than precision over multiple weeks when the flow of opportunities is almost continuous.
Diversification of revenue streams strengthens the business case for High-Density Computing to fund the build out of sustainable energy generation. HPC generates cash. PoW consensus generates bitcoin. Curtailment of computing during Peak electrical utility demand also generates cash.
Joint Ventures with cloud-computing hyperscalers on High-Density Computing infrastructure could accelerate vertical integration and scaling the co-located buildout of sustainable energy generation. Hyperscalers like Amazon, Microsoft, and Alphabet could also benefit from the expertise of High-Density Computing operators for HPC.
Larger addressable market business opportunities to scale sustainable energy generation capacity accelerates capital raises in public markets to also build out the right High-Density Computing infrastructure and semiconductor technology.
Curtailment of High-Density Computing during Peaks in energy demand can also be more variable when HPC is added to PoW. HPC curtailment can be proportional to the spot energy price vs. more binary with PoW consensus for bitcoin mining.
Diversification of revenue streams also poses a challenge if advances in different semiconductor technologies happen close to the same time. Ideally, step changes in capabilities and/or performance across different technologies happen at different times to help manage resource allocation, but when they overlap, difficult decisions are needed on how to prioritize and allocate capital and resources.
But the opportunities seem to outweigh the challenges. It will be interesting to see how HPC within the context of High-Density Computing evolves over time. Another observation across both bitcoin mining and HPC is a growing trend to aggregate computing resources from multiple companies and sites through a third party to scale capacity and redundancy. This third party “pool” builds a pipeline of contracted work with end-users and aggregates the required High-Density Computing. An analogy would be owning a grape vs. being part of a watermelon. If the watermelon keeps getting bigger then this becomes more attractive. I just need to maintain or grow my portion of the watermelon.
High-Density Computing operator IREN (previously known as Iris Energy) secured a partnership with Nvidia and Poolside AI in February 2024 to deploy the latest H100 GPUs to train AI models. The partnership allows the Nvidia GPU deployment to scale at IREN as the demand to train AI models through Poolside also scales. Bitcoin mining pools include Foundry USA and AntPool.
Some Final Thoughts
Demand for generative AI applications like ChatGPT enabled by the latest advances in semiconductor technology provides a growing market for HPC. The Power Density for the latest semiconductors to train the models for these generative AI applications is around 10x that for more conventional cloud computing. And as these AI applications expand from general purpose to almost every vertical market, the demand to train the AI models will continue to scale over the next decade.
Another advantage of this use case for HPC is the ability to train AI models in batches during Off-Peak energy demand when the cost for energy is lower. HPC can also be curtailed just like bitcoin mining during periods of Peak energy demand to generate cash under the terms of a Purchase Power Agreement.
Diversifying High-Density Computing with HPC opportunities helps to build more reliable revenue streams and reduce the volatility of boom and bust cycles associated with only one primary semiconductor technology. HPC with high Power Density also opens the opportunity for partnerships, project-based financing, and JVs with well-funded strategic partners like Amazon, Microsoft and Nvidia who will need the expertise of High-Density Computing infrastructure operators.
The upside impact of HPC on the addressable market for High Density Computing will help accelerate the build out for more capacity to generate and store sustainable energy. These build outs are aided by the co-location of High Density Computing with the sustainable energy generation and storage. Long delays in completing permits and interconnects between new energy generation and/or storage and electrical grid utility infrastructure makes more conventional projects challenging to realize similar returns on the investment.
High Density Computing fast tracks the build out of co-located sustainable energy with an immediate load for any surplus energy. This accelerates the transition to sustainable energy as its capacity scales and integrates with electrical grid utility infrastructure.
Best,
Stephen
I’m long BTC, IREN and TSLA mentioned in this post. Nothing in this Update is intended to serve as financial advice. Do your own research. The opinions and views expressed in this newsletter are those of the author. They do not purport to reflect the opinions, views or policies of any other organization, company or employer.