Axel Kloth
High-Performance Computing (HPC) refers to the practice of aggregating computing power in a way that delivers much higher horsepower than traditional computers and servers. Organizations use HPC to process large amounts of data quickly. The power of HPC helps humanity tackle some of the most vexing problems in business, engineering, and science.
The HPC market is growing steadily. Emergen Research reports that the worldwide HPC market will grow from US$42 billion in 2021 to US$72 billion in 2030, driven in part by digitization and cloud computing. While HPC is different from Cloud Computing as provided by the hyperscalers, HPC-as-a-Service is emerging, and some trends in HPC and the hyperscaler community converge. As the market evolves, HPC is at a crossroads between today’s current generation of supercomputers and the quantum and optical computers of the future.
The Emergence of Computing Humans have used math for millennia. Early on, humans likely used their fingers to add and subtract, and over time more complex math emerged. That necessitated tools such as the abacus and it led to the most complex mechanical device for solving math problems: Charles Babbage's Difference Engine. That mechanical complexity exceeded then-current manufacturing capabilities, leading to a focus on the then-new analog electronics. Over time, analog electronics turned out to be not precise enough for applications like bookkeeping and vote counting; therefore, digital electronics started to prevail. First used in non-programmable fixed-function calculators for the four basic math functions, digital electronics have taken over, leading to fully programmable computers.
Why do we use computers? We use them to keep score, analyze the past, and predict the future.
• Keeping score only requires that the computers store data, requiring minimal processing.
• Analyzing the past is much more mathematically complex since it requires storing, processing, and retrieving data – but it does not have any real-time requirements.
• Real-time requirements arise when a computational response is needed immediately. When a driver slams on the brakes, he or she expects the computer controlling the anti-lock brakes to do its job right away, not at some point in the future.
• Predicting the future is complex and it typically means working with large data sets and equally large instruction sets derived from physics and from experience. The underlying science might not be fully understood, but it is empirically derived.
Powerful applications run on supercomputers in the field of HPC. Most people have watched the weather forecast to decide whether it is a good idea to have friends over during the upcoming weekend. Weather forecasting is an HPC application since very large-scale data sets are taken in and then processed. This is accomplished using known behaviors of the weather in small cells, which is then extended to predict weather around the globe. What makes it challenging is that the results must be in before the predicted event takes place.
Technology Enabling Process
In the process of building these computers, processor architects and ASIC design engineers have learned to build processors, accelerators, and memories with ever-increasing densities. This density increase is oftentimes referred to as Moore’s Law, named after Gordon Moore of Intel. Early on, he observed that the number of transistors on any given chip doubled every 18 to 24 months. A similar observation applies to storage density of hard disks. This density increase allows digital electronics to penetrate every aspect of our modern life.
Moore’s Law has broken capability barriers that prevented individuals, society, companies, and organizations from advancing their offerings and services. A supercomputer from the 1990s – costing more than US$10 million – had barely the same computational performance as a 2020 smartphone costing US$800. This quantum leap allowed the IT industry to offer services for which the capabilities simply did not exist in 1990, 2000, or even in 2010.
We now collect data routinely and store it without even thinking about why we are doing it or what we want to use the data for. As a result, we amass more data than can be turned into information. This creates a challenge in that we do not always know how to extract new information from the data we already have. Big data was supposed to solve this problem, but it did not. Now artificial intelligence is considered a potential solution.
Learning from Data
The challenge is compounded by the fact that storing data has become so inexpensive that it does not make sense to spend human resources to sift through all the data we have collected. We live in an age at which we keep storing data that we might never need or exploit, but we somehow must manage it. This is increasingly becoming an issue for individuals, society, companies, and organizations including hyperscalers – such as Google, Apple, Microsoft’s Azure, and Amazon Web Services and social media companies like Facebook, Twitter, and TikTok.
“As the market evolves, HPC is at a crossroads between today’s current generation of supercomputers and the quantum and optical computers of the future.”
The more computing the hyperscalers provide, the more will be used and absorbed. The more data we create, the larger the pool of data we will need to chew through. At the same time, humans face unprecedented challenges. We need to provide food and fresh drinking water to over 7 billion people, while ensuring that people have a roof over their heads, access to education, and enough energy.
We need to ensure that their ambition leads to results. That in turn requires that every decision we make supports 7, 8, or even 10 billion people on this planet. We cannot leave decisions like this to unfounded opinions; we must be able to create a digital twin of earth and see what decisions have an impact that supports those goals, and which don’t. We will be better off the more decisions we analyze, including the impact of those decisions.
Accomplishing that will require more supercomputers that provide computational performance of levels beyond 1 ExaFLOPS (1 * 1018 or 1000000000000000000 Floating Point Operations Per Second or the performance of half a million M1-equipped Macs) at less than 20 MW of electric power including cooling.
Current supercomputers are built from commercially available servers that are connected via high-performance switches; sometimes they are equipped with general-purpose graphics processing units (GPGPUs) to accelerate the execution of certain mathematical functions. The way they are built today prevents a linear scale-out. Right now, a supercomputer with one million cores does not provide one million times the performance of one core. We are far from that, but we need these machines to scale linearly (or nearly linearly) for most applications we want to run on them, to solve above challenges.
Today’s Challenge & Opportunity
We are now in a predicament that requires a medium-term solution. Longer-term, we will likely have quantum computers with quantum memories and quantum pipelines to solve our computational problems. Right now, we lack performance scalability. While quantum computer research has progressed rapidly over the past 10 years, a few things are not yet available in quantum computers. There is still no such thing as a processor pipeline conceived for a quantum computer.
An important missing piece is the equivalent of the working memory in a traditional computer, usually referred to as DRAM. For data input and output, quantum computers will have to rely on traditional digital computers with analog-to-digital converters for input and digital-to-analog converters for output to actuators. Even quantum computers will rely on digital I/O frontends for pre- and post-processing of all input and output data. We can’t throw away our digital computers; not now, and not in 25 years.
Looking Ahead
We will also still need digital computers today as the basis for linearly scalable large-scale systems as a next-generation supercomputer before we can transition over to quantum computers. Several startups are working to meet this challenge and when they succeed it will enable us to tackle even more of the problems that humanity faces in the areas of business, technology, and society.
Axel Kloth is a Partner at Pegasus Tech Ventures. Axel is also the founder and CEO of Abacus Semiconductor Corporation. Abacus Semi is a fabless semiconductor company that is rethinking High Performance Compute and its energy footprint as well as its performance while at the same time making it easier to use. Axel is a serial entrepreneur and has been a Founder, Chief Technology Officer, Vice President of Engineering and System Architect in several high-tech ventures for over 30 years.
