ChatGPT, now ubiquitous to the point that imagining life without it is difficult, demands tens of thousands of GPUs to operate, with electricity costs exceeding KRW 600 billion alone. Addressing energy concerns is crucial for further advancement in AI. This is where the new DPU processors come into play. According to Allied Market Research, the DPU market is projected to grow by 26.9% annually, reaching $5.5 billion by 2031 from $553 million in 2021.

 Just a decade ago, artificial intelligence wasn't the pervasive technology that it is today. The turning point came with AlphaGo, capturing global interest and sparking widespread fascination with AI. Since then, AI has witnessed a surge in popularity, technological advancements, and commercialization, as evidenced by the launch of ChatGPT last year. With the exponential growth in data volume, the demand for GPU for AI learning and inference has soared, while the role of CPU in data processing has diminished. Whereas general-purpose CPUs were once efficient for traditional computations, DPUs have emerged as the solution to power consumption and price concerns, excelling in data transmission to GPUs and processing of results.

Since the advent of the first computers, we have lived in the era of the Von Neumann structure CPU. However, as the gaming market expanded, the need for handling vector operations arose for graphics, leading to the introduction of GPUs. Over time, GPUs evolved to efficiently compute large amounts of real-valued vector data in parallel, particularly in tasks like 3D rendering. While CPUs excel at swiftly performing single tasks such as simple math, the parallelism offered by GPUs proves more beneficial when processing extensive datasets. In the era of AI, algorithms like deep learning demand the calculation of vast amounts of real-valued data, making GPUs the preferred choice due to their superior ability to parallelize such data compared to CPUs, which are primarily designed for sequential processing. Indeed, the use of GPUs has become indispensable for AI development, with AI model training being approximately ten times faster on GPUs compared to CPUs. Especially with the recent advent of Artificial General Intelligence (AGI) technology, initiated by innovations like ChatGPT, AI models have grown in complexity, leading to the establishment of numerous large-scale AI infrastructures such as data centers. This surge underscores the imperative to reduce costs and power consumption. In response, simpler and more cost-effective DPUs, specializing in data processing, have emerged as viable alternatives to traditional general-purpose CPUs. These DPUs are frequently deployed in high-capacity data center computing environments.

This is going to represent one of the three major pillars of computing going forward,” stated NVIDIA CEO Jensen Huang.

“The CPU is for general-purpose computing, the GPU is for accelerated computing, and the DPU, which moves data around the data center, focuses on data processing.”

If a CPU is a jack-of-all-trades that specializes in general-purpose computing, a DPU is a specialist in one thing: data processing, which is efficient and economical. Originally created for gaming, GPUs excel in accelerated computing. Games require constant vector math for graphics, necessitating parallel processing. This capability allows for faster and more complex calculations to be performed simultaneously. GPUs are primarily employed for 3D rendering in games, AI training (model creation), and inference (model utilization).

CPU stands for Central Processing Unit, GPU stands for Graphics Processing Unit, and DPU stands for Data Processing Unit. But what about TPU, IPU, and NPU?

First, TPU stands for Tensor Processing Unit, developed by Google for AI research. NPU, on the other hand, refers to Neural Processing Unit, a semiconductor designed for constructing artificial neural networks akin to human cognition.

While these products share similarities with GPUs, they focus more on AI computation and less on graphics processing.

However, IPU diverges from the traditional approach and restructures to mitigate the GPU's computational latency. By integrating the core and RAM within a single processor, these components are in close proximity, operating in a 1:1 configuration. According to Graphcore, the company behind the IPU, it outperforms GPUs by up to 18 times in AI training and up to 600 times in AI inference.

In Korea, MangoBoost, a startup founded by Jangwoo Kim, a professor of electrical and information engineering at Seoul National University, continues to attract investment and expand, with global tech giants actively participating. In 2020, Nvidia acquired Israeli semiconductor company Mellanox Technologies to further accelerate DPU development. In the era of artificial intelligence, it's time to bid farewell to the CPU, which has been around for decades, and welcome the new DPU market. Once DPUs are developed and deployed, they will address both economic and performance issues.