Moore’s Law, the guiding concept in computing, is an observation made by Intel co-founder Gordon Moore. According to this law, the number of transistors on a device doubles roughly every two years, hence increasing performance. For years, this idea has driven the semiconductor sector.
In the fast-paced world of technology, not many ideas have been as impactful as Moore’s Law. The principle, first highlighted in 1965, predicted that the number of transistors on a microchip would double about every two years, resulting in rapid advancements in computing power. Yet, it has been observed that the transistor count does not merely follow Moore’s Law but surpasses it in significant ways.
With that thought, AIM has put together this article highlighting the number of times Moore’s Law was challenged.
NVIDIA’s Blackwell GPU Launch
NVIDIA CEO Jensen Huang unveiled the company’s next-generation Blackwell GPU at the NVIDIA GTC 2024, biding adieu to the era of Moore’s Law.
The GPU architecture features 208 billion transistors and processes trillion-parameter AI models 30 times faster than previous technologies. This advancement raises questions about whether NVIDIA has surpassed the limitations of Moore’s Law by achieving exponential increases in computational speed within a shorter timeframe than traditionally expected.
Reflecting on the rapid evolution of computing power, Huang highlighted that NVIDIA has achieved a thousandfold increase in computational capacity over the past eight years—surpassing the traditional benchmarks set during the peak of Moore’s Law. Yet, he lamented that despite this extraordinary progress, the industry’s ever-growing demands remain unmet.
Meanwhile, Intel has remained firmly committed to Moore’s Law, continuing to invest heavily in advancing transistor scaling. In contrast, Huang declared Moore’s Law “dead” and said he sees the emergence of what he calls “Hyper Moore’s Law” as the future.
Intel: Moore’s law is not dead.
Nvidia: Moore’s law is dead.
Who’s right? Check the market caps.— Pedro Domingos (@pmddomingos) March 5, 2024
The Shift to ‘More than Moore’ Strategy
In 2016, the International Technology Roadmap for Semiconductors (ITRS) shifted its focus from strictly adhering to Moore’s Law to a ‘More than Moore’ strategy. This change was driven by the realisation that advancements in semiconductor technology would need to prioritise application-specific innovations rather than merely scaling down transistor sizes. This strategic pivot indicated a recognition of the limitations imposed by traditional scaling approaches.
SambaNova Transforming the Hardware & Software Landscape
The demands of ever more AI have pushed classical computing hardware and software to its limits.
That is precisely what software-defined AI hardware startup SambaNova Systems is working towards with its Reconfigurable Dataflow Architecture (RDA), which reimagines how we can free AI from the constraints of traditional software and hardware.
The challenge lies with the amount of computing power needed to build and train many of the more advanced models being developed. The models are getting larger and larger, and for some applications, the volumes of data required to train them are also expanding. According to SambaNova’s vice president of product, Marshall Choy, this is caused by the slowing of performance gains for successive generations of processor chips, a trend that some have labelled the end of Moore’s Law.
AMD’s Claim About Moore’s Law
AMD CTO Mark Papermaster believes that Moore’s Law will remain relevant for another six to eight years. This view contrasts sharply with that of other industry leaders, including Huang, who declared Moore’s Law dead.
Papermaster acknowledged that while the doubling of transistor density every 18 to 24 months may not continue under the same cost constraints, AMD anticipates advancements in transistor technology that will sustain performance improvements in the near future.
According to the CTO, the company believes that Moore’s Law will remain relevant for another six to eight years, although it acknowledges that the doubling of transistor density every 18 to 24 months may not continue within the same cost constraints.
AMD is shifting focus towards chiplet architecture, which Papermaster describes as a “Moore’s Law equivalent”.
Intel’s Acknowledgment of Moore’s Law Limitations
Intel has openly acknowledged the challenges posed by physical limitations as transistors approach atomic scales. The company has indicated that while it continues to innovate, the traditional doubling of transistor counts is no longer a feasible goal, reflecting a broader industry shift away from strict adherence to Moore’s Law.
Earlier, it faced significant delays in bringing its 10nm and 7nm chips to market, raising concerns about its ability to keep pace with Moore’s Law.
The company acknowledged that while it once led in semiconductor innovation, it now grapples with the realities of physical limitations and increasing manufacturing costs.
Despite initial promises, it faced multiple delays, leading to concerns about its ability to keep pace with competitors like Taiwan Semiconductor Manufacturing Company (TSMC) and Samsung. This stagnation reflects the broader industry trend of the expected advancements in chip technology not materialising as quickly as anticipated, challenging the viability of Moore’s Law in practical terms.
Lightmatter’s Shift to Photonic Computing
Lightmatter is developing photonic computing technologies that aim to address the limitations of traditional silicon-based chips. As demand for AI processing power grows, Lightmatter’s approach reflects a shift away from merely scaling down transistors towards entirely new computing paradigms. This transition underscores the challenges faced by conventional semiconductor technologies in meeting modern computational demands, indicating that Moore’s Law may no longer be a viable framework for future advancements in computing.
Emergence of Alternative Computing Models
The semiconductor industry is exploring alternative computing paradigms, such as quantum computing and photonics. These approaches aim to overcome the limitations of transistor miniaturisation and represent a departure from traditional methods that have defined Moore’s Law.
One of the notable examples of these alternative paradigms is quantum computing.
Quantum computing uses principles of quantum mechanics, such as superposition and entanglement, to perform calculations that surpass the capabilities of classical computers. A key component in quantum computing is the quantum dot, which acts as a qubit—the fundamental unit of quantum information. Research has demonstrated that quantum dots can execute quantum computations with high fidelity, making them suitable for integration into nanoscale electronic circuits. The potential of quantum computers lies in their ability to solve complex problems significantly faster than classical systems, although substantial technical challenges remain before they can be widely adopted.
Cerebras Challenging Moore’s Law
Cerebras is making significant strides in challenging Moore’s Law through its innovative approach to design chips and process AI capabilities.
It developed the wafer-scale engine (WSE), a chip architecture that integrates a massive number of cores—up to 9,00,000—on a single silicon wafer. This design eliminates the need for multiple chips and external wiring, significantly reducing latency and increasing data throughput. The latest version, WSE-3, features 4 trillion transistors and is capable of handling AI models with up to 24 trillion parameters, outperforming traditional GPUs like Nvidia’s H100 by a factor of ten in certain applications.
Cerebras claims that its systems can deliver speeds that are an order of magnitude faster than GPU-based solutions. For instance, it has achieved speeds exceeding 1,800 tokens per second for Meta’s LLaMA 3.1 model, which positions it as a leader in AI inference tasks.
The post 2024 Marks the End of Moore’s Law appeared first on Analytics India Magazine.
