Is “NPU” the Reason for Apple’s AI Era?
Apple officially announced its entry into the “Apple Intelligence” era this year, where AI assistants and machine learning will lead the user experience of future iPhones, Macs, and iPads. Unfortunately, not all Apple users are “invited” to this party. Only the iPhone 15 Pro and 15 Pro Max models are eligible, and even the recently released iPhone 15 cannot use it. In less than a year, the phone that was launched in September last year has been deemed outdated. In comparison, Mac users only need an Apple Silicon computer released after 2020 to access Apple Intelligence. Among the entire iPhone product line, the requirements for the iPhone are relatively strict.
What limits the Apple Intelligence functionality of the iPhone? Some may speculate that it is the Neural Processing Unit (NPU) that hinders the progress of AI on the iPhone. Tech giants like Microsoft, Intel, and Qualcomm emphasize that having an NPU is crucial for the generation of AI on end devices.
What is an NPU? In the past, mobile phones and laptops primarily relied on the Central Processing Unit (CPU) and Graphics Processing Unit (GPU) for computation. The CPU excels at handling complex tasks, while the GPU specializes in graphics processing. However, AI technology also requires the capabilities of an NPU. An NPU has numerous small cores that excel at simultaneously processing a large number of repetitive tasks while consuming lower power. It has become an essential specification for the new generation of AI PCs and smartphones.
During his keynote speech at the 2024 Taipei International Computer Show, Cristiano Amon, the CEO of Qualcomm, emphasized the importance of NPU in current AI computing.
But is this the reason why iPhones below the iPhone 15 Pro cannot use Apple Intelligence? Andrew Williams, a journalist from renowned tech media “WIRED,” points out that the iPhone has actually been using the NPU since 2017. The first-generation Apple ANE (Apple Neural Engine) was introduced that year and was equipped in the iPhone 8 and iPhone X. It was already an NPU, indicating that Apple had started incorporating AI functionality into iOS before AI became a popular term.
The latest iPhone 15 series also has an NPU. The A16 chip in the iPhone 15 series has an NPU performance of 17 TOPS (trillion operations per second). Analyst Ming-Chi Kuo from TF International Securities pointed out that although Microsoft defines the basic specifications for AI PCs to require 40 TOPS, for Apple, 11 TOPS is sufficient for running AI on iPhones, and the remaining computing power can be supplemented by cloud computing.
“Perhaps Apple Intelligence is not so different from previous AI, but it just has a few additional features,” says Andrew.
So, the real key is not the NPU. The answer may be quite simple: it is because the RAM memory is not large enough. Ming-Chi Kuo pointed out that in the current iPhone 15 series, only the iPhone 15 Pro and iPhone 15 Pro Max models are equipped with 8GB of memory, while the iPhone 15 only has 6GB of memory, which is insufficient to support the computational needs of Apple Intelligence.
Why is RAM memory important for AI smartphones? When AI models run offline locally instead of being connected to the cloud, they need to be temporarily stored in RAM or vRAM (virtual memory), which requires a significant amount of capacity. For example, NVIDIA H200, which is used by ChatGPT, has 141GB of vRAM for a single card, and it requires hundreds of these cards to run the service.
Even the highest-spec iPhone 15 Pro Max, with 8GB of RAM, can only accommodate small and relatively simple AI models. According to Apple’s official website, the offline running capabilities include photo review, photo scene recognition, Siri suggestions, voice recognition, and translation functions.
“If we infer a little, we will find that these offline running functions do not include generative AI,” says Andrew. These functions that can run locally seem to be already available on other brand smartphones, but the exciting generative AI applications, such as image generation in chat rooms or document generation in emails, still rely on the cloud.
Ming-Chi Kuo pointed out that the currently launched Apple Intelligence functions require at least a 3B large language model (LLM). In the future, when Apple introduces more AI applications, it may need to upgrade to a 7B model, which will require even larger memory to execute. Perhaps in the future, the RAM size will be used to differentiate between Apple’s high-end and low-end models.
Therefore, there is considerable interest in the configuration of the iPhone 16, whether it will have access to the complete Apple Intelligence functionality or be limited to the Pro series. The answer will be revealed in September this year.
References:
WIRED, Ming-Chi Kuo on X
This article is a collaboration and is republished with permission from Digitimes.
