Intel Achieves Full NPU Support in MLPerf Client v0.6 Benchmark

Intel Core Ultra Series 2 processors, Image/Intel

Intel recently announced it is the first and only company to reach full neural processing unit (NPU) support in the newly released MLPerf Client v0.6 benchmark. As a result, Intel is now considered a leader in full NPU acceleration and industry-leading GPU performance for AI workloads on client PC platforms.

This is the industry’s first standardized determination of large language model (LLM) performance on client NPUs:

Intel’s measurements of MLPerf Client v0.6 show Intel Core Ultra Series 2 processors can produce output on both the graphics processing unit (GPU) and the NPU much faster than a typical human can read.

Full NPU Support in MLPerf Client v0.6 Benchmark, Image/Intel

Intel’s Core Ultra Series 2 processors achieved the fastest NPU response time. It took only 1.09 seconds to generate the first word (first token latency), instantly answering after receiving a prompt. The highest NPU throughput at 18.55 tokens per second, referring to how quickly the system can generate each additional piece of text, was also achieved. Intel attained GPU leadership in time to first token, faster than the competition. This achievement melded its NPU and GPU end-to-end AI acceleration advantage.

Intel Core Ultra Series 2 processors are accelerating AI driven PCs with outstanding AI compute performance. Llama 2 7B model is the AI model used for MLPerf Client v0.6 which uses four content generation and summarization use cases.

Testing Configuration

About NPU Benchmarking on MLPerf:
Developed collaboratively by MLCommons consortium members — including Intel, AMD, Microsoft, Nvidia and Qualcomm — MLPerf Client v0.6 extends beyond previous GPU-centric tests to now include dedicated NPU benchmarking.

Driven by close collaboration between Intel’s NPU hardware and OpenVINO software teams, Intel Core Ultra processors remain the only NPU to achieve complete NPU compliance in the final benchmark.