Dropping from 54GB to 4GB, Apple is in talks with PrismML. Is model compression technology about to boom?

What? Large language models are finally getting "weight loss shots" too?

This isn't just made up. According to CNBC reports, Apple is in talks with the startup PrismML, a company renowned for its recently launched model compression technology. Apple aims to evaluate the feasibility of running larger-scale AI models directly on iPhones using this technology.

(Image Source: CNBC)

You know, over the years, whenever the AI segment comes up at smartphone launch events, I usually instinctively reach for my water bottle.

It's not that I have any grudge against manufacturers. It's just that everyone is far too familiar with this routine. First, let the AI summarize various elements on the screen, then use image editing tools for personalized color grading or erase passersby from photos. This year, a new feature has been widely added: calling a voice assistant to order you a coffee.

But we can't blame smartphone manufacturers entirely. Current mainstream large models simply can't fit into mobile phones, and the trimmed-down on-device models lack sufficient intelligence. In the end, all manufacturers can promote are cloud-updated features. For example, once Doubao launched an AI podcast function, almost all mainstream manufacturers followed suit within three months.

Here's the question: if a full large model is slimmed down enough to fit on a phone, can on-device AI assistants finally become fully functional?

From 54GB to 4GB: Is model compression technology about to go mainstream?

First, let's explore these two questions together with Leitech (ID: leitech):

Who is PrismML?

According to its official website, PrismML is a startup specializing in model compression. It spun off from a California Institute of Technology research team and is backed by Khosla Ventures, Cerberus, and Google. Its research focus is on drastically reducing model size and operating costs while minimizing performance degradation.

(Image Source: PrismML)

What have they done?

PrismML's approach shares similarities with low-bit model paths like BitNet. It shrinks model size by drastically simplifying how internal AI model information is stored, restricting each weight in the model to binary or ternary representation, which significantly reduces the memory required to store and run the model.

Specifically, a single parameter in traditional large models typically requires 16-bit or even 32-bit for storage.

(Image Source: HuggingFace)

In this scenario, if a 27-billion-parameter model uses FP16 precision, 27 billion × 2 Byte ≈ 54GB — that's roughly the size of Qwen3.6-27B under FP16 precision.

Let alone smartphones, many consumer PCs struggle to run such models fully.

With PrismML's approach, 1-bit parameters are simplified to {-1, +1}. It's like how old photos stored 16 grayscale levels per pixel, but now only black and white are needed. Although there's significant information loss, the size can be compressed to 1/14 of the original, and inference performance can be restored through training.

(Image Source: PrismML)

Building on this technology, they officially launched the Bonsai-27B model on July 15. Fine-tuned from Qwen3.6-27B, it retains full context while shrinking the model from roughly 54GB to under 4GB, allowing it to run natively on iPhones with 12GB of memory.

For context, Google's Gemma 4 E4B for mobile and edge devices is about 3.65GB. PrismML essentially fits a nominally 27-billion-parameter dense model into a similar "footprint."

Leaving aside user experience for now, hardware manufacturers will definitely be thrilled to see this.

(Image Source: PrismML)

So it's no surprise that Apple is interested in this technology.

Apple's current on-device model has roughly 3 billion parameters, using methods like 2-bit quantization and cache sharing. But it can only handle functions like real-time translation, photo album search, and email summarization on the phone, with almost no Agent-related execution capabilities.

Yet the Bonsai-27B model still retains some of Qwen3.6-27B's Agent capabilities.

Of course, there's still performance loss. In PrismML's own tests, the ternary version retains about 95% of the full-precision model's overall performance, while the 1-bit version retains around 90%. For Agent-critical tasks like tool calling, the loss is more noticeable.

Some community testers reported: PrismML's ternary version still suffers from hallucinations and Agent loop issues compared to Q4_K_XL. But its advantage is the extremely small size, basically delivering performance comparable to a 17.9GB model at just 5.9GB.

(Image Source: Reddit)

But regardless, being usable is better than being unusable.

From physically impossible to fit, to reaching an acceptable level of practical experience — if this progress continues, the competition will surely intensify.

AI phones on the verge of explosive growth, on-device AI capabilities urgently need improvement

Interestingly, on July 15, seven on-device generative AI model services for mobile phones — Apple Intelligence, Huawei Xiaoyi, OPPO, Xiaomi, vivo, and others — all completed registration with the cyberspace administration authorities.

Indeed, this list looks quite lively. It's clear that smartphone manufacturers are now seriously prioritizing on-device AI.

(Image Source: Cyberspace Administration of China)

The reason is easy to understand: tasks like notification summarization, call organization, photo album search, and image recognition don't need to queue up in the cloud every time.

Especially for private information like chat records, photos, and files, it's best to handle them locally on the phone. Considering Grok's recent privacy scandal, I fully understand why people don't want their personal data traveling across the internet.

(Image Source: Leitech)

The problem is, from my personal experience, most current smartphone AI features still rely on the cloud. The vast majority of functions stop working once the device is offline.

Why is this the case? What level has on-device AI on smartphones actually reached today?

Coincidentally, I recently tried Gemma 4 E4B in the Google AI Edge Gallery and can share my experience with you all.

(Image Source: Google)

First of all, note that Gemma 4 E4B is already one of the most capable on-device models for smartphones, handling text, images, and audio. Once downloaded, it can chat even when offline.

For example, Ask Image implements multi-modal input that many previous on-device smartphone AIs struggled to deliver well.

After practical testing, Gemma 4 has strong image recognition capabilities. While it still struggles with anime characters, it excels at capturing features in scenes, and can recognize common food, hardware, and flowers.

(Image Source: Leitech)

Then there's Ask Audio, which supports uploading up to 30 seconds of audio for transcription, summarization, and more.

This feature is less impressive. Perhaps because my recording was unclear, the transcribed text had almost no relation to the original audio. Its usability is quite limited right now — it's better to stick to Doubao or Qwen for summarization tasks.

(Image Source: Leitech)

As for text processing...

I fed a roughly 2500-word article to several deployable mobile models and asked them to generate a corresponding summary.

In the end, only Gemma 3n E4B and Gemma 4 E4B completed the task. But the former took nearly two minutes and produced a summary that missed key points, while the latter delivered a more concise summary that captured almost all main points — completely sufficient for quickly reviewing materials.

Even some logic problems that were previously unsolvable can be tackled by Gemma 4 E4B through extended thinking — though the thinking time far exceeds the response time of online large models.

In Leitech's view, Gemma 4 E4B has proven that local smartphone models can indeed deliver results.

But I'm only willing to use it for tasks like summarization, rewriting, and basic image recognition. For slightly more complex tasks — especially those involving long Chinese text, detail judgment, and content creation — the gap with online large models remains obvious, not to mention tasks like Agent invocation.

You know, in terms of compression ratio and functionality, Gemma 4 is already the most powerful on-device AI for smartphones today.

To surpass this performance, Apple can only abandon existing compression methods. By fitting models with larger parameter counts in the same space, it might finally give smartphones a "brain" that's less prone to silly mistakes.

Parameter size isn't everything — efficiency is the key to model application

In the past, when people talked about large models, they always thought more parameters meant more prestige.

100 billion parameters was just the starting line, and even a trillion wasn't too many. Manufacturers would announce numbers at launch events like weighing produce at a market, emphasizing "our radishes are big and strong."

That seems fine, but real-world operation tells a different story.

According to the Chinchilla Scaling Law research by Jordan Hoffmann's team, with the same training compute, a 70B model with fully sufficient training data can comprehensively outperform under-trained 280B and 530B models. Even trillion-parameter MoE models still face significant challenges with video memory usage and memory bandwidth.