The new Siri has finally begun to understand users' "personal context"

At 1:00 a.m. Beijing Time on September 10, it is still 10:00 a.m. in Cupertino, California, and Apple's annual autumn launch event kicks off at Apple Park.

Cook, familiar to all audiences, only made a brief appearance at the opening this time, then handed over the stage entirely to John Ternus. This is the first autumn launch event after Ternus took over as Apple's CEO, and it is also his official "debut" in the real sense.

Both radical moves and steady, step-by-step strategies appeared at this debut. On the hardware front, Apple launched its first foldable screen iPhone Duo that users have expected for many years, which naturally grabbed the most attention. However, if you move your sight away from the foldable display, Apple's changes in the AI sector are also worth noticing.

The new Siri is confirmed to enter the Beta stage this month. For the first time, Apple Watch starts to intelligently perceive sounds around users through audio. Ternus also defined a new identity for iPhone - the world's best "intelligent personal hub" at present.

Ternus endowed the concept of intelligent personal hub to AI devices, stating that if you imagine its ideal state, the iPhone still stands out above all other products.

The "personal intelligence system" that Apple has been talking about for the past two years finally got a clearer outline at this launch event. The iPhone is still at the core position, Siri is the most direct AI entry for users, and devices such as Apple Watch have begun to take on new perception tasks.

Compared with the update of a single function, this cross-device collaboration may be the more noteworthy progress of Apple in AI this time.

01

The big day for the new Siri is finally confirmed

The question of when the new Siri will be truly available to ordinary users finally has a clear date.

At the just-concluded Apple autumn launch event, Siri AI was brought to the front stage again. Apple did not spend too much time re-explaining this system, but directly integrated it into the daily usage scenarios of iPhone 18 Pro, focusing on demonstrating its understanding of screen content and personal information.

This is also one of the most obvious differences between the old and new Siri.

For example, if a friend casually recommended a restaurant in a chat a few months ago, and you only vaguely remember that "someone mentioned it to me", Siri can find it from the historical messages. The confirmation email for hotel booking is buried deep in the mailbox, and you can also directly ask Siri for the confirmation number. When you want to find the photos you took with several friends during your last trip, you no longer need to flip through the album one by one.

For the first time, Siri truly has the ability to understand the user's "personal context".

It can also understand what is happening "in front of you". For example, if a friend suddenly invites you to a weekend dinner in the chat window, Siri can help you figure out what dish to bring based on the conversation on the screen, and then add the found recipe to the memo. After turning on the camera, the new Siri mode can directly understand the objects in front of the lens: take a picture of a plate of food, and it can provide nutritional information. When you get a bill that needs to be split equally, it can also continue to perform operations based on the picture.

As for those capabilities that are more similar to ChatGPT, the new Siri has also been supplemented. It can directly access the Internet to answer open questions, such as when the next solar eclipse will occur, when a certain singer will come to perform locally, and allow users to follow up with more questions. Conventional generative AI tasks such as writing emails and modifying texts have also been directly integrated into Siri. Apple has also developed an independent Siri App for it, and all past chat history with Siri can be synchronized across different Apple devices, so you can go back and continue the conversation later.

However, it needs to be clarified that from the perspective of specific functions, this launch event did not add any new features to Siri AI.

Personal context, screen understanding, online answering, cross-App operation, visual understanding in the camera, including the independent Siri App, were all fully announced by Apple at WWDC on June 8. At that time, Apple had already called Siri AI "a brand new set of Siri" and opened it for developer testing.

The key new information added at this autumn launch event is that Apple has finally set the launch timeline for ordinary users.

On September 14, Siri AI will officially enter the Beta phase along with iOS 27.

The first batch of devices are required to support Apple Intelligence, and the system and Siri are set to English. French, Japanese, Korean, Portuguese and Spanish support will follow in October. Apple Watch will also get the Siri AI Beta via watchOS 27 on September 14.

But Chinese users still have to wait for the time being.

The relevant footage of the launch event clearly states that "Siri AI will not be available in the European Union region and China".

02

Is Apple Watch "eavesdropping" in real time?

If you only look at Siri AI itself, there are not many surprises brought by this launch event. As mentioned earlier, the new Siri was unveiled back in June, and this time it is mainly to finalize the launch timeline.

On the Apple Watch side, Apple quietly added a notable new feature: Audio Intelligence.

According to Apple's own definition, it is a set of Apple Intelligence functions that use the microphone of Apple Watch, Apple chips, as well as on-device and cloud AI models to understand surrounding sounds.

At present, Audio Intelligence includes a total of four capabilities:

. Sound Recognition

. Automatic music recognition via Shazam

. Live Rewind

. Siri Recap

Sound Recognition is responsible for identifying important environmental sounds. When the doorbell rings, the alarm goes off, or a baby cries nearby, the watch can recognize it and send a reminder to the user, even if the iPhone is not around at that time. Apple especially emphasizes that this function is designed for deaf or hearing-impaired users.

Shazam makes "identify songs by listening" much more seamless.

Shazam has been available on Apple Watch for a long time, but in the past, if you wanted to use it to identify songs, you needed to open Shazam specially, just like the song identification function of any music App. Now the new Apple Watch will automatically recognize the music playing around you. You don't need to click "identify song", nor ask Siri. After recognizing it, it will directly display the song title and singer in the "Music Recognition" widget in the Smart Stack. Apple specifically emphasized that "you don't need to tap at all".

What's more interesting is the Live Rewind function.

Suppose you are chatting in a noisy restaurant, the other person just said a sentence that you didn't hear clearly, and you could only ask them to repeat it. Now double-click the Digital Crown of the watch, it can convert the words spoken in the past 15 seconds into text and display it. You can also continue to ask Siri what this sentence means, or save this text to the new Siri App to check later.

It can be seen that this rewind function is linked with Siri.

The last Siri Recap function also follows this logic.

For example, during a meeting, discussing things with colleagues, or when the coach gives a series of consecutive actions during fitness, as long as the user turns on Siri Recap in advance, Apple Intelligence can automatically generate a title and several key points after the conversation ends, and save them to the Siri App.

In other words, the watch will help you "take notes" at any time.

Obviously, the watch has begun to take on a previously unobvious role, that is, to perceive the real environment around the user for the entire Apple Intelligence system.

It should be noted that "the watch starts to listen to the surrounding world" does not mean that "all Apple devices are eavesdropping in real time from now on".

Apple currently strictly limits Audio Intelligence to Apple Watch Series 12 and Apple Watch Ultra 4.

Sound Recognition and automatic Shazam need to be paired with iPhone 11 or later, or iPhone SE 2nd generation or later, running iOS 27. While Live Rewind and Siri Recap have higher requirements, they require an iPhone 16 or later that supports Apple Intelligence, excluding iPhone 16e, with Apple Intelligence and Siri AI turned on at the same time. Users who only have an iPhone and do not own these two new watches cannot use the rewind and recap functions for the time being.

Apple has also added many restrictions to this set of capabilities. For example, Live Rewind will only recall the content of the previous 15 seconds after the user actively triggers it. Siri Recap also needs to be manually turned on, and the usage scope can be set according to time and location.

The system will not save the original recording. When it comes to the recap function, the audio will be encrypted and transmitted to the iPhone for processing, and Apple's private cloud computing will be called if necessary, and deleted after processing. Apple is well aware that when AI reaches the level of perceiving the real environment, privacy issues will become much more sensitive.

The new role that Apple Watch takes on this time is particularly noteworthy, which actually represents the gradual formation of the cross-device, ecosystem-deep "personal intelligence system" in Apple's vision.

03

Ternus says iPhone is

the best "intelligent personal hub"

For friends who have been paying attention to Apple's AI trends in the past two years, you may have been confused by a number of terms now, such as Apple Intelligence, Siri AI, Audio Intelligence, the "personal intelligence system" that Apple has repeatedly mentioned, and the new "intelligent personal hub" that appeared at this launch event.

In fact, after clarifying the relationship between these concepts, Apple's current AI roadmap is clear at a glance.

The earliest trace of Apple mentioning the concept of "personal intelligence system" can be dated back to 2024. When Apple first released Apple Intelligence that year, it defined it as a set of "personal intelligence system".

Two years later, Ternus added a very important concept "intelligent personal hub" at this launch event, and the answer he gave is iPhone.

Therefore, Apple Intelligence is the "personal intelligence system" covering the entire Apple ecosystem, and the iPhone is clearly placed at the core of this system by Apple.

Ternus first described his ideal AI device - it needs to understand the user's personal information and stay around at any time. It not only has a powerful enough chip to run the model locally, but also can connect to the cloud via the Internet. It also needs a screen, camera and microphone to see and hear the surrounding world, and can cooperate with other devices, Apps and services.

After describing according to this set of standards, he finally landed the answer on a very familiar product: no device is more suitable to become the user's "intelligent personal hub" than iPhone.

Apple later directly adopted this positioning in the official introduction of iPhone 18 Pro.

Siri AI has become one of the most direct entrances for users to call these capabilities. It can understand personal context, read screen content, and continue to complete operations according to user requirements.

At this time, looking back at the Audio Intelligence mentioned earlier, the new role of Apple Watch is easier to understand.

In the past, Apple Watch could already receive part of the Apple Intelligence capabilities on the iPhone, and this time it added the perception of surrounding sounds. Apple's own technical description states that Audio Intelligence uses the microphone and S11 chip of the watch, the Apple Intelligence model on the iPhone, as well as Apple's private cloud computing. In other words, a function can be completed across the watch, mobile phone and cloud.

This is a specific example of Apple's "personal intelligence system" starting to take shape.

If Apple continues to move forward along this path, devices such as headphones and tablets may also take on different tasks in the future, evolve different branches of "intelligence", and become the extended "senses" of the iPhone AI hub.

The more hardware Apple owns, the richer the information and interaction entrances this personal intelligence system can obtain.