Mark Zuckerberg burns the midnight oil on the American Jarvis
On September 8 local time in the United States, Meta launched Muse, which is billed as "the world's first personal AI agent". It is open to users aged 18 and above in the United States, accessible via iOS, Android, muse.ai and WhatsApp, and will later be integrated into Meta's AI glasses.
The biggest difference between Muse and ordinary chatbots is that it can directly handle tasks on behalf of users. When a user tells it a goal, it can open browsers, search web pages, fill out forms, send emails, book itineraries, and purchase goods on its own, and continue processing tasks even after the user closes the app.
Meta equips each Muse with an independent cloud virtual machine that comes with its own browser, file system and terminal. It can access services authorized by users such as mailboxes, calendars and fitness apps, and remember things users care about based on long-term conversations.
Going a step further, Muse will actively remind users, break down long-term goals, continue to advance background tasks, and can also generate web pages, PDFs, documents and interactive tools.
In other words, Meta wants ordinary people to directly use personal AI agents, and make the usage as simple as chatting as much as possible.
01
AI Evolving from Assistant to Executor
The most noteworthy part of Muse is that it has begun to handle a whole set of end-to-end tasks.
Mona Sarantakos, head of product at Meta, mentioned that during product testing, she asked her own Muse to take charge of preparations before her kid started school. Muse kept checking school emails and the school district website, added important dates to the family calendar, sorted out school supplies that needed to be purchased, helped her find the sweatshirt her kid wanted and spotted a discount, and also booked the dinner for the first day of school.
Among all the experiences, one detail made her trust the capability of the Agent.
The school email contained information about the upcoming sports tryouts for her kid, and there were only 12 hours left before the registration deadline. Muse actively noticed this matter and sent her a reminder before she boarded the plane. She then contacted her husband and completed the registration 4 hours before the deadline.
This kind of task is difficult to complete in a single chat. Muse needs to continuously check information, identify content worth noticing, and then decide when to remind the user. Therefore, Meta also designed background working capabilities for Muse. After the user assigns a task, even if the app is closed, it can still continue processing, and will only notify the user when new important results are found or user approval is needed for the next step of operation.
Users can also assign multiple tasks at the same time, without waiting for Muse to finish the previous one before proceeding.
To make these background tasks visible, Meta added an activity record feature. By clicking on the Muse avatar, users can see what it is doing, what it has done, and what permissions have been approved before.
For tasks that last for a long time, Muse also has a separate Goals page. It can break a relatively large goal into multiple steps and continuously track the progress.
These designs show that what Meta wants to solve is "how to let AI keep doing things for users continuously".
Roberto Nickson, an early adopter in the technology and creative fields, believes after getting early access to Muse that personal AI agents have long lacked real product-market fit, and Muse's advantage lies in the fact that Meta itself has a large amount of social relationship data and application data.
He is particularly optimistic about Muse's performance in online shopping and local recommendations. However, he also said that it is not yet certain whether Muse will immediately become the main agent he uses every day.
02
Giving AI Its Own Computer
Muse can access mailboxes, calendars, fitness apps, and also connect to Meta's own services such as Instagram and Facebook.
On the web, it can search for information, browse websites, fill out forms, complete ticket booking and shopping.
For example, in Meta's demo video, Muse found that the price of a flight ticket dropped by 40 US dollars, and then reminded the user whether to rebook it. It will also actively remind the user of the upcoming golf tee time, and tell the user which hole on the course is the most difficult.
In the shopping scenario, Muse can search for products on its own, compare options, and ask for user confirmation before final payment.
If the user has saved a recipe on Instagram, Muse can also organize it into a shopping list, arrange the menu for the party, remember the dietary restrictions of friends, and then assist in sending invitations.
More complex tasks rely on Muse's own file system and terminal programs.
Based on these, it can write code on its own and create the tools needed to complete tasks. It can also generate PDFs, web pages and other "Artifacts". For example, if users want to track their expenses, they can ask Muse to make an expense tracking tool that updates continuously; when preparing for study, they can ask it to make an interactive study guide.
When Meta's product team designed Muse, it also added the Ideas feature. Early tests found that Muse can do so many things that some users do not even know where to start.
Therefore, the system will actively propose some actionable things based on the user's goals, long-term conversations and discovered habits.
The interaction method of Muse has also been adjusted. It adopts a long-term continuous main conversation, where users can interrupt at any time and assign multiple tasks in a row. For projects that require independent context, Meta added side chats.
AI analyst @kimmonismus believes that Muse is positioned closer to providing ordinary users with a low-threshold entry for personal AI agents. It has its own computer and browser, and can also be used via WhatsApp.
For users who are already active in the Meta ecosystem, this access method may be easier to get started with than AI agents that require learning complex operations.
03
Handing Over Permissions to AI?
When AI directly operates mailboxes, shopping and payment, the biggest problem also arises: if an agent can open web pages, fill out forms, send emails and even shop on your behalf, the impact caused by its mistakes will be greater than that of ordinary chatbots.
To this end, Meta designed an independent security architecture for Muse. Each user has their own Muse Secure VM, which is an independent cloud virtual machine. The data of Muse and users are stored in it, and agents of other users cannot access it.
There is also an independent Sentinel agent in this virtual machine. When Muse wants to access the Internet, it must be approved by Sentinel. In situations that require user decision, the system will ask for further confirmation.
Passwords and payment information will not be directly exposed to Muse either.
After users connect to the service, the credentials will be stored in secure storage. Muse can call the credentials but cannot see the actual passwords. Meta also stated that passwords entered by users themselves in the browser will not be seen by Muse. When it comes to sensitive operations such as sending emails and shopping, Muse will ask for user confirmation.
Meta also provides users with permission control. For example, the mailbox can only allow Muse to read, or allow it to draft and reply to emails on behalf of the user.
Muse also provides complete operation records, so that users can see what it has done and what it is going to do.
Meta stated that users can choose not to let Muse's interaction data be used to train Meta's AI models. Conversations in Muse and virtual machine data will not be provided to Meta's advertising system either.
Later this year, Meta also plans to launch Muse Confidential VM. At that time, the entire virtual machine, including user data and conversations with Muse, will be encrypted with a key held only by the user, and even Meta cannot access it.
In addition to security, Meta has also launched a public vulnerability bounty program. Previously, the company had looked for problems through real scenario testing, agent red team testing and private vulnerability bounty programs internally.
04
Betting on a Billion-User Entry Point
Muse is not intended to be a stronger AI assistant, but to bring personal AI agents, the "Jarvis" in the Iron Man movies, to users of Meta's services.
Wang Tao, Chief AI Officer of Meta, said that making the product simple and easy to use is one of Muse's design principles. It does not require users to have technical experience, but hopes that ordinary people can directly use AI agents through a chat-like way.
Pricing also serves this positioning.
Most of Muse's features are provided for free, and users with higher computing needs can choose subscription tiers of $20 or $100 per month. Meta expects that most users will stay on the free version.
This is also where Muse differs from many current AI agent products. The latter's usage scenarios are still more focused on programming and business tasks, and a mature product model has not yet been formed in the consumer market.
Meta hopes that Muse will be oriented to daily life from the very beginning, and package personal AI agents as consumer products that ordinary users can understand and use.
Another advantage Meta has is its huge application entry points. Muse can already connect to Instagram, WhatsApp, as well as services such as Gmail, Google Calendar, Google Drive, Ticketmaster and OpenTable.
For other services with APIs, users can also ask Muse to create custom connectors. For Meta, this means that Muse does not need to cultivate a completely new user scenario on its own, it can directly enter the applications and services that users are already using.
This is highly consistent with the personal AI vision previously proposed by Mark Zuckerberg. He once said that in the future, everyone will have a powerful personal agent that understands their goals and concerns. Muse is Meta's first launch of this vision in the form of a consumer product.
Kyle Reidhead, a technology analyst, believes that consumer AI agents are becoming the next important trend in the AI industry. After Grok, Meta launched Muse, and Apple and Google are also expected to continue to integrate similar capabilities into their phones and systems. Based on this, he judges that the large-scale use of agents will continue to increase computing demand, and after digital agents become popular in the future, the demand for computing power from physical AI such as robots will be even higher.
However, Meta itself admits that it is still in the early stage.
Muse needs to obtain enough application permissions to really function effectively; and the more permissions it has, the higher users' requirements for privacy and security will be. Meta now chooses to make the entry point simple enough, and use virtual machines, permission control, Sentinel and manual confirmation mechanisms to control risks.
If users are willing to assign more daily affairs to Muse, personal AI agents will have the opportunity to evolve from a new feature to a tool that people use every day.
@alanchen, an early adopter on X, believes that the most prominent part of Muse is that it can really work continuously around user goals, while putting security and privacy at the core of the product.
This is also the most noteworthy part of Muse in the next stage. Technically, it is already able to undertake many tasks that previously required users to complete in person; in terms of product, Meta is also working hard to lower the threshold for use. But whether personal AI agents can eventually become high-frequency tools depends on whether users are willing to use them for a long time and gradually assign more real life affairs to them.
For Meta, the value of Muse is therefore not just adding an AI product.
It is more like an entry test for billions of users: if personal AI agents can enter daily applications, Meta will have the opportunity to bring AI from the chat window further into users' daily lives.
This article is from the WeChat official account "Tencent Tech", written by Su Yang, edited by Xu Qingyang, and published by 36Kr with authorization.