Meta’s Muse AI agent is acting unpredictably, spying on private messages and is having issues keeping straight its conversations with users, reports Ars Technica. Earlier this fall, Reuters exposed internal chats showing that Meta staff had worried about the incidents.
While testing the AI agent, Meta staff discovered that Muse was apparently accessing encrypted private messages that the bot was not expected to review, seemed to be mixing up conversations between users, and even potentially sharing sensitive information with users.
Some testers also experienced incidents where Muse responded to questions that staffers had never asked.
“Hmmm,” one Meta staffer wrote in an instance where Muse sent the definition of “money laundering” unbidden. “Got a random ping from Muse about something I definitely did not ask for… .”
Muse also told a staffer that it saw an out-of-context offer from the agent to message a seller about a coffee table that it had “pulled in some context about a coffee table pickup that doesn’t seem to be yours.”
That employee made sure to flag the odd reply, writing, “this was ‘not my conversation’” and “expressing concern about a ‘possible cross-account context leak.’”
Most concerningly, testers saw random outputs the appeared to share users’ private data. One Meta employee was conducting a conversation with Muse about fishing, when they received a random response saying, “BTW, two bank updates, your Capital One balance is $2722.58, your Charles Schwab balance is $5899.57.” The tester did not have an account with either firm.
Another staffer reported losing money while testing Muse, says the Reuters report. The staffer reported that Muse “accidentally rolled me into a premium insurance plan” for their dog that charged them $888 per month instead of their prior rate of $200. “Couldn’t recover,” the staffer said.
In some cases, instead of bank details, Muse hallucinated bitcoin balances, one team member reported.
Meta spokeshole Daniel Roberts, told Reuters that these were “isolated incidents,” and internally employees were told that “the agent appeared to be hallucinating messages in response to built-in prompts pushing it to engage with users in between sessions.”
Another Meta mouthpiece told Ars Technica that “employee testing is core to the product development process and the entire point is to get feedback, so we can implement safety and privacy protections and improve products. Importantly, employees have access to different versions of Muse than the public, so they’re often reviewing features, functionality, and behavior not available elsewhere.”
Roberts reiterated to Reuters that “many of the messages flagged by employees as potential account mix-ups were hallucinations” that Meta had already addressed.
Reuters emphasized that the outlet was unable to independently “establish whether Muse was giving users other people’s bank account balances and other sensitive data, or making up the information.”
This isn’t the first report we’ve seen about Muse possibly violating users’ privacy. In June, Inc. Magazine reporter, Jason Aten, was shocked to discover that Muse was digging through private messages on his Mac without permission, the reporter said that he had “explicitly” chose to deny Muse that access. When he called out Muse on its actions, the agent tried to glasslight him by telling him that it was simply parsing text in push notifications, but Aten found that “wasn’t true.” Instead, the agent was syncing messages from a local database, which Aten said was activated without his knowledge.
Meta mouthpiece David Singleton responded to Aten on social media, insisting that Aten must have opted into granting “macOS system-level Full Disk Access” with the “Messages connector” enabled, as that’s the only reason Muse would read his private messages.
That response did not satisfy Aten, as he responded with, “Email me, David. Full Disk Access off. Muse synced 187k lines from my messages chat [database]. Something’s not great. Also, that Full Disk Access doesn’t say anything about your message database, and I specifically gave Muse a goal that had nothing to do with messages, so it had no business doing that even if I had given it permission.”
Other users reported Muse digging where it shouldn’t be around that same time period.
Singleton attempted to assure users that Muse wasn’t acting purposefully, saying, “We investigated with urgency and determined that this was caused by a bug introduced in the product that caused the model to hallucinate and was NOT a security or privacy issue. The bug has now been fixed.”
Reuters, in cooperation with Signal and the AI Now Institute, attempted to replicate Meta staffer’s and others’ Muse experiences.
On Android devices, it was discovered that Muse could read Signal messages by bypassing Meta’s “Connector” tool and instead relying on permissions the user granted during initial setup.
Meanwhile, on the iPhone, requests were blocked. However, on Apple’s macOS, Muse used “settings Apple offers to help people with disabilities operate their computers” to retrieve messages, including screenshot tools.
That testing “indicated the issue described by the Meta employees was ongoing and could be affecting others,” Reuters reported.
Apple has showed concern about Muses manipulation of the Mac’s accessibility tools for people with disabilities. In October, Apple announced changes were coming in response to users complaining about Muse accessing private messages without their knowledge or explicit consent. Apple had concluded that “some developers were using the Full Disk Access feature in ways that could put users at riskm” and they promised to “introduce additional controls to ensure that users who genuinely wish to grant an app this extraordinary level of access can only do so with very explicit user action.”
“We are committed to ensuring users understand exactly what they are allowing, so they can make informed choices about their data and privacy,” Apple said.