The Apple Watch Series 12 and Apple Watch Ultra 4, which were unveiled on Wednesday, offer several new Audio Intelligence features, including Sound Recognition, Music Recognition, Live Rewind, and Siri Recap. When you have these features activated, they are continually listening to what’s going on around the wearer.
Siri Recap is arguably the most impressive of the features, as it allows your new Apple Watch to listen to your conversations throughout the day and quietly takes notes about those interactions,. While that can definitely come in handy, especially for an old codger like me, shouldn’t we be concerned what happens to all of those notes?
Apple knew there would be questions about the new features, so it has posted details about the privacy features that it has in place.
-
Audio is not recorded or stored. Audio Intelligence features don’t create audio recordings. No audio can be accessed by the operating system, apps, you, or even Apple. There is no recording to share, forward, or produce if requested by any party, because no recording exists.
-
Raw audio is never accessible. Audio flows into a protected buffer inside the S11 chip’s Secure Exclave — a hardware-isolated compartment built into the silicon of Apple Watch and iPhone that processes sensor data separately from the rest of the system. Data that is processed and analyzed in the Secure Exclave can’t be accessed by the operating system, apps, you, or even Apple. Once processed, this data is immediately deleted.
-
Siri Recap and Live Rewind help jog your memory, not provide a detailed record. Siri Recap doesn’t produce a transcript. It generates a brief, high-level summary — comparable to notes you might jot down after a conversation. Live Rewind shows a text snippet of what was just said in the last 15 seconds, so you can catch something you missed.
-
Your data is not accessible to Apple. For Siri Recap, Apple Foundation Models running on Private Cloud Compute (PCC) summarize and format the final output. Data in PCC is never stored or accessible by Apple or anyone else. It is used only to fulfill your request, and independent security experts can verify this promise at any time.
-
Your data auto-deletes if you take no action. Siri Recaps auto-delete after 7 days if you don’t save or export them. Live Rewind 15-second text snippets disappear permanently from your Apple Watch approximately 30 seconds after the Apple Watch display dims, unless you save them to the Siri app or tap “Ask Siri”.
-
Your data is end-to-end encrypted syncing with iCloud. If you use iCloud with two-factor authentication and have a device passcode, any Siri Recap summary or Live Rewind text snippet you choose to save syncs across your devices end-to-end encrypted in the Siri app. Only you can access this information, and only on devices where you are signed in to iCloud. Apple does not hold the encryption keys. No one else, not even Apple, can access the end-to-end encrypted data that you saved in the Siri app.
The features are designed to protect not only your privacy, but also the privacy of those nearby.
-
No speaker attribution. By design, Audio Intelligence features don’t identify and attribute speakers. Siri Recap may contain names if mentioned in the conversation, but doesn’t directly attribute speakers (for example, “John said…” or labeling as “Speaker A” versus “Speaker B”). Similarly, Live Rewind may transcribe what someone said word-for-word, but does not attribute quotes to speakers.
-
Audible and visible signals for Live Rewind. When activated, an audible chime plays from the speaker on your Apple Watch, even if your Apple Watch is on silent or you have headphones connected. A full-display animation and microphone indicator appear on the display, and a distinct double-press gesture is required to activate Live Rewind. This is to alert those nearby that the feature has been activated.
-
Omitted content. Siri Recaps are designed to omit potentially harmful content, such as language that promotes self-harm or hateful speech, and sensitive information, such as financial data, government-assigned identifiers, authentication data, and certain personal identifiers.
-
Be mindful of those around you. While audio is not recorded and stored, and no speakers are attributed, consider those around you where conversations might be private or sensitive.
None of your data can ever be accessed by Apple.
-
Audio is protected in hardware. All four features are enabled by the new S11 chip on Apple Watch and its Secure Exclave, the dedicated secure buffer that processes audio privately in a protected environment that no one can access, not even Apple.
-
Shazam shares only an acoustic signature. If you open the Smart Stack on Apple Watch and music has been detected, the Secure Exclave will process the audio in the secure buffer and create an acoustic signature of that song (a compact representation that can’t be reversed into the original audio). That signature — not the raw audio — is then sent to Shazam servers to identify the song.
-
Sound Recognition and Live Rewind are processed on device. All processing, including speech recognition, happens on your Apple Watch and iPhone. If you choose to ask Siri about the content of your Live Rewind text snippet or save it to the Siri app, it will be securely sent to Private Cloud Compute.
-
Siri Recap uses Private Cloud Compute. The Secure Exclave on Apple Watch encrypts the detected audio and transmits it to the Secure Exclave on the paired iPhone. The encrypted audio is then decrypted, transcribed, and condensed inside the iPhone Secure Exclave. To create a summary, Siri Recap sends an encrypted, condensed version of the conversation transcript from your iPhone to Private Cloud Compute. This condensed text is designed to be less than half the length of the original transcript and distills the conversation to its core meaning. Content suggestive of conversation tone, non-essential language, filler words, and redundancies are removed, while preserving the topics discussed. This condensed text is sent to PCC along with some contextual information to generate the summary. The finished, text-based summary is encrypted and sent from Private Cloud Compute back to iPhone and Apple Watch.
-
End-to-end encrypted syncing with iCloud. Siri Recap summaries and saved Live Rewind text snippets live in the Siri app in the Recaps tab, end-to-end encrypted with iCloud syncing.
-
All within Apple’s hardware and software architecture. Audio Intelligence features are designed with a tightly-integrated architecture in which your data is processed only on-device and with Private Cloud Compute.
