No ****! And 5 other Apple Watch Live Rewind testing takeaways
Macworld ·
Macworld Live Rewind, one of the more compelling features demonstrated at the launch of the Apple Watch Series 12 back in September, has finally become available with the launch (in beta form) of watchOS 27.3 and iOS 27.3. I’ve spent several days testing Live Rewind, and can report that it’s a thoroughly impressive proof of concept. But there are flaws and limitations to be aware of, even if several of the latter are the result of good intentions. Here are six takeaways from my hands-on testing with Live Rewind on the Apple Watch Series 12. Live Rewind is surprisingly accurate I was skeptical, but Live Rewind does a surprisingly decent job of understanding and transcribing what it hears. I started by playing YouTube videos of restaurant conversations for people learning English as a foreign language: slow, clear, with largely simple vocabulary. I sat close to the iPad playing the video. An easy task to begin with, and Live Rewind coped easily. The transcripts were virtually perfect, not to mention beautifully formatted with line breaks to denote separate short speeches. I stepped up the difficulty by looking up some menus online (I selected this fancy restaurant ) and acting the part of the waiter myself. I selected esoteric dishes I hadn’t heard of, spoke quickly and was careless with my enunciation, chucked in some ums and ers, varied volume and occasionally mumbled a little. And the feature produced a mostly accurate transcript that would have been extremely useful to a diner who had missed an option on the list. This time the formatting was more of a challenge–Live Rewind couldn’t keep track of separate sentences, so the whole thing was jumbled together with a sometimes misleading number of commas–and occasionally I had to decipher a soundalike guess when the feature didn’t recognise a term. (“IV” instead of Ivy, “7 and Y” instead of the British rivers Severn and Wye, for example.) But the information was mostly there. The most difficult task was when I used Live Rewind (after obtaining consent!) to recall snatches of random conversations with my family. This was more mixed. It caught the gist a lot of the time. But there were more soundalike guesses than before, and some sentences were missed entirely. This mostly related to distance, I think, and I’ll come back to that later. So Live Recall is by no means perfect. But considering the difficulty of the task–handling variations in accent and diction, mumbling and muffled speech, irregular grammar, slang, in-jokes, proper nouns, brand names, technical or foreign terms–it does a very solid job. My verdict is that the accuracy is high enough to be useful as backup if you missed something, but not yet accurate enough to depend on as an alternative to paying attention. That should be “tenderstem broccoli,” obviously. David Price / Foundry Live Rewind has a short memory Like the proverbial goldfish, Live Rewind has a memory of 15 seconds. Which is shorter than you might think. On one test I read out a list of just four main courses, but made sure to include the additional verbiage for each one, as I suspect many waiters in high-end establishments would be encouraged to do. Nothing extreme, just an extra sentence detailing ingredients and/or preparation. The result of this slight bulking out was that Live Rewind, despite being triggered the very moment I finished my list, missed a big chunk of it. The first main and half the second were lost to history. Bear in mind, too, that while lists of specials in restaurants may frequently fit in the 15-second window (assuming the waiter refrains from prolixity), other scenarios may not work so well. Will the list of cocktails at a bar fit in 15 seconds, or the list of films at a cinema, or the list of changes your boss told you to make to a document? I have my doubts, although Siri Recap, if you use it, does not have a time limit and may be able to step in. The short memory, it’s worth noting, applies whether or not Live Rewind is currently hearing anything. If you trigger the feature, it won’t rewind to the last 15 seconds of audio it heard; it will show you any speech it has heard in the past 15 seconds, ending when you double-press the Digital Crown. If you’ve been sitting in silence for at least 15 seconds, you’ll see the message “No voice detected.” And even if you do manage to capture the transcript of some speech, Live Rewind will by default forget it the moment you tap the X button to close it. You need to explicitly save it to the Siri app if you want to keep the record. (You’ll save only the transcript, needless to say, not the audio from which it was created.) This short memory is deliberate, by the way, as we’ll discover. Live Rewind has a short range My first impressions of the feature were not good. When I tried to use it on a conversation with my daughter, who was sitting perhaps 6 or 8 meters away and speaking unusually clearly, it produced a transcript of all my words and literally none of hers. But after more testing it became apparent that this was an issue of range, not accuracy. Live Rewind is good at picking up speech that’s close to you, and far less effective when someone is speaking even from the other side of the room. I haven’t been able to find any evidence that this is deliberate (I know, for example, that the closely related feature Siri Recap can and will listen to people further away, since Apple gives the example of a lecturer “in an otherwise quiet room” in its notes ) but suspect that it may relate to privacy. If the waiter leans in and tells you about today’s delicious cod cheeks, one may assume they consent to being heard. If the speaker is three tables away, things become murkier. Live Rewind doesn’t approve of swearing I don’t want to shock you, but while testing this feature I happened to hear a YouTuber use the word “shit.” Live Rewind transcribed this as ****. (Siri Recap did the same, incidentally.) The closest I can find to an explanation is in the support document, where Apple says Siri Recap is “designed to omit potentially harmful content, such as language that promotes self-harm or hateful speech.” But that’s a different issue, and “shit” does not, as far as I can see, constitute hateful speech. So it’s a bit of a mystery. But very much the sort of family-values prudishness we’ve come to expect from Apple. How very shocking. David Price / Foundry The range of privacy measures is admirable… For better and worse, the issue of privacy is absolutely central to Live Rewind. Apple is conscious that most of us are creeped out by the idea of a wrist-mounted surveillance bot listening in on conversations. It’s conscious that previous surveillance-adjacent tech (Google Glass, for example) has encountered significant public pushback, and wants to avoid bad PR. So everything has been done with privacy in mind. To trigger Live Rewind, you have to double-press the Digital Crown, a distinctive gesture intended to be hard for creeps to disguise. A similar intent lies behind the on-screen animation and non-mutable sound effect, both of which alert anyone close enough to be recorded by the feature that you are summoning a transcript of what they just said. It’s also worth mentioning that the feature works on your device. (The processing legwork happens on the companion iPhone, not the Apple Watch, by the way, so make sure it’s nearby.) It doesn’t happen in the cloud, and Apple insists it does not have access to the data. I won’t pretend I’m completely on board with listening tech in general. I fear that such technologies are becoming more and more widespread and normalised. But with Live Rewind, Apple has done everything it can to take privacy concerns into consideration. …and sometimes frustrating I applaud Apple for its efforts to safeguard privacy. But like the AirTag, which quite rightly took steps to combat stalking and in the process made itself a far less effective anti-theft device , Live Rewind finds itself in a position where good intentions lead to many of its most frustrating limitations. The short memory is there to limit the feature’s surveillance capabilities. Ditto, I suspect, the short physical range. It also doesn’t attempt, by design and for explicitly privacy-related reasons, to identify or differentiate between multiple speakers (not even “speaker A” and “speaker B”), despite this being an obvious way to make transcripts easier to parse. When viewing the transcript on your Apple Watch, you’ll find that the screenshot function is disabled, presumably to make it slightly less convenient for bad actors to eavesdrop and then share the results. But this also makes life less convenient if you want to share them for innocent reasons, such as sending the list of specials to a latecomer on their way to the restaurant, or publishing an article about how well the feature works. Live Rewind would be much more useful if these limitations were cast aside, but it would also be more controversial, and Apple doesn’t like that. You’ll need a new Apple Watch Series 12 to use Live Rewind. Mahmoud Itani / Foundry Conclusions I’m way more impressed by Live Rewind than I expected to be. The accuracy is very solid. Yes, it makes some mistakes when tested under the most challenging conditions, but a lot of those mistakes will actually be phonetic best guesses that can be deciphered with little effort. And it usually gets the important details right. It’s not quite sharp enough that you can ignore the waiter listing off specials or the senior colleague asking you a question and assume everything will be transcribed perfectly. But it’s a handy backup for the parts you missed while mostly paying attention. The main issues I have with Live Rewind, if they can be called issues, are the limitations deliberately put in place for privacy reasons. It’s got very short range, for instance, and the 15-second memory window is on the low side in terms of usefulness. But privacy matters, and I understand why Apple made these decisions. Speaking only for myself, I still find the concept a tiny bit creepy. But Apple has done its best to minimise that factor, and this is overall a very decent effort.