ZDNET’s key takeaways
- Onscreen awareness is a Siri AI feature.
- The feature lets you chat with Siri AI.
- Get quick answers to onscreen content.
One of the many cool skills baked into the new and improved Siri AI is onscreen awareness. Here, Siri can “see” what’s on your iPhone’s screen to provide information, answer questions, carry out requests, and chat with you about the content.
Also: How to claim up to $95 today from Apple’s Siri AI settlement
I’ve mostly been using this feature on my iPhone with iOS 27 installed. However, onscreen awareness is also available on an iPad with iPadOS 27 and a Mac with MacOS 27. You can even use it on an Apple Watch with WatchOS 27, but for now, the feature’s use on the watch is limited. Only a few built-in apps support it, such as Messages, Notifications, and Calendar. But more watch apps should start to jump on the bandwagon.
The one downside here is that Siri can only “see” what’s visible. If you open a long email, web page, or file, the AI won’t see anything above or below the content on the screen. Keep that in mind if you need help with something that extends beyond the visible screen.
Get started
To get started on your iPhone, first make sure you’ve upgraded to iOS 27. Just head to Settings, select General, and tap Software Updates. Select the option to upgrade to iOS 27 and let the new version download and install.
You’ll then need to sign up to use the Siri AI beta. After your phone has restarted, go to Settings and tap the option to try the Siri AI beta. You may have to wait a few hours or even a day to be registered for the public beta. Then you can use Siri AI.
Use Siri’s onscreen awareness
To try the new onscreen awareness, navigate to a specific screen on your phone. Activate Siri by saying “Hey Siri” or “Siri.” Alternatively, press the side button. With Siri popping up on the screen, you can ask your question or make a request. Siri can view your current screen and provide the help or information you need.
How I use Siri’s onscreen awareness
Now, here are five ways I like to use Siri AI’s onscreen awareness.
1. Ask Siri for directions to a specific location
I often get emails about plays, conventions, and other events I’ve registered for. Depending on the location, I’ll typically need directions.
With Siri AI, I simply open the email with the address (or often just the venue name) and ask for directions. In response, Siri confirms the address and then takes me to the Maps app with turn-by-turn directions. I can do the same thing with a text message, web page, image, or other screen that displays a location name or address.
Screenshot by Lance Whitney/ZDNET2. Ask Siri for help with a screen on my phone
I sometimes need to tweak the settings on my iPhone, both for personal and professional reasons. Though I’m familiar with most of the settings, there are always some I may not fully understand or know how to set. But here I can enlist Siri.
Also: Eager to try iOS 27’s upgraded Siri? Get ready for usage limits
With the appropriate Settings screen open, I can ask Siri: “Explain this screen and give me advice on how I should set it.” In return, Siri gives me a rundown of the settings on the screen and helps me determine how to set it. I can do this with any screen on my iPhone, though it’s especially useful when I need to figure out what to do with a particular setting.
Screenshot by Lance Whitney/ZDNET3. Ask Siri to share a photo or other item
When my wife and I are out and about, and I’m snapping photos, she’ll often ask me to send her a photo that I just took. In the past, I had to open the Photos app or Messages app, attach the photo, and then text it to her. But now I can just ask Siri to do it.
With the photo on the screen, I say: “Siri, text this photo to my wife.” Siri confirms the action, and then shares the photo with her via a text message. Naturally, I can do this with any file, screen, or other item on my phone.
Screenshot by Lance Whitney/ZDNET4. Add items to a list
I sometimes browse the web for different recipes. When I find one I like, I’ll buy the necessary ingredients next time I’m at the grocery store. For that, I’ll add them to a note or list.
As one example, I was looking for a recipe for key lime pie. With the ingredients on the screen, I asked Siri to add them to my shopping list, which it did quickly and easily.
Screenshot by Lance Whitney/ZDNET5. Revise a photo or other file
As an avid photographer, I’m always taking photos on my phone. Sometimes a shot doesn’t look good. Sure, I can always manually tweak the image using the different tools in the Photos app. But often it’s quicker to let Siri do the hard work and then review the results.
Also: After your iOS 27 upgrade, do these 5 things first
Here, I open a photo in Photos and tell Siri: “Enhance this photo to make it lighter.” Siri then makes the necessary adjustments automatically. If I’m happy with the results, I can then save the revised version.
Screenshot by Lance Whitney/ZDNETThese are just five ways I’ve taken advantage of Siri’s onscreen awareness. There are many other questions, requests, and tasks where Siri can come in handy. With the new AI flavor on your iPhone (or iPad or Mac), try using Siri to help you with a screen on your device and see how it performs.
