Every New Apple Intelligence Feature iPhone Users Should Know

The long-awaited iOS 18 is here for download on all compatible devices. The update was fairly light, with small (but welcome) refinements and added features to an already solid platform. However, if you remember, Apple broke its habitual silence earlier this year to tell people that iOS 18 would be its biggest update, ever. So now we’ve got the update, where is the big part?

The answer is the upcoming Apple Intelligence (AI), Apple’s response to Gemini and all the amazing AI features on the Pixel and Samsung Galaxy. However, Apple won’t release any Intelligence features until 18.1 in October. Over the course of the next year, your iPhone is going to take little AI leaps forward — as long as it’s an iPhone 15 Pro or above.

Apple Intelligence was the climax of the 2024 WWDC presentation, and it ended up being so meaty that many features didn’t get a mention and were only discovered later by beta testers. We’ve tested all the best features for you, but you might’ve missed one or two. Apple had to pack it all into a 20-minute segment, making it easy for things to fall through the cracks. So in that spirit, let’s look at all the Apple Intelligence features on iPhones that you should know about.

Fully revamped Siri

In a competition against other digital assistants, Siri sometimes looks like a drunk running around with a bucket over her head. She struggles to understand you, fails to carry out some of the most simple tasks imaginable, can’t remember previous context, and will direct you to a Google search rather than grabbing the results for you. That’s about to change with future iOS 18 updates.

Siri 2.0 will let you speak with natural language rather than the stilted variety you use when interacting with computers. She’ll have a deeper knowledge of your personal context — your conversations, emails, photos, documents — so that she can be something approximating an actual assistant. The WWDC 2024 Siri demonstration shows her solving fairly challenging problems and using your screen’s content to know what she needs to help with.

The downside is this is that all these new Siri features are only going to trickle in over the next year. She’ll first appear in 18.1 in October, but only with an updated UI, text-to-Siri, a more human-like voice, and improved comprehension, which means that you can stutter and stumble while still getting the point across. She’ll remember previous questions so she can follow up on them, and provide basic tech support. As for the Jarvis-level Siri we were promised, that’s not coming until 2025 with 18.4 – so, you might want to wait for the iPhone 17 instead of upgrading to the 16.

Query ChatGPT with no account

Perhaps the most surprising announcement at WWDC 2024 was that Apple will partner with its competitor, Microsoft, by implementing ChatGPT directly into iOS 18. ChatGPT has a ton of unexpected uses, but the only way to use it on an iOS device is with the app or the website. By 18.2, you’ll be able to query ChatGPT quickly and easily through Siri, or use it with Writing Tools. Typically, Siri will only suggest using ChatGPT if she’s tasked with an especially difficult request, such as, say, spitting out a recipe. The upshot of the partnership is that Apple seems to have strong-armed OpenAI into a privacy-centric implementation.

On iPhone and all other supported devices you will be able to use ChatGPT without an account. Further, ChatGPT pledges not to store any of these Apple-device-based requests and to protect your IP address. If you have a ChatGPT subscription, then you can connect your account to use the features you paid for, like GPT-4o. Tim Cook has confirmed the ChatGPT/Siri release by the end of 2024 in an earnings call. Apple typically releases its x.2 update in December, so it’s likely you’ll be able to use this new feature before Christmas.

Genmoji

Have you ever found that even despite a vast library of emojis, none of them describe that particular feeling you’re trying to convey in the moment? iOS 18 solves that.

Apple spent a ton of time at the WWDC presentation teasing image generation features coming to iOS 18, one of which was Genmoji. Basically, you describe what you’re envisioning with a text prompt, and your phone produces a tiny image that behaves like an emoji, which is perfect for chats in iMessage. The image generator creates multiple options for you to choose from in case the first one isn’t quite what you were looking for. We’ve yet to see this feature in beta, but if it works, it’s probably going to be one of iOS 18’s most cherished features, one the competition is bound to replicate.

Genmoji can go a step further by depicting your friends. Simply type in their name as part of the prompt, and it will use the picture in your contacts as a reference. Genmojis can also be saved and used for tapback reactions in iMessage, although it remains to be seen if third-party messengers like WhatsApp and Telegram will support them in any capacity. You’ll have to wait for this feature, too, as it’s believed we won’t see it until 18.2 in December.

Image Playground

Everyone and their dog has gotten on the image generation bandwagon lately, and Apple couldn’t help but add air to the AI bubble. Image Playground is its answer to Midjourney and DALL-E. You can use it directly from iMessage, or in the coming Image Playground app. It works almost exactly the same as the competition: describe the image in your mind’s eye and then wait for your iPhone to generate it, albeit completely on-device.

To streamline the process and make it run efficiently, Apple has put some pretty heavy guardrails in place. The images are small, and can only adhere to three non-realistic styles: Illustration, Sketch, and Animation — so, no risk of AI misinformation here. Like Genmoji, it can produce pictures of your friends using your photo library as a reference.

To be clear, Image Playground is not meant to go toe-to-toe with its far more powerful competitors. It’s a very basic image generator for spur-of-the-moment, goofy pics that you generate to get a chuckle from the group chat. Don’t expect to use this to make HD phone backgrounds or hero images for your blog.

The results we’ve seen at WWDC were a mixed bag. Like most AI art, they have a unnerving, soulless quality to them, and there’s been only one image generated outside the scripted confines of WWDC – Senior Vice President of Software Engineering Craig Federighi’s dog celebrating its birthday. Here’s to hoping it improves a lot when it releases, allegedly, for that December 18.2 window.

Better note-taking in Notes

Apple Notes is great, but it could be better. iOS 18’s Apple Intelligence aims to supercharge it to new levels. The marquee feature here is Writing Tools, which can help tidy up, rewrite, change the tone of, and proofread your rough copy. This will be a game-changer for messy notetakers, since you can summarize big walls of text or break them up into key points, tables, and lists. If your hands are aching from all that thumb-tapping, then you can transcribe those thoughts directly into a note. If you need to do basic math equations, you can add an equal sign to the end of a formula and let Apple do the tally, and run equations by hand with Math Notes, if you like.

For those who prefer strictly handwritten notes, Smart Script will beautify even the worst chicken scratch and let you edit, copy, paste, and reorder your handwriting as if it were a built-in font—though to be clear, it’s not yet known if Smart Script in particular will come on the iPhone, since it has only been demonstrated on iPad. Last, Image Wand turns any back-of-the-napkin sketches on the margins of your notes into beautiful graphic elements. If you can’t draw at all, then Apple Intelligence can generate an illustration using keywords it finds in nearby text.

Combined with the options to highlight text and collapse headers, Notes is shaping up to be one of the best note-taking apps on the market. All these features release at different times. Expect stuff like Writing Tools by 18.1, with Image Wand and others by 18.2.

Better Mail management

iPhone’s Mail app has never been something to write home about — pun intended — but it looks like Apple is trying to change that. It’s streamlining things by intelligently categorizing your emails so you can sift important correspondence out of a raft of promotions and account alerts.

Using Apple Intelligence, it will pull priority emails (ones it deems important based on content) and all unread emails to the top. Email summaries will save you time by boiling down long, tedious emails into key points. If you have a lot of similar emails from one particular sender, then Digest View will condense all the most relevant information into one page. Most of this stuff has existed in some form or another with competing apps for years, but it’s better late than never.

Writing Tools also comes to your aid if you want to touch up an email before it goes out. If your words sound belligerent, Writing Tools can re-jig it to sound friendly or professional. Perhaps the biggest feature, though, is Smart Reply. Apple Intelligence reads through the email and then asks you questions on how you plan to respond. Using your answers, it spits out a bespoke response in the same way you’d use ChatGPT. This could be an absolutely massive time saver if you spend a chunk of your day replying to long emails. The improved Mail update will likely appear in December with 18.2.

Camera Control visual intelligence

Apple has always been behind in image processing AI features compared to Android. Google introduced Google Lens, which lets you snap a picture of something and identify it, years back and recently debuted Circle to Search to image-search for anything that appears on your screen. Apple’s Visual Look Up kind of does the latter, albeit only with photos in your library and very poorly. While we wait for it to improve the latter and brew a proper Circle to Search competitor for iOS 19 (among other things), iOS 18 is catching up slowly through Camera Control’s visual intelligence.

Like Google Lens, you open the camera app, point the camera at something you wish to identify, and then hold down the new Camera Control button, which is only available if you upgrade to the iPhone 16. Examples of things to identify might be a restaurant, an unknown breed of dog, or an event poster. Your iPhone then uses on-device intelligence to recognize them and find the Apple Maps info for that restaurant, the breed of that dog, or details for the event. Even better, Apple is opening the door to third parties so you can use Camera Control with Google or ChatGPT.

We don’t yet know when it releases since Apple did its typical, cryptic “later this year,” so you can likely with expect it with 18.2.

Clean Up tool

Speaking of needed image-related AI features, another big one is Google’s Magic Eraser. Pixel 6 users way back in 2021 were able to remove unwanted background elements from their pictures. It worked very well at the time, and the iPhone is getting in on the action with the Clean Up tool.

It works more or less the same as the Magic Eraser it’s copying. Simply tap predetermined elements, circle an area, or scratch out whatever it is you don’t like. Give it a second to process as it uses machine learning to fill in the blank. Since the newer iPhones capture depth of field information in portraits, Clean Up is able to do a fairly good job since it knows better how to delineate subject from background.

Of course, this is not going to be on par with Photoshop. It’s best if you use it for small objects on a relatively uniform background. Any Clean Up edits you make show up in the metadata so your recipients can be aware it’s been doctored. You can reverse these AI touchups at any time, since your iPhone will keep the original, unaltered image on ice. Expect to see this one by 18.1 in October.

Memory movies

Social media sites like Facebook and Instagram have long been able to create slideshows for your pictures and videos, complete with music and transitions. There’s just one problem — to make these slideshows, you need to trust Meta and its atrocious privacy record. One reason people buy iPhones in the first place is for the sake of privacy. So, fans of said slideshows will be happy to know that Apple Photos in iOS 18 is bringing its own version in the form of Memory movies.

Memory movies works with prompts, just like Image Playground. You can specify the location, the occasion — birthday parties, anniversaries, etc. — and the mood and style you’re looking for, then let your iPhone do the rest.

Users who have tried the feature in beta say it only takes a few seconds and that the results are decent, though nothing groundbreaking. If you’re unhappy with the movie, you just re-enter the prompt and make manual edits to taste until you get what you’re looking for. It’ll be highly useful anytime you want to commemorate an important day on social media, or a lifesaver if you need to whip up a slideshow for an upcoming event in a pinch. It looks like this one is also coming to 18.1 in October.

A streamlined, distraction-free Safari

Safari on iPhone is, in this writer’s opinion, is the best browser for iOS — but probably only because Apple forces competing browser developers to use WebKit outside the EU instead of their own engines. So until that changes, there’s not much incentive to switch to another one. To Safari’s credit, it is good, and it does keep getting better, so that bitter pill of your preferred browser being neutered is easier to swallow. In iOS 18, Safari’s Reader Mode gets a new coat of paint. In addition to an improved, cleaner reading experience, Reader Mode summarizes article pages with AI for a quick overview which, according to beta users, only takes a couple of seconds at most.

Outside of Reader Mode, there may be some annoying webpage elements that keep bugging you. Distraction Control allows you to manually select and remove anything — ads, cookie popups, banners, you name it — to trim the page and free up resources. Ostensibly, Safari will remember which elements you wiped out so the page won’t reset if you refresh it or come back later, though keep in mind your preferences won’t sync to other Apple devices. If an element you removed was important, don’t worry, you can simply unhide it later. The summarize feature isn’t coming until 18.1, but you can try out Distraction Control on iOS 18 right now.

Apple Intelligence Report

During WWDC 2024, Apple really hammered home the privacy focus of all their Apple Intelligence features. It worked hard to pull off everything it could on-device, and built new privacy-honed server architecture (Private Cloud Compute) for any requests that need to be handled in the cloud.

Regardless, iPhones are closed-source, meaning users can only trust Apple’s word that their privacy is being respected. Further, while Private Cloud Compute will have independent third-party audits, these probably won’t be happening all that frequently and won’t provide personalized insight for individual users. To alleviate those concerns, Apple Intelligence will create automated privacy reports so users can see exactly when their information is leaving their device to Apple’s Private Cloud Compute.

In the Privacy & Security section under Apple Intelligence Report (once, of course, Apple Intelligence releases in 18.1) you’ll be able to export your activity. This information is protected with Face ID. The only downside is that these reports don’t stick around for long. You can set them to expire within 15 minutes or 7 days. It would be nice if they were kept for an indefinite period at the user’s discretion, but regardless, this sets a high bar for AI transparency compared to the rest of the tech industry.

Originally Appeared Here