After months of teases and rumours, Apple finally unveiled its AI-powered software to be included in its devices at its WWDC 24 conference – Apple Intelligence.
The AI software is heading to the iPhone, iPad and Mac, integrated into the latest operating systems for each of the device groups. The AI “harnesses the power of Apple silicon to understand and create language and images, take action across apps, and draw from personal context to simplify and accelerate everyday tasks,” the company explains in a statement following the WWDC reveal.
This particular statement explains the AI capabilities of Apple Intelligence quite well. They are broken down into three sections: natural language, image generation, and actions across apps with an upgraded Siri.
Natural language with Apple Intelligence
The first feature of Apple Intelligence is one anyone who has used generative AI before may be familiar with. These natural language processes, like ChatGPT, can understand concepts and contexts like a human can understand them, and give answers as if from a human mind.
Apple has integrated these processes across apps, including Mail, Messages, Notes, Notifications. and other apps.
“Whether tidying up class notes, ensuring a blog post reads just right, or making sure an email is perfectly crafted, Writing Tools help users feel more confident in their writing. With Rewrite, Apple Intelligence allows users to choose from different versions of what they have written, adjusting the tone to suit the audience and task at hand,” the statement reads.
Apple calls this writing software Rewrite, and like ChatGPT it can be used to write cover letters, emails, party invitations and essentially any other text or writing you may require. Another piece of the software, Summarize, does exactly what it says, in that it can be used in apps like Mail, to summarise emails, or in Notes, to summarise long pieces of text.
Apple Intelligence pop up with Mail, asking the user how they would like the paragraph to sound, and other options.
But the natural language processes also apply to voice, and now apps like Notes can record, transcribe, and summarize audio. This also applies to phone calls. For example, if you need to record an important phone call for later, you can use Apple Intelligence to turn on the audio transcriber.
Apple says that the other person on the call will be notified that their audio is being recorded.
Image generation and magic editing
Another generative AI staple is image generation, and of course, Apple has integrated image generation technology into their software. This is in the guise of new apps and integration into other apps.
First off, is the new app Image Playground.
Here, the generative AI behind Apple Intelligence can be given certain prompts and image themes to generate new images. Using a selfie, and a pre-set prompt for “space” and “astronaut”, the software then produces an image of you as an astronaut.
This is an interesting workaround for one of the biggest problems with complex image generation. Learning how to prompt properly to get the final product you want is time-consuming. Apple is flexing its accessibility muscles here, making a powerful image generation tool that is easy to use.
Like the natural language processes, Image Playground is integrated into other apps too. In Messages, Apple Intelligence will suggest prompts for generating images using what is being spoken about.
“For example, if a user is messaging a group about going hiking, they’ll see suggested concepts related to their friends, their destination, and their activity, making image creation even faster and more relevant,” the company explains.
This applies to Notes as well. With the Notes app now able to generate images within the notes you are taking using the context of the notes. This includes rough sketches too, which is similar to the capabilities of OpenAI’s Dall-E image generator. Check this out in action in the video below:
The final improvement in terms of visuals, is Apple Intelligence integration into the Photos app. You can now use the AI to find particular photos, which will be a godsend for certain people who take millions of photos and or in several busy WhatsApp groups and their Photos folders contain more images than the Louvre.
For example, on Photos, you will be able to prompt “Find me the picture of John dancing during the wedding” and Intelligence will bring up a few suggestions that most closely match the prompt.
When viewing particular photos, you can now use Clean Up to remove pesky photo bombers from family portraits. This is similar to Google’s Magic Eraser tool with Gemini.
Siri can now use apps for you thanks to Apple Intelligence
Finally, Siri is the AI-powered assistant that it was first envisioned to be thanks to the innovation of generative AI.
“With richer language-understanding capabilities, Siri is more natural, more contextually relevant, and more personal, with the ability to simplify and accelerate everyday tasks,” Apple describes.
“It can follow along if users stumble over words and maintain context from one request to the next. Additionally, users can type to Siri, and switch between text and voice to communicate with Siri in whatever way feels right for the moment,” the company adds.
If that sounds familiar, it’s because this is how the ChatGPT app works. Imagine that but with access to your personal app preferences. This could potentially be a game changer for smartphones.
Siri now has onscreen awareness. The software will now be able to understand and take action with user content. For example, if a friend texts you a new address in Messages, you can ask Siri to add the address to the friend’s contact card.
This integration is across Apple and even third-party apps. A user could say, “Bring up that article about cicadas from my Reading List,” or “Send the photos from the barbecue on Saturday to Malia,” and Siri will “take care of it.” Apple says.
Finally, Apple has revealed its generative AI partner, which is why we kept bringing them up. Apple Intelligence is using an external generative AI model from OpenAI, likely GPT-3.5 or even GPT-4 to power all of these features on the in-house-made M4 chip.
This was long expected, as rumours held that Apple would be bringing generative AI to its devices but it would not be building its own large language models. The partnerships also extends to ChatGPT, with the software integrated into apps like Siri to search the internet and ask questions, as well as into Writing Tools and more.
This is a major coup for OpenAI in the AI space, with two of the world’s wealthiest companies in Apple and Microsoft choosing it as its generative AI partner. If there wasn’t a leader in the AI software space before, there is definitely one now.
Apple says that Intelligence is free for all users, and will be available first in beta as part of iOS 18, iPadOS 18 and macOS Sequoia. It will only be available in US English at first, but more features and languages will be added over the course of next year.
Only the latest Apple devices will get Apple intelligence – iPhone 15 Pro, iPhone 15 Pro Max, and iPad and Mac with M1 and later. The base iPhone 15 is excluded, which means that Apple Intelligence could be marketed as a Premium feature.
Also, it has to be said that Apple Intelligence absolutely destroys the meagre AI offering that Samsung is currently peddling, most of which is not even generative AI. Expect Samsung to not let this sleight go without a response.
