What is Apple Intelligence? Apple’s new AI toolkit explained

What is Apple Intelligence? Apple’s new AI toolkit explained


Apple Intelligence is official, and it’s a game-changer not only for the iPhone, but the iPad and Mac too. It brings with it an overhauled Siri experience, complete with new advanced functionality and features and even integration with competing LLMs.

Whether you want to use Siri to generate a recipe for dinner based on what’s in the fridge, create a custom emoji using Genmoji or simply rewrite your snarky email to sound a little professional, Apple Intelligence is on its way to help.

Here’s everything you need to know about Apple Intelligence, from the key new features and functionality to device support and the latest information on its release. 

What is Apple Intelligence?

Apple Intelligence is Apple’s spin on artificial intelligence – there’s no coincidence that they’re both shortened to ‘AI’. 

It’s essentially a system that utilises the ever-growing capabilities of generative AI to make your iPhone, iPad and Mac experience that little bit smarter and more helpful.

GenAI-powered features available under the Apple Intelligence umbrella range from small features like AI automatically summarising incoming notifications and emails to more powerful tools like ChatGPT integration within Siri, the ability to generate your own custom emoji for use in iMessage, tools to proofread and completely rewrite text and much more. 

Apple Intelligence on iPhoneApple Intelligence on iPhone
Image Credit (Trusted Reviews)

What does Apple Intelligence do?

Apple Intelligence encompasses a range of AI-powered features and functionality across iPhone, iPad and Mac, but there are a few ‘main’ features that you’ll likely want to take advantage of.

All-new Siri experience

With Apple Intelligence, Siri has had its biggest upgrade to date, not only sounding more natural than ever, but able to understand and do more with your requests. This is because Siri has been essentially rebuilt to utilise large language models, meaning it’s smarter than ever. 

Aside from a general face-lift, with a new all-screen Siri UI that looks much cleaner and still allows access to your apps during use, Siri can understand what’s going on on-screen, allowing Siri to perform actions like saving a friend’s new home address that he just texted you in the Messages app.

It also supports advanced in-app functionality, meaning you can get very specific with Siri when looking up photos, for example. 

New Siri UINew Siri UI

As well as searching for people and places, you can get very specific with the images you want to find. Apple showcased the example of Siri correctly finding all photos of a woman called Stacey in New York, but only those where she’s wearing a pink jacket.

It can also maintain conversational awareness across multiple prompts, allowing you to follow up contextually to previous Siri queries and responses and continue the conversation.

There’s also a new awareness of your personal information, garnered from everything from incoming messages to photos, event invites and even the links that you share with friends, and can use this information to help you in day-to-day life. 

Adding drivers license information to formAdding drivers license information to form

Say, for example, you need your driver’s license ID number, Siri can look up images of your driver’s license, copy the ID number and paste it into the app in question. You can also ask Siri what your plans are, and it’ll not only check the Calendar app but messages, emails and more to make sure nothing has been missed. 

Siri can also double up as a product expert, having been trained extensively on Apple products, and if you don’t want to speak to Siri out loud, you can double-tap the bottom of the screen to activate Type to Siri. 

It’ll also support advanced in-app functionality, meaning you can get very specific with Siri when looking up photos, for example.

Improving your writing skills

Considering that Apple Intelligence utilises Large Language Models to function, it comes as no surprise that it offers the ability to change the tone, or completely rewrite, chunks of text. It can also summarise large chunks of text, be it on a website, email or heartbroken text from an ex, and it’ll also enable smart replies in the Mail app, a la Gmail. 

This functionality works across your iPhone, iPad or Mac regardless of the app, and if you’d rather Apple Intelligence leave your words alone, you can always ask for a simple proofread. 

Proofreading on macOSProofreading on macOS

Genmoji and Images Playground

Genmoji may be one of the best features of Apple intelligence for one simple reason; it lets you create your own custom emoji to send to friends and family. 

GenmojiGenmoji

It’s baked directly into the Messages app, and essentially generates entirely new images based on your description. You can further customise this however you’d like, including the option of basing an emoji character’s look on a friend or family member, and then share away. 

Another related feature is Images Playground, which takes the concept and cranks it up a notch. The functionality is both baked into apps like Messages and Notes as well as being available in a standalone Images Playground app, and essentially lets you generate images based on prompts.

Photo generation in Images PlaygroundPhoto generation in Images Playground

From within the app, you can select a range of options from themes to costumes, accessories and more, and just like Genmoji, you can add images of friends and family to get more personal images generated. You then choose the style of the images – animated, sketched and illustrated – and wait for the magic to happen. 

These two features aren’t available right now, though they are slated to come as part of iOS 18.2, iPadOS 18.2 and macOS 15.2 in the coming months.

A smarter Photos app

Not to be outdone by GenAI editing tools from the likes of Google and Samsung, Apple Intelligence has brought generative fill capabilities to the Photos app via its new Clean Up tool. 

It’s not quite as advanced as what you’ll find on the Pixel 8 series, with features like Best Take essentially mashing a slew of photos together to make sure everyone’s looking at the camera and smiling, but the new Clean Up tool gives you the ability to remove people and other distracting objects from photos with a decent level of accuracy. 

Natural language search in Photos appNatural language search in Photos app

As well as editing photos, the new Photos app uses AI to help surface the exact content you’re looking for, and it’ll even search through video clips to surface the relevant results too.

It’s also way better at creating montage videos, with a new understanding of your images and videos that allows it to craft more of a storyline than the random smattering of images and videos you were previously provided with. 

Streamline your emails

Apple’s Mail app now includes a section at the top of your inbox (Priority Messages) which lists the most urgent and time-sensitive emails, such as a same-day party invitation or a boarding pass. 

That’s not the only change with the app. While email previews are limited to the first few lines of text on non-supported iPhones, Apple Intelligence users will instead see short summaries of the emails without needing to open them. In addition, for longer email threads Apple Intelligence allows you to view important details with just a tap, saving you the hassle of scrolling through countless emails.

Finally, there’s also Smart Reply which provides suggestions for a quick response. 

Priority Messages screenshot on iOS 18Priority Messages screenshot on iOS 18

Summarise phone calls

The use of Apple Intelligence promises you’ll never have to scramble for a pen and paper while on a phone call again. Simply initiate a recording while on a phone call on your iPhone and once the call ends, Apple Intelligence will generate a summary of key points for easy recall. This tool can also be used in the Notes app for summarising key points of audio recordings. 

As privacy is a big factor of Apple Intelligence, all phone call participants will be automatically notified that this tool has been enabled. Google offers a similar feature with its Call Notes tool, which also offers users an AI-powered summary of a phone call.

Audio transcript from phone call on iOS 18Audio transcript from phone call on iOS 18

ChatGPT integration

Apple Intelligence is a cool new feature for Apple devices, but it still can’t quite do everything that competing LLMs can do – and that’s why Apple is actively working on integrating third-party LLMs into the Apple Intelligence experience, starting with ChatGPT.

That means that Siri can lean on ChatGPT to help you with your queries. Say you want to create a menu for dinner based on the ingredients in your fridge – in that scenario, Siri is smart enough to know that ChatGPT will be the most useful, and it’ll automatically redirect the query to ChatGPT to handle. 

ChatGPT integrationChatGPT integration

It also means that you can use the ChatGPT integration to get help using images, i.e. asking for advice on what colour to paint your wall with an image of your front room, and if you’re a ChatGPT Pro subscriber, you’ll be able to access all your premium features from within Siri too.

It’s limited to ChatGPT for now, but Apple has teased that other LLMs could be supported in future. 

And, like Genmoji, it isn’t available just yet, though it’s another feature expected to roll out with the iOS 18.2, iPadOS 18.2 and macOS 15.2 update in the near future. 

What is Apple Intelligence powered by?

The core reason why Apple Intelligence isn’t available across every Apple product – something we come to in more detail shortly – is down to the fact that the AI functionality runs entirely on device, not requiring an internet connection or cloud servers to perform its actions. 

As you’d imagine, that requires quite a bit of power, and that comes exclusively in the form of Apple silicon chipsets.

More specifically, only recent iPhone and iPad chips have powerful enough NPUs (neural processing units) to handle the demands of Apple Intelligence, specifically the A17 Pro, A18 and A18 Pro, while Apple Intelligence on Mac is only available for Apple M-series chipsets; no Intel Mac offers support for the functionality, and that’s unlikely to change. 

What is Apple Intelligence available on?

As noted, Apple Intelligence is available across iPhone, iPad and Mac, though not every single Apple product is powerful enough to power the on-device generative AI experience, especially when it comes to Apple’s phone and tablet collections.

That said, here’s a complete list of every iPhone that supports Apple Intelligence:

  • iPhone 15 Pro
  • iPhone 15 Pro Max
  • iPhone 16
  • iPhone 16 Plus
  • iPhone 16 Pro
  • iPhone 16 Pro Max

Here are all the iPads that support Apple Intelligence:

  • iPad Mini (2024)
  • iPad Air (2022)
  • iPad Air (2024)
  • iPad Pro (2021)
  • iPad Pro (2022)
  • iPad Pro (2024)

And here are all the Macs that support Apple Intelligence:

  • MacBook Air (2020)
  • MacBook Air (2022)
  • MacBook Air (2023)
  • MacBook Air (2024)
  • MacBook Pro (202)
  • MacBook Pro (2022)
  • MacBook Pro (2023)
  • MacBook Pro (2024)
  • iMac (2021)
  • iMac (2023)
  • iMac (2024)
  • Mac Mini (2020)
  • Mac Mini (2023)
  • Mac Mini (2024)
  • Mac Studio (2022)
  • Mac Studio (2023)
  • Mac Pro (2023)

Apple Intelligence release date

The most confusing aspect of Apple Intelligence is arguably its availability. To say that it’s confusing is a bit of an understatement.

First up, there’s no Apple Intelligence in countries in the EU right now, down to strict laws around the use of AI, though Apple says that Intelligence will come to the EU sometime in April 2025.

Instead, the initial rollout of Apple Intelligence is focused on the US, with the first handful of Apple Intelligence features rolled out as part of iOS 18.1, iPadOS 18.1 and macOS 15.1 in late October. Despite being US-only, those in the UK and other (non-EU) regions can change their device’s region and language to US English and get access now.

As for the wider rollout, it’s set to be staggered. Apple has confirmed that UK English support is coming alongside support for English in Canada, Australia, Ireland, New Zealand and South Africa as part of the iOS 18.2, iPadOS 18.2 and macOS 15.2 updates likely to drop towards the end of 2024. 

Support for other languages, including Chinese, French, German, Italian, Japanese, Korean, Portuguese, Spanish, Vietnamese and other languages are also said to be coming a little later down the line in April 2025. 

It’s also worth noting that not all Apple Intelligence features are available right now. Instead, the first Apple Intelligence feature drop focused on some of the more basic features like notification summaries, the Clean Up tool, writing tools and a new Siri UI, with a truly more capable Siri, ChatGPT integration, Genmoji and Image Playgrounds all still in development. 



Source link

Draugai: - Marketingo agentūra - Teisinės konsultacijos - Skaidrių skenavimas - Fotofilmų kūrimas - Miesto naujienos - Šeimos gydytojai - Saulius Narbutas - Įvaizdžio kūrimas - Veidoskaita - Nuotekų valymo įrenginiai - Teniso treniruotės - Pranešimai spaudai -