Apple Intelligence sets a new standard for privacy in AI by understanding personal context to provide helpful and relevant insights.
Apple has unveiled Apple Intelligence, a new personal intelligence system for iPhone, iPad, and Mac. This system combines generative models with personal context to provide highly useful and relevant insights. Integrated into iOS 18, iPadOS 18, and macOS Sequoia, Apple Intelligence leverages the power of Apple silicon to understand and generate language and images, perform actions across apps, and utilize personal context to simplify and speed up everyday tasks. With Private Cloud Compute, Apple ensures top-notch privacy in AI, balancing on-device processing with larger server-based models on dedicated Apple silicon servers.
“We’re excited to introduce a new chapter in Apple innovation,” said Tim Cook, Apple’s CEO. “Apple Intelligence will revolutionize what users can achieve with our products and what our products can offer them. Our approach combines generative AI with personal context to provide truly helpful intelligence, accessing this information privately and securely. This is AI as only Apple can deliver it, and we can’t wait for users to experience its capabilities.”
New Capabilities for Understanding and Creating Language
Apple Intelligence introduces new features to help users improve their writing and communicate more effectively. With the integration of Writing Tools into iOS 18, iPadOS 18, and macOS Sequoia, users can now rewrite, proofread, and summarize text across various applications such as Mail, Notes, Pages, and third-party apps.
These Writing Tools assist users in refining their work, whether they are organizing class notes, perfecting a blog post, or crafting a well-written email. The Rewrite feature allows users to select from different versions of their text, adjusting the tone to match the intended audience and purpose. Whether it’s refining a cover letter or infusing humor into a party invitation, Rewrite helps users choose the most suitable words for any occasion. Proofread analyzes grammar, word choice, and sentence structure, offering edits and explanations that users can review or accept. With Summarize, users can condense selected text into a concise paragraph, bulleted points, a table, or a list.
For instance, a user can access the Writing Tools menu while composing an email to select Proofread or Rewrite. Similarly, another user can utilize the Summarize tool in the Notes app to condense their notes on holistic health.
Managing Emails:
In the Mail app, managing emails has become simpler than ever. The introduction of Priority Messages presents a new section at the top of the inbox, highlighting the most pressing emails, such as same-day dinner invitations or boarding passes. Rather than displaying the first few lines of each email, users can now see summaries across their inbox without having to open each message. Additionally, for lengthy email threads, users can access relevant details with a single tap. Smart Reply offers quick response suggestions, and it can even identify questions within an email to ensure comprehensive responses.
A deep understanding of language also applies to Notifications. Priority Notifications are positioned at the top of the stack to highlight the most important information, while summaries aid users in quickly scanning long or stacked notifications by displaying key details directly on the Lock Screen. For instance, users can easily identify when a group chat is highly active. Additionally, to assist users in staying focused on their tasks, Reduce Interruptions introduces a new Focus feature that only displays notifications requiring immediate attention, such as a text message about an early pickup from daycare.
In the Notes and Phone apps, users now have the ability to record, transcribe, and summarize audio. When users start a recording during a phone call, all participants are automatically notified. After the call concludes, Apple Intelligence generates a summary to assist in recalling key points discussed during the conversation.
Image Playground Makes Communication and Self‑Expression Even More Fun
Apple Intelligence enables innovative image creation features to facilitate new modes of communication and self-expression for users. With Image Playground, users can effortlessly craft engaging images in mere seconds, selecting from three distinct styles: Animation, Illustration, or Sketch. Image Playground is seamlessly integrated into various apps, including Messages, ensuring easy access for users. Additionally, it’s accessible as a standalone app, ideal for exploring various concepts and styles. Notably, all images are generated on the device, granting users the flexibility to experiment with an unlimited number of images.
Using Image Playground, users have the option to select from various concepts categorized under themes, costumes, accessories, and places. They can also input a description to specify their desired image, incorporate someone from their personal photo library into the image, and choose their preferred style.
Through the Image Playground feature within Messages, users can promptly generate entertaining images to share with their friends. Additionally, they have the option to receive personalized suggested concepts tailored to their ongoing conversations. For instance, if a user is discussing plans for a hiking trip with a group, they’ll receive suggestions related to their friends, the destination, and the activity, streamlining the image creation process and making it more pertinent to the discussion.
In Notes, users can now utilize Image Playground via the new Image Wand tool in the Apple Pencil tool palette, enhancing the visual appeal of their notes. This feature allows users to transform rough sketches into charming images, and they can even select empty spaces within notes to generate images based on the surrounding context. Furthermore, Image Playground is accessible in other applications such as Keynote, Freeform, and Pages, as well as in third-party apps that integrate the new Image Playground API.
Related post: Apple Pencil Pro: Advance Creativity for iPad Users
Genmoji Creation to Fit Any Moment
Elevating emojis to a whole new level, users now have the ability to craft their own unique Genmoji to convey their personality. By typing a description, their personalized Genmoji instantly materializes, offering additional customization options. Users can even create Genmoji representations of their friends and family using their photos. Similar to emojis, Genmoji can be seamlessly inserted into messages, shared as stickers, or used as reactions in a Tapback.
New Features in Photos Give Users More Control
With Apple Intelligence, searching for photos and videos becomes notably more convenient. Users can employ natural language to search for specific photos, like “Maya skateboarding in a tie-dye shirt” or “Katie with stickers on her face.” Moreover, video search capabilities are enhanced, allowing users to pinpoint specific moments within clips, enabling them to navigate directly to the relevant segment. Additionally, the new Clean Up tool can identify and remove distracting objects from the background of a photo, ensuring that the subject remains unaltered.
With Memories, users can curate the story they desire by typing a description. Leveraging language and image comprehension, Apple Intelligence selects the finest photos and videos based on the description, constructs a narrative with chapters derived from identified themes within the photos, and organizes them into a movie with its own storyline. Additionally, users receive song recommendations from Apple Music to complement their memory. As with all Apple Intelligence features, user photos and videos remain private on the device and are not shared with Apple or any other entity.
Siri Enters a New Era
Leveraging Apple Intelligence, Siri achieves deeper integration into the system experience. With enhanced language-understanding capabilities, Siri becomes more natural, contextually relevant, and personalized, enabling the simplification and acceleration of everyday tasks. It can seamlessly follow along if users stumble over words and retain context from one request to the next. Furthermore, users have the option to type to Siri and seamlessly switch between text and voice inputs based on their preferences. Siri also sports a fresh design featuring an elegant glowing light that encircles the edge of the screen when Siri is active.
Users Device Support:
Siri now offers users device support wherever they are, providing answers to thousands of questions about using iPhone, iPad, and Mac. Users can learn various tasks, ranging from scheduling emails in the Mail app to switching between Light and Dark Mode.
With onscreen awareness, Siri will gradually gain the ability to comprehend and act upon users’ content in a wider range of apps. For instance, if a friend sends their new address via Messages, the recipient can simply say, “Add this address to their contact card,” prompting Siri to take appropriate action.
With Apple Intelligence, Siri gains the capability to perform hundreds of new actions within both Apple and third-party apps. For instance, a user could command, “Retrieve the article about cicadas from my Reading List,” or “Send the photos from the barbecue on Saturday to Malia,” and Siri will execute the task accordingly.
Siri will have the ability to provide personalized intelligence based on the user’s on-device information. For instance, a user can request, “Play the podcast that Jamie recommended,” and Siri will locate and play the episode, eliminating the need for the user to remember whether it was mentioned in a text or email. Similarly, they could inquire, “When is Mom’s flight landing?” prompting Siri to retrieve the flight details and cross-reference them with real-time flight tracking to provide an arrival time.
A New Standard for Privacy in AI
For Apple Intelligence to truly assist users, it relies on comprehending deep personal context while prioritizing user privacy. A fundamental principle of Apple Intelligence is on-device processing, with many of its models operating entirely on the user’s device. For more complex tasks requiring additional processing power, Private Cloud Compute is employed, extending the privacy and security measures of Apple devices into the cloud to enhance intelligence capabilities.
With Private Cloud Compute, Apple Intelligence can adapt and scale its computational resources, accessing larger, server-based models for handling intricate requests. These models operate on servers equipped with Apple silicon, establishing a framework that ensures data is never stored or exposed.
Independent experts have the opportunity to review the code running on Apple silicon servers to validate privacy measures. Moreover, Private Cloud Compute employs cryptographic methods to ensure that iPhones, iPads, and Macs only communicate with servers whose software has undergone public logging for inspection. Apple Intelligence, combined with Private Cloud Compute, establishes a new benchmark for privacy in AI, delivering intelligence that users can rely on with confidence.
ChatGPT Gets Integrated Across Apple Platforms
Apple is integrating ChatGPT access into experiences within iOS 18, iPadOS 18, and macOS Sequoia, enabling users to access its expertise, along with its image- and document-understanding capabilities, seamlessly without needing to switch between tools. Siri can leverage ChatGPT’s expertise when it’s beneficial. Users are prompted for permission before any questions, along with associated documents or photos, are sent to ChatGPT. Siri then delivers the answer directly to the user.
Furthermore, ChatGPT will be accessible in Apple’s systemwide Writing Tools, aiding users in generating content for any topic they’re writing about. Through Compose, users can also utilize ChatGPT image tools to create images in various styles to complement their writing.
Users accessing ChatGPT have integrated privacy protections, with OpenAI obscuring their IP addresses and not retaining requests.
Users who opt to connect their accounts are subject to ChatGPT’s data-use policies. ChatGPT, powered by GPT-4o, will be available on iOS 18, iPadOS 18, and macOS Sequoia later this year. Users can access it for free without creating an account, while ChatGPT subscribers can connect their accounts to access paid features directly from these experiences.
What are generative models?
Generative models are algorithms within the realm of artificial intelligence (AI) that have the capability to generate new content, such as images, text, or music, based on patterns learned from existing data. These models have seen significant advancements in recent years, particularly in fields like deep learning and neural networks.
2. How is Apple using generative models in its products?
Apple has been integrating generative models into its flagship products, including the iPhone, iPad, and Mac, to enhance user experience in various ways. For example, these models are used in computational photography to improve image quality, in natural language processing for features like Siri and predictive text, and in creative applications like GarageBand and Logic Pro X to assist musicians in composing music.
3. What benefits do generative models bring to Apple products?
Generative models enable Apple to offer features that anticipate user needs, enhance creativity, and improve efficiency. For instance, in photography, these models help enhance photos in real-time, while in music composition, they provide tools for generating virtual session musicians and suggesting chord progressions.
4. How does Apple ensure user privacy and data security when using generative models?
Apple takes user privacy and data security seriously and has implemented stringent measures to protect user data. This includes using on-device processing whenever possible to minimize the need for data to be sent to servers, as well as employing encryption and other security protocols to safeguard user information.
5. What are some ethical considerations surrounding the use of generative models?
Ethical considerations around generative models include concerns about data privacy, algorithmic bias, and the potential misuse of AI-generated content. Apple commits to addressing these concerns by prioritizing user privacy, implementing responsible AI practices, and advocating for industry-wide ethical guidelines.
6. How will generative models continue to evolve in Apple products in the future?
As technology advances, generative models will likely play an even greater role in shaping the future of Apple products. This could involve further improvements in areas like photography, music composition, and natural language processing, as well as the development of new applications and features that harness the power of AI.
7. Where can I learn more about Apple’s use of generative models?
For more information about how Apple is using generative models in its products, you can visit the company’s official website, read press releases and announcements, and explore developer documentation and resources related to AI and machine learning. Additionally, tech news outlets and industry publications often cover developments in this field.