We’re excited to announce the release of Mastering SwiftUI 7, fully updated for iOS 26 and Xcode 26.

This latest edition reflects the most recent SwiftUI APIs and development tools, ensuring your learning stays current with the evolving Apple ecosystem.

In addition to the complete update, we’ve added several new chapters covering some of the latest features introduced in iOS 26:

• Exploring WebView and WebPage in SwiftUI for iOS 26
• Getting Started with Foundation Models in iOS 26
• Working with @Generable and @Guide in Foundation Models
• Using Tool Calling to Supercharge Foundation Models
• Developing Siri Shortcuts with App Intents
• Developing Live Activities in SwiftUI Apps

You can preview the latest content in the sample book here.
​

Live Activities, first introduced in iOS 16, are one of Apple's most exciting updates for creating apps that feel more connected to users in real time. Instead of requiring users to constantly reopen an app, Live Activities let information remain visible right on the Lock Screen and Dynamic Island. Whether it's tracking a food delivery, checking sports scores, or monitoring progress toward a goal, this feature keeps important updates just a glance away.

Later in iOS 17, Apple expanded Live Activities even further by supporting push updates from the server side, which makes them even more powerful for apps that rely on real-time information. But even without server-driven updates, Live Activities are incredibly useful for client-side apps that want to boost engagement and provide timely feedback.

In this tutorial, we'll explore how to implement Live Activities by building a Water Tracker app. The app allows users to log their daily water intake and instantly see their progress update on the Lock Screen or Dynamic Island. By the end of the tutorial, you'll understand how to integrate Live Activities into your SwiftUI apps.

A Quick Look at the Demo App

Our demo app, Water Tracker, is a simple and fun way to keep track of your daily water intake. You’ve probably heard the advice that drinking eight glasses of water a day is a good habit, and this app helps you stay mindful of that goal. The design is minimal on purpose: there's a circular progress bar showing how far along you are, and every time you tap the Add Glass button, the counter goes up by one and the progress bar fills a little more.

Behind the scenes, the app uses a WaterTracker class to manage the logic. This class keeps track of how many glasses you’ve already logged and what your daily goal is, so the UI always reflects your current progress. Here’s the code that makes it work:

import Observation

@Observable
class WaterTracker {
    var currentGlasses: Int = 0
    var dailyGoal: Int = 8
    
    func addGlass() {
        guard currentGlasses < dailyGoal else { return }
            
        currentGlasses += 1
    }
    
    func resetDaily() {
        currentGlasses = 0
    }
    
    var progress: Double {
        Double(currentGlasses) / Double(dailyGoal)
    }
    
    var isGoalReached: Bool {
        currentGlasses >= dailyGoal
    }
    
}

What we are going to do is to add Live Activities support to the app. Once implemented, users will be able to see their progress directly on the Lock Screen and in the Dynamic Island. The Live Activity will show the current water intake alongside the daily goal in a clear, simple way.

Creating a Widget Extension for Live Activities

Live Activities are built as part of an app's widget extension, so the first step is to add a widget extension to your Xcode project.

In this demo, the project is called WaterReminder. To create the extension, select the project in Xcode, go to the menu bar, and choose Editor > Target > Add Target. When the template dialog appears, select Widget Extension, give it a name, and make sure to check the Include Live Activity option.

When Xcode asks, be sure to activate the new scheme. It will then generate the widget extension for you, which appears as a new folder in the project navigator along with the starter code for the Live Activity and the widget.

We’ll be rewriting the entire WaterReminderWidgetLiveActivity.swift file from scratch, so it’s best to clear out all of its existing code before proceeding.

Since the Live Activity doesn’t rely on the widget, you can optionally remove the WaterReminderWidget.swift file and update the WaterReminderWidgetBundle struct like this:

struct WaterReminderWidgetBundle: WidgetBundle {
    var body: some Widget {
        WaterReminderWidgetLiveActivity()
    }
}

Defining the ActivityAttributes Structure

The ActivityAttributes protocol describes the content that appears in your Live Activity. We have to adopt the protocol and define the dynamic content of the activity.

Since this attributes structure is usually shared between both the main app and widget extension, I suggest to create a shared folder to host this Swift file. In the project folder, create a new folder named Shared and then create a new Swift file named WaterReminderWidgetAttributes.swift.

Update the content like this:

import Foundation
import ActivityKit

struct WaterReminderWidgetAttributes: ActivityAttributes {
    public struct ContentState: Codable, Hashable {
        var currentGlasses: Int
        var dailyGoal: Int
    }
    
    var activityName: String
}

extension WaterReminderWidgetAttributes {
    static var preview: WaterReminderWidgetAttributes {
        WaterReminderWidgetAttributes(activityName: "Water Reminder")
    }
}

extension WaterReminderWidgetAttributes.ContentState {
     static var sample: WaterReminderWidgetAttributes.ContentState {
        WaterReminderWidgetAttributes.ContentState(currentGlasses: 3, dailyGoal: 8)
     }
     
    static var goalReached: WaterReminderWidgetAttributes.ContentState {
        WaterReminderWidgetAttributes.ContentState(currentGlasses: 8, dailyGoal: 8)
     }
}

The WaterReminderWidgetAttributes struct adopts the ActivityAttributes protocol and includes an activityName property to identify the activity. To conform to the protocol, we define a nested ContentState struct, which holds the data displayed in the Live Activity—specifically, the number of glasses consumed and the daily goal.

The extensions are used for SwiftUI previews, providing sample data for visualization.

Please take note that the target membership of the file should be accessed by both the main app and the widget extension. You can verify it in the file inspector.

Implementing the Live Activity View

Next, let’s implement the live activity view, which handles the user interface in different settings. Open the WaterReminderWidgetLiveActivity.swift file and write the code like below:

import ActivityKit
import WidgetKit
import SwiftUI

struct WaterReminderLiveActivityView: View {
    
    let context: ActivityViewContext<WaterReminderWidgetAttributes>
    
    var body: some View {
        VStack(alignment: .leading, spacing: 10) {
            HStack {
                Text("💧")
                    .font(.title)
                Text("Water Reminder")
                    .font(.headline)
                    .fontWeight(.semibold)
                Spacer()
            }
            
            HStack {
                Text("Current: \(context.state.currentGlasses)")
                    .font(.title2)
                    .fontWeight(.bold)
                Spacer()
                Text("Goal: \(context.state.dailyGoal)")
                    .font(.title2)
            }
            
            // Progress bar
            Gauge(value: Double(context.state.currentGlasses), in: 0...Double(context.state.dailyGoal)) {
                EmptyView()
            }
            .gaugeStyle(.linearCapacity)
        }

    }
}

This view defines the main interface of the Live Activity, which appears on both the Lock Screen and the Dynamic Island. It displays a progress bar to visualize water intake, along with the current number of glasses consumed and the daily goal.

Next, create the WaterReminderWidgetLiveActivity struct like this:

struct WaterReminderWidgetLiveActivity: Widget {
    var body: some WidgetConfiguration {
        ActivityConfiguration(for: WaterReminderWidgetAttributes.self) { context in
            // Lock screen/banner UI goes here
            WaterReminderLiveActivityView(context: context)
                .padding()
        } dynamicIsland: { context in
            DynamicIsland {
                // Expanded UI goes here.  Compose the expanded UI through
                DynamicIslandExpandedRegion(.center) {
                    WaterReminderLiveActivityView(context: context)
                        .padding(.bottom)
                }
            } compactLeading: {
                Text("💧")
                    .font(.title3)
            } compactTrailing: {
                
                if context.state.currentGlasses == context.state.dailyGoal {
                    Image(systemName: "checkmark.circle")
                        .foregroundColor(.green)
                } else {
                    ZStack {
                        Circle()
                            .fill(Color.blue.opacity(0.2))
                            .frame(width: 24, height: 24)
                        
                        Text("\(context.state.dailyGoal - context.state.currentGlasses)")
                            .font(.caption2)
                            .fontWeight(.bold)
                            .foregroundColor(.blue)
                    }
                }

            } minimal: {
                Text("💧")
                    .font(.title2)
            }
        }
    }
}

The code above defines the Live Activity widget configuration for the app. In other words, you configure how the live activity should appear under different configurations.

To keep it simple, we display the same live activity view on the Lock Screen and Dynamic Island.

The dynamicIsland closure specifies how the Live Activity should look inside the Dynamic Island. In the expanded view, the same WaterReminderLiveActivityView is shown in the center region. For the compact view, the leading side displays a water drop emoji, while the trailing side changes dynamically based on the progress: if the daily goal is reached, a green checkmark appears; otherwise, a small circular indicator shows how many glasses are left. In the minimal view, only the water drop emoji is displayed.

Lastly, let’s add some preview code to render the preview of the Live Activity:

#Preview("Notification", as: .content, using: WaterReminderWidgetAttributes.preview) {
   WaterReminderWidgetLiveActivity()
} contentStates: {
    WaterReminderWidgetAttributes.ContentState.sample
    WaterReminderWidgetAttributes.ContentState.goalReached
}

#Preview("Dynamic Island", as: .dynamicIsland(.expanded), using: WaterReminderWidgetAttributes.preview) {
    WaterReminderWidgetLiveActivity()
} contentStates: {
    WaterReminderWidgetAttributes.ContentState(currentGlasses: 3, dailyGoal: 8)
    
    WaterReminderWidgetAttributes.ContentState(currentGlasses: 8, dailyGoal: 8)
}


#Preview("Dynamic Island Compact", as: .dynamicIsland(.compact), using: WaterReminderWidgetAttributes.preview) {
    WaterReminderWidgetLiveActivity()
} contentStates: {
    WaterReminderWidgetAttributes.ContentState(currentGlasses: 5, dailyGoal: 8)
    
    WaterReminderWidgetAttributes.ContentState(currentGlasses: 8, dailyGoal: 8)
}

Xcode lets you preview the Live Activity in different states without needing to run the app on a simulator or a real device. By setting up multiple preview snippets, you can quickly test how the Live Activity will look on both the Lock Screen and the Dynamic Island.

Managing Live Activities

Now that we’ve prepare the view of the live activity, what’s left is to trigger it when the user taps the Add Glass button. To make our code more organized, we will create a helper class called LiveActivityManager to managing the live activity cycle.

import Foundation
import ActivityKit
import SwiftUI

@Observable
class LiveActivityManager {
    private var liveActivity: Activity<WaterReminderWidgetAttributes>?
    
    var isLiveActivityActive: Bool {
        liveActivity != nil
    }
    
    // MARK: - Live Activity Management
    
    func startLiveActivity(currentGlasses: Int, dailyGoal: Int) {
        guard ActivityAuthorizationInfo().areActivitiesEnabled else {
            print("Live Activities are not enabled")
            return
        }
        
        // End any existing activity first
        endLiveActivity()
        
        let attributes = WaterReminderWidgetAttributes(activityName: "Water Reminder")
        let contentState = WaterReminderWidgetAttributes.ContentState(
            currentGlasses: currentGlasses,
            dailyGoal: dailyGoal
        )
        
        do {
            liveActivity = try Activity<WaterReminderWidgetAttributes>.request(
                attributes: attributes,
                content: ActivityContent(state: contentState, staleDate: nil),
                pushType: nil
            )
            print("Live Activity started successfully")
        } catch {
            print("Error starting live activity: \(error)")
        }
    }
    
    func updateLiveActivity(currentGlasses: Int, dailyGoal: Int) {
        guard let liveActivity = liveActivity else { return }
        
        Task {
            let contentState = WaterReminderWidgetAttributes.ContentState(
                currentGlasses: currentGlasses,
                dailyGoal: dailyGoal
            )
            
            await liveActivity.update(ActivityContent(state: contentState, staleDate: nil))
            print("Live Activity updated: \(currentGlasses)/\(dailyGoal)")
        }
    }
    
    func endLiveActivity() {
        guard let liveActivity = liveActivity else { return }
        
        Task {
            await liveActivity.end(nil, dismissalPolicy: .immediate)
            self.liveActivity = nil
            print("Live Activity ended")
        }
    }

}

The code works with WaterReminderWidgetAttributes that we have defined earlier for managing the state of the live activity.

When a new Live Activity starts, the code first checks whether Live Activities are enabled on the device and clears out any duplicates. It then configures the attributes and uses the request method to ask the system to create a new Live Activity.

Updating the Live Activity is straightforward: you simply update the content state of the attributes and call the updatemethod on the Live Activity object.

Finally, the class includes a helper method to end the currently active Live Activity when needed.

Using the Live Activity Manager

With the live activity manager set up, we can now update the WaterTracker class to work with it. First, declare a property to hold the LiveActivityManager object in the class:

let liveActivityManager = LiveActivityManager()

Next, update the addGlass() method like this:

func addGlass() {
    guard currentGlasses < dailyGoal else { return }
    
    currentGlasses += 1
    
    if currentGlasses == 1 {
        liveActivityManager.startLiveActivity(currentGlasses: currentGlasses, dailyGoal: dailyGoal)
    } else {
        liveActivityManager.updateLiveActivity(currentGlasses: currentGlasses, dailyGoal: dailyGoal)
    }
}

When the button is tapped for the first time, we call the startLiveActivity method to start a live activity. For subsequent taps, we simply update the content states of the live activity.

The live activity should be ended when the user taps the reset button. Therefore, update the resetDaily method like below:

func resetDaily() {
    currentGlasses = 0
    
    liveActivityManager.endLiveActivity()
}

That’s it! We’ve completed all the code changes.

Updating Info.plist to Enable Live Activities

Before your app can execute Live Activities, we have to add an entry called Supports Live Activities in the Info.plist file of the main app. Set the value to YES to enable Live Activities.

Great! At this point, you can try out Live Activities either in the simulator or directly on a real device.

Summary

In this tutorial, we explored how to add Live Activities to SwiftUI apps. You've learned how these features boost user engagement by delivering real-time information directly to the Lock Screen and the Dynamic Island, reducing the need for users to reopen your app. We covered the entire process, including creating the data model, designing the user interface, and managing the Live Activity lifecycle. We encourage you to integrate Live Activities into your current or future applications to provide a richer, more convenient user experience.

Have you ever wondered how to make your app’s features accessible from the built-in Shortcuts app on iOS? That’s what the App Intents framework is designed for. Introduced in iOS 16 and macOS Ventura, the framework has been around for over two years. It provides developers with a powerful way to define actions that users can trigger through Shortcuts. With App Intents, your app can integrate seamlessly with the Shortcuts app, Siri, and even system-wide Spotlight search.

In this tutorial, we’ll explore how to use the App Intents framework to bring your app’s functionality into Shortcuts by creating an App Shortcut. Using the Ask Me Anything app as our example, we’ll walk through the process of letting users ask questions right from Shortcuts.

This tutorial assumes you're familiar with the Ask Me Anything app from our Foundation Models tutorial. If you haven't read it yet, please review that tutorial first.

Using App Intents

The Ask Me Anything app allows users to ask questions and then it provides answers using the on-device LLM. What we are going to do is to expose this feature to the Shortcuts app. To do that, all you need to do is to create a new Struct and adopt the App Intents framework.

Let’s create a new file named AskQuestionIntent in the AskMeAnything project and update its content like below:

import SwiftUI
import AppIntents

struct AskQuestionIntent: AppIntent {
    static var title: LocalizedStringResource = "Ask Question"
    static var description = IntentDescription("Ask a question to get an AI-powered answer")
    
    static let supportedModes: IntentModes = .foreground
    
    @Parameter(title: "Question", description: "The question you want to ask")
    var question: String
    
    @AppStorage("incomingQuestion") var storedQuestion: String = ""
    
    init() {}
    
    init(question: String) {
        self.question = question
    }
    
    func perform() async throws -> some IntentResult {
        storedQuestion = question
        
        return .result()
    }
}

The code above defines a struct called AskQuestionIntent, which is an App Intent using the AppIntents framework. An App Intent is basically a way for your app to “talk” to the Shortcuts app, Siri, or Spotlight. Here, the intent’s job is to let a user ask a question and get an AI-powered answer.

At the top, we have two static properties: title and description. These are what the Shortcuts app or Siri will show the user when they look at this intent.

The supportedModes property specifies that this intent can only run in the foreground, meaning the app will open when the shortcut is executed.

The @Parameter property wrapper defines the input the user needs to give. In this case, it's a question string. When someone uses this shortcut, they'll be prompted to type or say this question.

The @AppStorage("incomingQuestion") property is a convenient way to persist the provided question in UserDefaults, making it accessible to other parts of the app.

Finally, the perform() function is where the intent actually does its work. In this example, it just takes the question from the parameter and saves it into storedQuestion. Then it returns a .result() to tell the system it’s done. You’re not doing the AI call directly here — just passing the question into your app so it can handle it however it wants.

Handling the Shortcut

Now that the shortcut is ready, executing the “Ask Question” shortcut will automatically launch the app. To handle this behavior, we need to make a small update to ContentView.

First, declare a variable to retrieve the question provided by the shortcut like this:

@AppStorage("incomingQuestion") private var incomingQuestion: String = ""

Next, attach the onChange modifier to the scroll view:

ScrollView {

...


}
.onChange(of: incomingQuestion) { _, newQuestion in
    if !newQuestion.isEmpty {
        question = newQuestion
        incomingQuestion = ""
        
        Task {
            await generateAnswer()
        }
    }
}

In the code above, we attach an .onChange modifier to the ScrollView so the view can respond whenever the incomingQuestionvalue is updated. Inside the closure, we check whether a new question has been received from the shortcut. If so, we trigger the generateAnswer() method, which sends the question to the on-device LLM for processing and returns an AI-generated answer.

Adding a Preconfigured Shortcut

In essence, this is how you create a shortcut that connects directly to your app. If you’ve explored the Shortcuts app before, you’ve probably noticed that many apps already provide preconfigured shortcuts. For instance, the Calendar app includes ready-made shortcuts for creating and managing events.

With the App Intents framework, adding these preconfigured shortcuts to your own app is straightforward. They can be used right away in the Shortcuts app or triggered hands-free with Siri. Building on the AskQuestionIntent we defined earlier, we can now create a corresponding shortcut so users can trigger it more easily. For example, here’s how we could define an “Ask Question” shortcut:

struct AskQuestionShortcut: AppShortcutsProvider {
    static var appShortcuts: [AppShortcut] {
        AppShortcut(
            intent: AskQuestionIntent(),
            phrases: [
                "Ask \(.applicationName) a question",
                "Ask \(.applicationName) about \(.applicationName)",
                "Get answer from \(.applicationName)",
                "Use \(.applicationName)"
            ],
            shortTitle: "Ask Question",
            systemImageName: "questionmark.bubble"
        )
    }
}

The AskQuestionShortcut adopts the AppShortcutsProvider protocol, which is how we tell the system what shortcuts our app supports. Inside, we define a single shortcut called “Ask Question,” which is tied to our AskQuestionIntent. We also provide a set of example phrases that users might say to Siri, such as “Ask [App Name] a question” or “Get answer from [App Name].”

Finally, we give the shortcut a short title and a system image name so it’s visually recognizable inside the Shortcuts app. Once this code is in place, the system automatically registers it, and users will see the shortcut ready to use—no extra setup required.

Testing the Shortcut

To give the shortcut a try, build and run the app on either the simulator or a physical iOS device. Once the app has launched at least once, return to the Home Screen and open the Shortcuts app. You should now find the “Ask Question” shortcut we just created, ready for you to use.

The new shortcut not only appears in the Shortcuts app but is also available in Spotlight search.

When you run the “Ask Question” shortcut, it should automatically prompt you for question. Once you type your question and tap Done, it brings up the app and show you the answer.

Summary

In this tutorial, we explored how to use the App Intents framework to expose your app’s functionality to the Shortcuts app and Siri. We walked through creating an AppIntent to handle user input, defining a preconfigured shortcut, and testing it right inside the Shortcuts app. With this setup, users can now ask questions to the Ask Me Anything app directly from Shortcuts or via Siri, making the experience faster and more convenient.

In the next tutorial, we’ll take it a step further by showing you how to display the AI’s answer in a Live Activity. This will let users see their responses in real time, right on the Lock Screen or in the Dynamic Island—without even opening the app.

In the previous tutorials, we explored how Foundation Models work in iOS 26 and how you can start building AI-powered features using this new framework. We also introduced the @Generable macro, which makes it easy to convert generated responses into structured Swift types.

Now, in Part 3 of the Foundation Models series, we’ll dive into another powerful capability: Tool Calling — a feature that lets the model interact with your app’s functions to perform tasks, retrieve data, or trigger actions based on user input.

The on-device language model isn’t capable of answering every type of question, especially those that require real-time data, like the current weather or the latest stock prices. In other cases, you might want the model to access your app’s own data to respond accurately. That’s where Tool Calling comes in that it allows the model to delegate specific tasks to your app's functions or external APIs.

In this tutorial, we’ll extend the Ask Me Anything app. While the on-device model can handle general queries, it doesn’t have access to up-to-date information about trending movies. To bridge that gap, we’ll use Tool Calling to integrate with the The Movie Database (TMDB) API, enabling the model to respond to movie-related questions using live data.

Using TMDB APIs

If you ask the Ask Me Anything app about trending movies, the on-device language model won’t have the answer—it simply doesn’t have access to that kind of real-time information and may suggest checking other sources instead. Let’s fix that using Tool Calling and the TMDB API. With this setup, whenever a user asks a movie-related question, the model won’t respond with “I don’t know.” Instead, it will automatically call the external API and return the relevant information directly in the app.

In the Xcode project, create a MovieService file and insert the following code:

// Model for a Movie
struct Movie: Codable, Identifiable {
    let id: Int
    let title: String
    let overview: String
    
    // Coding keys to match API response
    enum CodingKeys: String, CodingKey {
        case id
        case title
        case overview
    }
}

// Model for the API response
struct TrendingMoviesResponse: Codable {
    let results: [Movie]
}

// Service class to fetch trending movies
class MovieService {
    // Base URL for TMDB API
    private let baseURL = "https://api.themoviedb.org/3"
    
    private let apiKey = "<your-api-key>"
    
    // Function to fetch trending movies using async/await
    func fetchTrendingMovies() async throws -> [Movie] {
        
        // Construct the URL for trending movies
        let urlString = "\(baseURL)/trending/movie/day?api_key=\(apiKey)"
        guard let url = URL(string: urlString) else {
            throw URLError(.badURL)
        }
        
        // Perform the network request
        let (data, response) = try await URLSession.shared.data(from: url)
        
        // Check for valid HTTP response
        guard let httpResponse = response as? HTTPURLResponse,
              (200...299).contains(httpResponse.statusCode) else {
            throw URLError(.badServerResponse)
        }
        
        // Decode the JSON response
        let decoder = JSONDecoder()
        let trendingResponse = try decoder.decode(TrendingMoviesResponse.self, from: data)
        return trendingResponse.results
    }
}

Make sure you replace the value of apiKey with your own TMDB API key. If you haven’t signed up yet, head over to themoviedb.org and register for a free account to get your API key.

The code above is fairly straightforward: it calls the web API to fetch trending movies, then parses the response and decodes it into an array of Movie objects.

Using Tool Calling in Foundation Models

Next, we’ll use Tool Calling to trigger the code in MovieService whenever the user asks about trending movies. To get started, create a new file named GetTrendingMoviesTool.swift and add the following code:

import FoundationModels

struct GetTrendingMoviesTool: Tool {
    let name = "getTrendingMovies"
    let description = "Get trending movies and their information"
    
    let service = MovieService()

    @Generable
    struct Arguments {
        
    }
    
    func call(arguments: Arguments) async throws -> [String] {
        let movies = try await service.fetchTrendingMovies()
       
        let formattedMovies = movies.map { movie in
            "\(movie.title): \(movie.overview)"
        }
        
        return formattedMovies
    }
}

We define a GetTrendingMovieTool struct that conforms to the Tool protocol — this is the standard way to implement Tool Calling in the Foundation Models framework. The protocol requires you to specify a name and description for the tool, along with an Arguments struct to represent any parameters the tool might need. In this case, we don’t require additional input, so we define an empty Arguments structure.

If you wanted to filter trending movies by genre, you could define Arguments like this:

@Generable
struct Arguments {
		@Guide(description: "The genre to fetch trending movies")
		var genre: String
}

When the tool is triggered by the model, the call method is automatically executed. Inside it, we call the fetchTrendingMovies() method from our service. After receiving the results, we format them to display each movie’s title and overview.

With the trending movie tool in place, integrating it into your app is straightforward. Simply open ContentView and update the LanguageModelSession initialization as follows:

@State private var session = LanguageModelSession(tools: [GetTrendingMoviesTool()])

You can provide custom tools by passing them through the tools parameter when initializing the language model session. That’s it! The language model will automatically invoke GetTrendingMoviesTool whenever it detects a question related to trending movies.

Build and run the app, then try asking the same question again. This time, the model will successfully respond with trending movie information by invoking the tool.

Summary

In this tutorial, we explored tool calling, a powerful addition to the Foundation Models framework in iOS 26. Unlike basic text generation, tool calling enables the on-device language model to interact with your app’s functions or access external services.

With tool calling, you can significantly extend the model’s capabilities. Whether it’s running custom logic or fetching real-time data through APIs, the model can now perform context-aware tasks beyond its built-in knowledge.

I hope you’ve enjoyed this tutorial series and feel inspired to start building smarter, AI-powered features using the Foundation Models framework.

In the previous tutorial, we introduced the Foundation Models framework and demonstrated how to use it for basic content generation. That process was fairly straightforward — you provide a prompt, wait a few seconds, and receive a response in natural language. In our example, we built a simple Q&A app where users could ask any question, and the app displayed the generated text directly.

But what if the response is more complex — and you need to convert the unstructured text into a structured object?

For example, suppose you ask the model to generate a recipe, and you want to turn that response into a Recipe object with properties like name, ingredients, and instructions.

Do you need to manually parse the text and map each part to your data model?

The Foundation Models framework in iOS 26 provides two powerful new macros called Generable and @Guide to help developers simplify this process.

In this tutorial, we’ll explore how these macros work and how you can use them to generate structured data directly from model output.

The Demo App

We will build a simple Quiz app that demonstrates how to use Foundation Models to generate structured content. In this case, it’s the vocabulary questions for English learners.

The app displays a multiple-choice question with four answer options, allowing users to test their knowledge interactively. Each question is generated by the on-device language model and automatically parsed into a Swift struct using the @Generable macro.

This demo app shows how developers can move beyond basic text generation and use Foundation Models to create structured content.

Using @Generable and @Guide

Let’s get started with building the demo app. As said before, unlike the previous Q&A demo, this quiz app presents a multiple-choice question with several answer options. To represent the question, we’ll define the following structure in Swift:

struct Question {
    let text: String
    let choices: [String]
    let answer: String
    let explanation: String
}

Later, we will ask the on-device language model to generate quiz questions. The challenge is how we can convert the model’s unstructured text response into a usable Question object. Fortunately, the Foundation Models framework introduces the @Generable macro to simplify the conversion process.

To enable automatic conversion, simply mark your struct with @Generable, like this:

import FoundationModels

@Generable
struct Question {
    @Guide(description: "The quiz question")
    let text: String
    @Guide(.count(4))
    let choices: [String]
    let answer: String
    @Guide(description: "A brief explanation of why the answer is correct.")
    let explanation: String
}

The framework also introduces the @Guide macro, which allows developers to provide specific instructions to the language model when generating properties. For instance, to specify that each question should have exactly 4 choices, you can use @Guide(.count(4)) on the choices array property.

With array, other than controlling the exact number of element, you can also use the following guides:

.minimumCount(3)
.maximumCount(100)

You can also add a descriptive explanation to a property to give the language model more context about the kind of data it should generate. This helps ensure the output is more accurate and aligned with your expectations.

It’s important to pay attention to the order in which properties are declared. When using a Generable type, the language model generates values sequentially based on the order of the properties in your code. This becomes especially important when one property's value relies on another. For example, in the code above, the explanation property depends on the answer, so it should be declared after the answer to ensure it references the correct context.

Building the Quiz App

With the Question structure ready, we dive into the implementation of the Quiz app. Switch back to ContentView and update the code like this:

import FoundationModels

struct ContentView: View {
    
    @State private var session = LanguageModelSession(instructions: "You are a high school English teacher.")
    
    @State private var question: Question?
    
    var body: some View {
        VStack(spacing: 20) {
            
            if let question {
                QuestionView(question: question)
            } else {
                ProgressView("Generating questions ...")
            }
            
            Spacer()
            
            Button("Next Question") {
                Task {
                    do {
                        question = nil
                        question = try await generateQuestion()
                    } catch {
                        print(error)
                    }
                }
            }
            .padding()
            .frame(maxWidth: .infinity)
            .background(Color.green.opacity(0.18))
            .foregroundStyle(.green)
            .font(.headline)
            .cornerRadius(10)

        }
        .padding(.horizontal)
        .task {
            do {
                question = try await generateQuestion()
            } catch {
                print(error)
            }
        }
    }
    
    func generateQuestion() async throws -> Question {
        
        let response = try await session.respond(to: "Create a vocabulary quiz for high school students. Generate one multiple-choice question that tests vocabulary knowledge.", generating: Question.self)
        
        return response.content
    }
}

The user interface code for this app is simple and easy to follow. What’s worth highlighting, however, is how we integrate the Foundation Models framework to generate quiz questions. In the example above, we create a LanguageModelSession and provide it with a clear instruction, asking the language model to take on the role of an English teacher.

To generate a question, we use the session’s respond method and specify the expected response type using the generatingparameter. The session then automatically produces a response and maps the result into a Question object, saving you from having to parse and structure the data manually.

Next, we’ll implement the QuestionView, which is responsible for displaying the generated quiz question, handling user interaction, and verifying the selected answer. Add the following view definition inside your ContentView file:

struct QuestionView: View {
    let question: Question
    
    @State private var selectedAnswer: String? = nil
    @State private var didAnswer: Bool = false

    var body: some View {
        ScrollView {
            VStack(alignment: .leading) {
                Text(question.text)
                    .font(.title)
                    .fontWeight(.semibold)
                    .padding(.vertical)
                
                VStack(spacing: 12) {
                    ForEach(question.choices, id: \.self) { choice in
                        
                        Button {
                            if !didAnswer {
                                selectedAnswer = choice
                                didAnswer = true
                            }

                        } label: {
                            if !didAnswer {
                                Text(choice)
                            } else {
                                HStack {
                                    if choice == question.answer {
                                        Text("✅")
                                    } else if selectedAnswer == choice {
                                        Text("❌")
                                    }
                                    
                                    Text(choice)
                                }
                            }
                        }
                        .disabled(didAnswer)
                        .padding()
                        .frame(maxWidth: .infinity)
                        .background(
                            Color.blue.opacity(0.15)
                        )
                        .foregroundStyle(.blue)
                        .font(.title3)
                        .cornerRadius(12)

                    }
                }
                
                if didAnswer {
                    
                    VStack(alignment: .leading, spacing: 10) {
                        Text("The correct answer is \(question.answer)")
                        
                        Text(question.explanation)
                    }
                    .font(.title3)
                    .padding(.top)
                }
            }
            

        }
    }
    
}

This view presents the question text at the top, followed by four answer choices rendered as tappable buttons. When the user selects an answer, the view checks if it’s correct and displays visual feedback using emojis (✅ or ❌). Once answered, the correct answer and an explanation are shown below. The @State properties track the selected answer and whether the question has been answered, allowing the UI to update reactively.

Once you've implemented all the necessary changes, you can test the app in the Preview canvas. You should see a generated vocabulary question like the one shown below, complete with four answer choices. After selecting an answer, the app provides immediate visual feedback and an explanation.

Summary

In this tutorial, we explored how to use the Foundation Models framework in iOS 26 to generate structured content with Swift. By building a simple vocabulary quiz app, we demonstrated how the new @Generable and @Guide macros can turn unstructured language model responses into typed Swift structs.

Stay tuned — in the next tutorial, we’ll dive into another powerful feature of the Foundation Models framework.

With iOS 26, Apple introduces the Foundation Models framework, a privacy-first, on-device AI toolkit that brings the same language models behind Apple Intelligence right into your apps. This framework is available across Apple platforms, including iOS, macOS, iPadOS, and visionOS, and it provides developers with a streamlined Swift API for integrating advanced AI features directly into your apps.

Unlike cloud-based LLMs such as ChatGPT or Claude, which run on powerful servers and require internet access, Apple’s LLM is designed to run entirely on-device. This architectural difference gives it a unique advantage: all data stays on the user’s device, ensuring privacy, lower latency, and offline access.

This framework opens the door to a whole range of intelligent features you can build right out of the box. You can generate and summarize content, classify information, or even build in semantic search and personalized learning experiences. Whether you want to create a smart in-app guide, generate unique content for each user, or add a conversational assistant, you can now do it with just a few lines of Swift code.

In this tutorial, we’ll explore the Foundation Models framework. You’ll learn what it is, how it works, and how to use it to generate content using Apple’s on-device language models.

To follow along, make sure you have Xcode 26 installed, and that your Mac is running macOS Tahoe, which is required to access the Foundation Models framework.

Ready to get started? Let’s dive in.

The Demo App: Ask Me Anything

It’s always great to learn new frameworks or APIs by building a demo app — and that’s exactly what we’ll do in this tutorial. We’ll create a simple yet powerful app called Ask Me Anything to explore how Apple’s new Foundation Models framework works in iOS 26.

The app lets users type in any questions and provides an AI-generated response, all processed on-device using Apple’s built-in LLM.

By building this demo app, you'll learn how to integrate the Foundation Models framework into a SwiftUI app. You'll also understand how to create prompts and capture both full and partial generated responses.

Using the Default System Language Model

Apple provides a built-in model called SystemLanguageModel, which gives you access to the on-device foundation model that powers Apple Intelligence. For general-purpose use, you can access the base version of this model via the default property. It’s optimized for text generation tasks and serves as a great starting point for building features like content generation or question answering in your app.

To use it in your app, you'll first need to import the FoundationModels framework:

import FoundationModels

With the framework now imported, you can get a handle on the default system language model. Here’s the sample code to do that:

struct ContentView: View {
    
    private var model = SystemLanguageModel.default
    
    var body: some View {
        switch model.availability {
        case .available:
            mainView
        case .unavailable(let reason):
            Text(unavailableMessage(reason))
        }
    }
    
    private var mainView: some View {
        ScrollView {
            .
            .
            .
        }
    }

    private func unavailableMessage(_ reason: SystemLanguageModel.Availability.UnavailableReason) -> String {
        switch reason {
        case .deviceNotEligible:
            return "The device is not eligible for using Apple Intelligence."
        case .appleIntelligenceNotEnabled:
            return "Apple Intelligence is not enabled on this device."
        case .modelNotReady:
            return "The model isn't ready because it's downloading or because of other system reasons."
        @unknown default:
            return "The model is unavailable for an unknown reason."
        }
    }
}

Since Foundation Models only work on devices with Apple Intelligence enabled, it's important to verify that a model is available before using it. You can check its readiness by inspecting the availability property.

Implementing the UI

Let’s continue to build the UI of the mainView. We first add two state variables to store the user question and the generated answer:

@State private var answer: String = ""
@State private var question: String = ""

For the UI implementation, update the mainView like this:

private var mainView: some View {
    ScrollView {
        ScrollView {
            VStack {
                Text("Ask Me Anything")
                    .font(.system(.largeTitle, design: .rounded, weight: .bold))
                
                TextField("", text: $question, prompt: Text("Type your question here"), axis: .vertical)
                    .lineLimit(3...5)
                    .padding()
                    .background {
                        Color(.systemGray6)
                    }
                    .font(.system(.title2, design: .rounded))
                
                Button {

                } label: {
                    Text("Get answer")
                        .frame(maxWidth: .infinity)
                        .font(.headline)
                }
                .buttonStyle(.borderedProminent)
                .controlSize(.extraLarge)
                .padding(.top)
                
                Rectangle()
                    .frame(height: 1)
                    .foregroundColor(Color(.systemGray5))
                    .padding(.vertical)
                
                Text(LocalizedStringKey(answer))
                    .font(.system(.body, design: .rounded))
            }
            .padding()
        }

    }
}

The implementation is pretty straightforward - I just added a touch of basic styling to the text field and button.

Generating Responses with the Language Model

Now we’ve come to the core part of app: sending the question to the model and generating the response. To handle this, we create a new function called generateAnswer():

private func generateAnswer() async {
    let session = LanguageModelSession()
    do {
        let response = try await session.respond(to: question)
        answer = response.content
    } catch {
        answer = "Failed to answer the question: \(error.localizedDescription)"
    }
}

As you can see, it only takes a few lines of code to send a question to the model and receive a generated response. First, we create a session using the default system language model. Then, we pass the user’s question, which is known as a prompt, to the model using the respond method.

The call is asynchronous as it usually takes a few second (or even longer) for the model to generate the response. Once the response is ready, we can access the generated text through the content property and assign it to answer for display.

To invoke this new function, we also need to update the closure of the “Get Answer” button like this:

Button {
    Task {
        await generateAnswer()
    }
} label: {
    Text("Show me the answer")
        .frame(maxWidth: .infinity)
        .font(.headline)
}

You can test the app directly in the preview pane, or run it in the simulator. Just type in a question, wait a few seconds, and the app will generate a response for you.

Reusing the Session

The code above creates a new session for each question, which works well when the questions are unrelated.

But what if you want users to ask follow-up questions and keep the context? In that case, you can simply reuse the same session each time you call the model.

For our demo app, we can move the session variable out of the generateAnswer() function and turn it into a state variable:

@State private var session = LanguageModelSession()

After making the change, try testing the app by first asking: “What are the must-try foods when visiting Japan?” Then follow up with: “Suggest me some restaurants.”

Since the session is retained, the model understands the context and knows you're looking for restaurant recommendations in Japan.

If you don’t reuse the same session, the model won’t recognize the context of your follow-up question. Instead, it will respond with something like this, asking for more details:

“Sure! To provide you with the best suggestions, could you please let me know your location or the type of cuisine you're interested in?”

Disabling the Button During Response Generation

Since the model takes time to generate a response, it’s a good idea to disable the "Get Answer" button while waiting for the answer. The session object includes a property called isResponding that lets you check if the model is currently working.

To disable the button during that time, simply use the .disabled modifier and pass in the session’s status like this:

Button {
    Task {
        await generateAnswer()
    }
} label: {
    .
    .
    .
}
.disabled(session.isResponding)

Working with Stream Responses

The current user experience isn't ideal — since the on-device model takes time to generate a response, the app only shows the result after the entire response is ready.

If you’ve used ChatGPT or similar LLMs, you’ve probably noticed that they start displaying partial results almost immediately. This creates a smoother, more responsive experience.

The Foundation Models framework also supports streaming output, which allows you to display responses as they're being generated, rather than waiting for the complete answer. To implement this, use the streamResponse method rather than the respond method. Here's the updated generateAnswer() function that works with streaming responses:

private func generateAnswer() async {
    
    do {
        answer = ""
        let stream = session.streamResponse(to: question)
        for try await streamData in stream {             
		        answer = streamData.asPartiallyGenerated()
        }
    } catch {
        answer = "Failed to answer the question: \(error.localizedDescription)"
    }
}

Just like with the respond method, you pass the user's question to the model when calling streamResponse. The key difference is that instead of waiting for the full response, you can loop through the streamed data and update the answer variable with each partial result — displaying it on screen as it’s generated.

Now when you test the app again and ask any questions, you'll see responses appear incrementally as they're generated, creating a much more responsive user experience.

Customizing the Model with Instructions

When instantiating the model session, you can provide optional instructions to customize its use case. For the demo app, we haven't provided any instructions during initialization because this app is designed to answer any question.

However, if you're building a Q&A system for specific topics, you may want to customize the model with targeted instructions. For example, if your app is designed to answer travel-related questions, you could provide the following instruction to the model:

“You are a knowledgeable and friendly travel expert. Your job is to help users by answering travel-related questions clearly and accurately. Focus on providing useful advice, tips, and information about destinations, local culture, transportation, food, and travel planning. Keep your tone conversational, helpful, and easy to understand, as if you're speaking to someone planning their next trip.”

When writing instructions, you can define the model’s role (e.g., travel expert), specify the focus of its responses, and even set the desired tone or style.

To pass the instruction to the model, you can instantiate the session object like this:

var session = LanguageModelSession(instructions: "your instruction")

Summary

In this tutorial, we covered the basics of the Foundation Models framework and showed how to use Apple’s on-device language model for tasks like question answering and content generation.

This is just the beginning — the framework offers much more. In future tutorials, we’ll dive deeper into other new features such as the new @Generable and @Guide macros, and explore additional capabilities like content tagging and tool calling.

If you're looking to build smarter, AI-powered apps, now is the perfect time to explore the Foundation Models framework and start integrating on-device intelligence into your projects.

In iOS 26, SwiftUI finally introduced one of its most highly anticipated components: WebView, a native solution for displaying web content. Before this update, SwiftUI developers had to rely on the UIKit framework, using UIViewRepresentable to wrap WKWebView or SFSafariViewController in order to embed a web view. With the arrival of WebView, Apple now provides a fully native SwiftUI approach to integrating web browsing capabilities into apps. In this tutorial, I’ll give you a quick overview of the new WebView and show you how to use it in your own app development.

The Basic Usage of WebView

To load a web page using the new WebView, you simply import the WebKit framework and instantiate the view with a URL. Here is an example:

import SwiftUI
import WebKit

struct ContentView: View {
    var body: some View {
        WebView(url: URL(string: "https://www.appcoda.com"))
    }
}

With just a single line of code, you can now embed a full-featured mobile Safari experience directly in your app—powered by the same WebKit engine that runs Safari.

An Alternative Way of Loading Web Content

In addition to WebView, the WebKit framework also introduces a new class called WebPage. Rather than passing a URL directly to WebView, you can first create a WebPage instance with the URL and then use it to display the web content. Below is the sample code that achieves the same result:

struct ContentView: View {
    @State private var page = WebPage()
    
    var body: some View {
        
        WebView(page)
            .ignoresSafeArea()
            .onAppear {
                if let pageURL = URL(string: "https://www.appcoda.com") {
                    let urlRequest = URLRequest(url: pageURL)
                    page.load(urlRequest)
                }
            }
    }
}

Working with WebPage

In most cases, if you simply need to display web content or embed a browser in your app, WebView is the most straightforward approach. If you need finer control over how web content behaves and interacts with your application, WebPage offers more detailed customization options like accessing web page properties and programmatic navigation.

For example, you can access the title property of the WebPage object to retrieve the title of the web page:

Text(page.title)
Text(page.url)

You can also use the url property to access the current URL of the web page.

If you want to track the loading progress, the estimatedProgress property gives you an approximate percentage of the page’s loading completion.

Text(page.estimatedProgress.formatted(.percent.precision(.fractionLength(0))))

Other than accessing its properties, the WebPage class also lets you control the loading behavior of a web page. For example, you can call reload() to refresh the current page, or stopLoading() to halt the loading process.

Loading Custom HTML Content Using WebPage

Besides loading a URLRequest, the WebPage class's load method can also handle custom HTML content directly. Below is the sample code for loading a YouTuber player:

struct ContentView: View {
    
    @State private var page = WebPage()
    
    private let htmlContent: String = """
        <div class="videoFrame">
        <iframe width="960" height="540" src="https://www.youtube.com/embed/0_DjDdfqtUE?si=iYAmkvDghnGaVcAC" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
        </div>
        """
    var body: some View {
        
        WebView(page)
            .onAppear {
                page.load(html: htmlContent, baseURL: URL(string: "about:blank")!)
            }
    }
}

If you place the code in Xcode, the preview should show you the YouTube player.

Executing Javascript

The WebPage object not only lets you load HTML content—it also allows you to execute JavaScript. You can use the callJavaScript method and pass in the script you want to run. Here is an example:

struct ContentView: View {
    
    @State private var page = WebPage()
    
    private let snippet = """
        document.write("<h1>This text is generated by Javascript</h1>");
        """
    
    var body: some View {
        
        WebView(page)
            .task {
                do {
                    try await page.callJavaScript(snippet)
                } catch {
                    print("JavaScript execution failed: \(error)")
                }
            }
    }
}

Summary

The new native WebView component in SwiftUI makes it much easier to display web content within iOS apps, removing the need to rely on UIKit wrappers. SwiftUI developers can choose between two key approaches:

• WebView: Ideal for straightforward use cases where you just need to load and display a web page.
• WebPage: Offers more granular control, allowing you to access page properties, track loading progress, reload or stop loading, and even execute JavaScript.

This native SwiftUI solution brings a cleaner, more streamlined experience to embedding web content in your apps.

We've covered iOS localization in several tutorials, including one that shows how to fully localize an app using String Catalogs. However, these tutorials rely on the system language to determine the app’s language. But what if you want to give users the ability to choose their preferred language, regardless of the system setting? And what if you want the language to update instantly—without restarting the app? That’s exactly what this tutorial will teach you.

Before we get started, I recommend reviewing the earlier iOS localization tutorial if you're not familiar with String Catalogs. The demo app used in this tutorial builds on the one from that guide.

The Demo App

We're reusing the demo app from our iOS localization tutorial—a simple app with basic UI elements to illustrate localization concepts. In this tutorial, we'll extend it by adding a Settings screen that lets users select their preferred language. The app will then update the language instantly, with no need to restart.

Adding App Languages and App Settings

Before we start building the Setting screen, let’s first add an AppLanguage enum and an AppSetting class to the project. The AppLanguage enum defines the set of languages that your app supports. Here is the code:

enum AppLanguage: String, CaseIterable, Identifiable {
    case en, fr, jp, ko, zhHans = "zh-Hans", zhHant = "zh-Hant"
    
    var id: String { rawValue }
    
    var displayName: String {
        switch self {
        case .en: return "English"
        case .fr: return "French"
        case .jp: return "Japanese"
        case .ko: return "Korean"
        case .zhHans: return "Simplified Chinese"
        case .zhHant: return "Traditional Chinese"
        }
    }
}

Each case in the enum corresponds to a specific language, using standard locale identifiers as raw values. For example, .en maps to "en" for English, .fr to "fr" for French, and so on. The displayName computed property provides a user-friendly label for each language. Instead of displaying raw locale codes like "en" or "zh-Hans" in the UI, this property returns readable names such as "English" or "Simplified Chinese."

The AppSetting class, which conforms to the ObservableObject protocol, is a simple observable model that stores the user’s selected language. Here is the code:

class AppSetting: ObservableObject {
    @Published var language: AppLanguage = .en
}

By default, the language is set to English. Later, when the user selects a different language from the Settings screen, updating this property will cause SwiftUI views that rely on the app’s locale to re-render using the new language.

Building the Setting Screen

Next, let’s build the Settings screen. It’s a simple interface that displays a list of all the supported languages. Below is the code for implementing the setting view:

struct SettingView: View {
    
    @Environment(\.dismiss) var dismiss
    
    @EnvironmentObject var appSetting: AppSetting
    
    @State private var selectedLanguage: AppLanguage = .en
    
    var body: some View {
        NavigationStack {
            Form {
                Section(header: Text("Language")) {
                    ForEach(AppLanguage.allCases) { lang in
                        
                        HStack {
                            Text(lang.displayName)
                            
                            Spacer()
                            
                            if lang == selectedLanguage {
                                Image(systemName: "checkmark")
                                    .foregroundColor(.primary)
                            }
                                
                        }
                        .onTapGesture {
                            selectedLanguage = lang
                        }
                    }
                }
            }
            
            .toolbar {
                ToolbarItem(placement: .topBarTrailing) {
                    Button("Save") {
                        appSetting.language = selectedLanguage
                        dismiss()
                    }
                }

                ToolbarItem(placement: .topBarLeading) {
                    Button("Cancel") {
                        dismiss()
                    }
                }
            }
            .navigationTitle("Settings")
            .navigationBarTitleDisplayMode(.inline)
            
        }
        .onAppear {
            selectedLanguage = appSetting.language
        }
    }
}

#Preview {
    SettingView()
        .environmentObject(AppSetting())
}

The view simply lists the available languages as defined in AppLanguage. The currently selected language shows a checkmark next to it. When the user taps "Save," the selected language is saved to the shared AppSetting object, and the view is dismissed.

In the main view, we add a Setting button and use the .sheet modifier to display the Setting view.

struct ContentView: View {
    
    @EnvironmentObject var appSetting: AppSetting
    
    @State private var showSetting: Bool = false
    
    var body: some View {
        VStack {
            
            HStack {
                Spacer()
                
                Button {
                    showSetting.toggle()
                } label: {
                    Image(systemName: "gear")
                        .font(.system(size: 30))
                        .tint(.primary)
                }

                
            }
                
            Text("ProLingo")
                .font(.system(size: 75, weight: .black, design: .rounded))
            
            Text("Learn programming languages by working on real projects")
                .font(.headline)
                .padding(.horizontal)
              
           .
           .
           .
           .
           .
           .
            
        }
        .padding()
        .sheet(isPresented: $showSetting) {
            SettingView()
                .environmentObject(appSetting)
        }

    }
}

Enabling Real-Time Language Changes

At this point, tapping the gear button will bring up the Settings view. However, the app doesn't update its language when the user selects their preferred language. To implement dynamic language switching, we have to attach the .environment modifier to ContentView and update the locale to match the user’s selection like this:

VStack {
   ...
}
.environment(\.locale, Locale(identifier: appSetting.language.id))

This line of code injects a custom Locale into the SwiftUI environment. The \.locale key controls which language and region SwiftUI uses for localizable views like Text. The locale is set to match the language the user selected in settings.

The app can now update its language on the fly. For example, open the Settings view and select Traditional Chinese. After saving your selection and returning to the main view, you'll see the UI instantly updated to display all text in Traditional Chinese.

Using LocalizedStringKey

You may notice a bug in the app. After changing the language to Traditional Chinese (or other languages) and reopening the Settings view, the language names still display in English.

Let’s take a look at the code that handles the display of language name:

Text(lang.displayName)

You may wonder why the Text view doesn’t handle the localization automatically. In this case, SwiftUI treats lang.displayName as a plain text, which means no automatic localization happens, even if the string matches a key in the String Catalog file. To make the localization work, you need to convert the String to a LocalizedStringKey like this:

Text(LocalizedStringKey(lang.displayName))

Using LocalizedStringKey triggers the localization lookup process. When you run the app again, you'll see the language names in the Settings view displayed in your chosen language.

Summary

In this tutorial, you learned how to implement in-app language switching in iOS using SwiftUI, allowing users to change languages without restarting the app. We explored how to create a Settings screen for language selection, enabled real-time localization updates, and learned the importance of using LocalizedStringKey for proper string localization.

The code and concepts presented here provide a foundation for implementing language switching in your own iOS apps. Feel free to adapt this approach for your own iOS apps that require multi-language support.

With the release of iOS 18, Apple has unveiled a suite of exciting features under the Apple Intelligence umbrella, and one standout is the ImagePlayground framework. This powerful API empowers developers to generate images from text descriptions using AI, opening up a world of creative possibilities for iOS apps. Whether you’re building a design tool, a storytelling app, or just want to add some flair to your UI, ImagePlayground makes it seamless to integrate AI-driven image generation.

In this tutorial, we’ll walk you through building a simple app using SwiftUI and the ImagePlayground framework. Our app will let users type a description—like “a serene beach at sunset”—and generate a corresponding image with a tap. Designed for developers with some iOS experience, this guide assumes you’re familiar with Swift, SwiftUI, and Xcode basics. Ready to dive into iOS 18’s image generation capabilities?

Let’s get started!

Prerequisites

Before we get started, make sure you’ve got a few things ready:

• Device: Image Playground is supported on iPhone 15 Pro, iPhone 15 Pro Max, and all iPhone 16 models.
• iOS Version: Your device must be running iOS 18.1 or later.
• Xcode: You’ll need Xcode 16 or later to build the app.
• Apple Intelligence: Ensure that Apple Intelligence is enabled on your device. You can check this in Settings > Apple Intelligence & Siri. If prompted, request access to Apple Intelligence features.

Setting up the Xcode Project

First, let’s begin by creating a new Xcode project named AIImageGeneration using the iOS app template. Make sure you choose SwiftUI as the UI framework. Also, the minimum deployment version is set to 18.1 (or later). The ImagePlayground framework is only available on iOS 18.1 or up.

Using ImagePlaygroundSheet

Have you tried Image Playground app in iOS 18 before? The app leverages Apple Intelligence to create images based on user inputs, such as text descriptions. While Playground is an independent app on iOS, developers can bring this functionality into your apps using ImagePlaygroundSheet, a SwiftUI view modifier that presents the image generation interface.

Let’s switch over to the Xcode project and see how the sheet works. In the ContentView.swift file, add the following import statement:

import ImagePlayground

The ImagePlaygroundSheet view is included in the ImagePlayground framework. For the ContentView struct, update it like below:

struct ContentView: View {
    @Environment(\.supportsImagePlayground) private var supportsImagePlayground
    
    @State private var showImagePlayground: Bool = false
    
    @State private var generatedImageURL: URL?
    
    var body: some View {
        if supportsImagePlayground {
            
            if let generatedImageURL {
                AsyncImage(url: generatedImageURL) { image in
                    image
                        .resizable()
                        .scaledToFill()
                } placeholder: {
                    Color.purple.opacity(0.1)
                }
                .padding()
            }

            Button {
                showImagePlayground.toggle()
            } label: {
                Text("Generate images")
            }
            .buttonStyle(.borderedProminent)
            .controlSize(.large)
            .tint(.purple)
            .imagePlaygroundSheet(isPresented: $showImagePlayground) { url in
                generatedImageURL = url
            }
            .padding()

        } else {
            ContentUnavailableView("Not Supported", systemImage: "exclamationmark.triangle", description: Text("This device does not support Image Playground. Please use a device that supports Image Playground to view this example."))
        }
    }
}

Not all iOS devices have Apple Intelligence enabled. That’s why it’s important to do a basic check to see if ImagePlayground is supported on the device. The supportsImagePlayground property uses SwiftUI’s environment system to check if the device can use Image Playground. If the device doesn’t support it, we simply show a “Not Supported” message on the screen.

For devices that do support it, the demo app displays a “Generate Images” button. The easiest way to add Image Playground to your app is by using the imagePlaygroundSheet modifier. We use the showImagePlayground property to open or close the playground sheet. After the user creates an image in Image Playground, the system saves the image file in a temporary location and gives back the image URL. This URL is then assigned to the generatedImageURL variable.

With the image URL ready, we use the AsyncImage view to display the image on the screen.

Run the app and test it on your iPhone. Tap the “Generate Image” button to open the Image Playground sheet. Enter a description for the image, and let Apple Intelligence create it for you. Once it’s done, close the sheet, and the generated image should appear in the app.

Working with Concepts

Previously, I showed you the basic way of using the imagePlaygroundSheet modifier. The modifier provides a number of parameters for developers to customize the integration. For example, we can create our own text field to capture the description of the image.

In ContentView, update the code like below:

struct ContentView: View {
    @Environment(\.supportsImagePlayground) private var supportsImagePlayground
    
    @State private var showImagePlayground: Bool = false
    
    @State private var generatedImageURL: URL?
    @State private var description: String = ""
    
    var body: some View {
        if supportsImagePlayground {
            
            if let generatedImageURL {
                AsyncImage(url: generatedImageURL) { image in
                    image
                        .resizable()
                        .scaledToFill()
                } placeholder: {
                    Color.purple.opacity(0.1)
                }
                .padding()
            } else {
                Text("Type your image description to create an image...")
                    .font(.system(.title, design: .rounded, weight: .medium))
                    .multilineTextAlignment(.center)
                    .frame(maxWidth: .infinity, maxHeight: .infinity)
            }

            Spacer()
            
            HStack {
                TextField("Enter your text...", text: $description)
                    .padding()
                    .background(
                        RoundedRectangle(cornerRadius: 12)
                            .fill(.white)
                    )
                    .overlay(
                        RoundedRectangle(cornerRadius: 12)
                            .stroke(Color.gray.opacity(0.2), lineWidth: 1)
                    )
                    .font(.system(size: 16, weight: .regular, design: .rounded))
                
                Button {
                    showImagePlayground.toggle()
                } label: {
                    Text("Generate images")
                }
                .buttonStyle(.borderedProminent)
                .controlSize(.regular)
                .tint(.purple)
                .imagePlaygroundSheet(isPresented: $showImagePlayground,
                                      concept: description
                    ) { url in
                    generatedImageURL = url
                }
                .padding()
            }
            .padding(.horizontal)

        } else {
            ContentUnavailableView("Not Supported", systemImage: "exclamationmark.triangle", description: Text("This device does not support Image Playground. Please use a device that supports Image Playground to view this example."))
        }
    }
}

We added a new text field where users can directly enter an image description. The imagePlaygroundSheet modifier has been updated with a new parameter called concept. This parameter accepts the image description and passes it to the creation UI to generate the image.

.imagePlaygroundSheet(isPresented: $showImagePlayground,
                      concept: description
) { url in
      generatedImageURL = url
}

The concept parameter works best for short descriptions. If you want to allow users to input a longer paragraph, it’s better to use the concepts parameter, which takes an array of ImagePlaygroundConcept. Below is an example of how the code can be rewritten using the concepts parameter:

.imagePlaygroundSheet(isPresented: $showImagePlayground,
                      concepts: [ .text(description) ]
) { url in
      generatedImageURL = url
}

The text function creates a playground concept by processing a short description of the image. For longer text, you can use the extracted(from:title:) API, which lets the system analyze the text and extract key concepts to guide the image creation process.

Adding a Source Image

The imagePlaygroundSheet modifier also supports adding a source image, which acts as the starting point for image generation. Here is an example:

.imagePlaygroundSheet(isPresented: $showImagePlayground,
                      concepts: [.text(description)],
                      sourceImage: Image("gorilla")
    ) { url in
    generatedImageURL = url
}
.padding()

You can either use the sourceImage or sourceImageURL parameter to embed the image.

Summary

In this tutorial, we explored the potential of the ImagePlayground framework in iOS 18, showcasing how developers can harness its AI-driven image generation capabilities to create dynamic and visually engaging experiences. By combining the power of SwiftUI with ImagePlayground, we demonstrated how simple it is to turn text descriptions into stunning visuals.

Now it’s your turn to explore this innovative framework and unlock its full potential in your own projects. I'm eager to see what new AI-related frameworks Apple will introduce next!

We're thrilled to announce that our Mastering SwiftUI ebook has been fully updated for iOS 18 and Xcode 16. This comprehensive update includes:

• All content and source code now compatible with the latest iOS and Xcode versions
• Brand new chapters covering iOS 18's new APIs
• Learn to implement translation features, create stunning animated text effects, master hero animations, and much more!

Our SwiftUI book caters to both beginners and intermediate developers who are eager to learn the ins and outs of the new SwiftUI framework. Each chapter in our book features a minimum of one simple project, allowing you to gain hands-on experience. By working on these projects, you will understand how to work with various types of UI elements and build interactive UIs, plus learn the new APIs coming with the latest version of SwiftUI.

Once you have grasped the fundamentals, you will delve into building a personal finance app using SwiftUI. All the projects and accompanying source code can be downloaded, serving as valuable references. Feel free to incorporate the code into your own projects—whether personal or commercial.

Bonus: Get the Photo Translator App for Free

Our professional package is your all-in-one solution for mastering iOS app development. It includes four comprehensive Swift & SwiftUI books, along with additional app templates to enhance your learning experience. For a limited time until October 17th, you'll also receive the Photo Translator project as a bonus. This cutting-edge app showcases iOS 18's new Translator framework and Text Recognition APIs, providing you with hands-on experience in implementing these advanced features using Swift and SwiftUI.

To purchase the Mastering SwiftUI book, visit the official page here.

Apple's engineers may have long recognized the widespread desire among iOS developers to recreate the elegant hero animation featured in the App Store app. Understanding the complexity and time investment typically required to implement such animations from scratch, Apple has incorporated this feature into the iOS 18 SDK.

With this update, you can now achieve a similar animated transition effect in your own apps using just a few lines of code. This significant enhancement empowers developers to create visually appealing and seamless transitions, elevating the overall user experience of their apps.

In this tutorial, we'll explore how to leverage the new NavigationTransition protocol and the matchedTransitionSource modifier to create hero animations during view transitions.

The Simple Demo App

Let's dive into a demo app to explore the new APIs. We'll begin with a simple app that shows a list of cafes in a standard scroll view. Our goal is to implement a feature where tapping on a cafe takes you to a new screen displaying a full image with a hero animation.

Using Navigation Transition Protocol

To display a full image and animate the view transition using hero animations, you can use the following steps:

1. Embed the scroll view within a navigation stack.
2. Use NavigationLink to enable tapping on the card view.
3. Declare a namespace with @Namespace to support the hero animation.
4. Attach the matchedTransitionSource modifier to the excerpt mode card view.
5. Attach the navigationTransition modifier to the full content mode card view.

By completing these steps, SwiftUI will automatically generate a smooth hero animation, expanding the selected cafe item into a full-screen image when tapped.

Creating Hero Animations for View Transitions

Now we'll modify the project to support navigation. To start, let's embed the scroll view within a navigation stack, as shown below:

NavigationStack {
	ScrollView {
	
		// Existing code
		
	}
}

Next, create the detail view for displaying the full image like below. It accepts a cafe object as input and displays its image in a full-screen view.

struct DetailView: View {
    var cafe: Cafe
    @Environment(\.dismiss) var dismiss
    
    var body: some View {
        Image(cafe.image)
            .resizable()
            .scaledToFill()
            .frame(minWidth: 0, maxWidth: .infinity)
            .clipped()
            .overlay(alignment: .topTrailing) {
                Button {
                    dismiss()
                } label: {
                    Image(systemName: "xmark.circle.fill")
                        .font(.system(size: 30))
                        .foregroundStyle(Color.white)
                        .opacity(0.7)
                        .padding()
                        .padding(.top, 30)
                }
            }
            .ignoresSafeArea()
    }
}

To enable interaction with the cafe photos, we can use NavigationLink to manage the navigation. When tapped, the app displays the detail view which shows the image in full screen view.

ForEach(sampleCafes) { cafe in
    
    NavigationLink {
        DetailView(cafe: cafe)
        
    } label: {
        Image(cafe.image)
            .resizable()
            .scaledToFill()
            .frame(minWidth: 0, maxWidth: .infinity)
            .frame(height: 400)
            .clipShape(RoundedRectangle(cornerRadius: 20))
    }
    .padding()
}

In preview mode, you can navigate between the detail view and the list view. At this stage, the transition uses the default animation for navigation stacks, without any custom effects.

Now it comes to the fun part. Let’s create the hero animation by using the new NavigationTransition protocol. The very first step is to define a namespace for the animation. In ContentView, declare the following namespace variable:

@Namespace var namespace

Next, apply the matchedTransitionSource modifier to the source view, which is the image view in the list. Then, use the navigationTransition modifier on the detail view. Update your code as shown below:

NavigationLink {
    DetailView(cafe: cafe)
        .navigationTransition(.zoom(sourceID: cafe.id, in: namespace))
        .toolbarVisibility(.hidden, for: .navigationBar)
    
} label: {
    Image(cafe.image)
        .resizable()
        .scaledToFill()
        .frame(minWidth: 0, maxWidth: .infinity)
        .frame(height: 400)
        .clipShape(RoundedRectangle(cornerRadius: 20))
        .matchedTransitionSource(id: cafe.id, in: namespace)
}

To enhance the visual experience, I've included the toolbarVisibility modifier to conceal the navigation bar. This removes the Back button from view, creating a more immersive full-screen presentation of the cafe image.

In preview mode, test the app by tapping a cafe image. It will display a full-screen image with a hero animation. To return to the list, either tap the "X" button or simply drag the image downwards. The app will animate the image back to its original position in the list, providing a fluid and intuitive user experience.

Summary

The new NavigationTransition protocol has made it remarkably simple for developers to create hero animations for view transitions, allowing for a richer user experience with just a few lines of code. Consider exploring this new feature to elevate your app's interactivity and user satisfaction.

It's important to note that this API is only compatible with iOS 18 and later. If your app needs to support older iOS versions, you'll have to implement the animation yourself. Our "Mastering SwiftUI" book provides guidance on how to achieve this.

The Vision framework has long included text recognition capabilities. We already have a detailed tutorial that shows you how to scan an image and perform text recognition using the Vision framework. Previously, we utilized VNImageRequestHandler and VNRecognizeTextRequest to extract text from an image.

Over the years, the Vision framework has evolved significantly. In iOS 18, Vision introduces new APIs that leverage the power of Swift 6. In this tutorial, we will explore how to use these new APIs to perform text recognition. You will be amazed by the improvements in the framework, which save you a significant amount of code to implement the same feature.

As always, we will create a demo application to guide you through the APIs. We will build a simple app that allows users to select an image from the photo library, and the app will extract the text from it in real time.

Let’s get started.

Loading the Photo Library with PhotosPicker

Assuming you’ve created a new SwiftUI project on Xcode 16, go to ContentView.swift and start building the basic UI of the demo app:

import SwiftUI
import PhotosUI

struct ContentView: View {
    
    @State private var selectedItem: PhotosPickerItem?
    
    @State private var recognizedText: String = "No text is detected"
    
    var body: some View {
        VStack {
            ScrollView {
                VStack {
                    Text(recognizedText)
                }
            }
            .contentMargins(.horizontal, 20.0, for: .scrollContent)
            
            Spacer()
            
            PhotosPicker(selection: $selectedItem, matching: .images) {
                Label("Select a photo", systemImage: "photo")
            }
            .photosPickerStyle(.inline)
            .photosPickerDisabledCapabilities([.selectionActions])
            .frame(height: 400)
            
        }
        .ignoresSafeArea(edges: .bottom)
    }
}

We utilize PhotosPicker to access the photo library and load the images in the lower part of the screen. The upper part of the screen features a scroll view for display the recognized text.

We have a state variable to keep track of the selected photo. To detect the selected image and load it as Data, you can attach the onChange modifier to the PhotosPicker view like this:

.onChange(of: selectedItem) { oldItem, newItem in
    Task {
        guard let imageData = try? await newItem?.loadTransferable(type: Data.self) else {
            return
        }
    }
}

Text Recognition with Vision

The new APIs in the Vision framework have simplified the implementation of text recognition. Vision offers 31 different request types, each tailored for a specific kind of image analysis. For instance, DetectBarcodesRequest is used for identifying and decoding barcodes. For our purposes, we will be using RecognizeTextRequest.

In the ContentView struct, add an import statement to import Vision and create a new function named recognizeText:

private func recognizeText(image: UIImage) async {
    guard let cgImage = image.cgImage else { return }
    
    let textRequest = RecognizeTextRequest()
    
    let handler = ImageRequestHandler(cgImage)
    
    do {
        let result = try await handler.perform(textRequest)
        let recognizedStrings = result.compactMap { observation in
            observation.topCandidates(1).first?.string
        }
        
        recognizedText = recognizedStrings.joined(separator: "\n")
        
    } catch {
        recognizedText = "Failed to recognized text"
        print(error)
    }
}

This function takes in an UIImage object, which is the selected photo, and extract the text from it. The RecognizeTextRequest object is designed to identify rectangular text regions within an image.

The ImageRequestHandler object processes the text recognition request on a given image. When we call its performfunction, it returns the results as RecognizedTextObservation objects, each containing details about the location and content of the recognized text.

We then use compactMap to extract the recognized strings. The topCandidates method returns the best matches for the recognized text. By setting the maximum number of candidates to 1, we ensure that only the top candidate is retrieved.

Finally, we use the joined method to concatenate all the recognized strings.

With the recognizeText method in place, we can update the onChange modifier to call this method, performing text recognition on the selected photo.

.onChange(of: selectedItem) { oldItem, newItem in
    Task {
        guard let imageData = try? await newItem?.loadTransferable(type: Data.self) else {
            return
        }
        
        await recognizeText(image: UIImage(data: imageData)!)
    }
}

With the implementation complete, you can now run the app in a simulator to test it out. If you have a photo containing text, the app should successfully extract and display the text on screen.

Summary

With the introduction of the new Vision APIs in iOS 18, we can now achieve text recognition tasks with remarkable ease, requiring only a few lines of code to implement. This enhanced simplicity allows developers to quickly and efficiently integrate text recognition features into their applications.

What do you think about this improvement of the Vision framework? Feel free to leave comment below to share your thought.

iOS already includes a system-wide translation feature that allows users to easily translate text into various languages. With the release of iOS 17.4 (and iOS 18), you can now leverage the new Translation API to integrate this powerful translation capability into your apps.

Apple provides two options for developers to utilize the Translation API. The quickest and simplest method is to use the .translationPresentation modifier, which displays a translation overlay in your app. For a more flexible solution, you can directly call the Translation API to build a custom translation feature.

In this tutorial, we will explore both approaches and guide you through their implementation using a SwiftUI demo app. Please note that you will need Xcode 16 to follow along.

Using the translationPresentation Modifier

Let's start with the straightforward approach: the .translationPresentation modifier. In Safari, users can highlight any text to access the translation option, which then displays a translation overlay with the translated text.

If you want to bring this translation overlay to your app, all you need is to import the Translation package and use the .translationPresentation modifier. Take a look at the following sample code:

import SwiftUI
import Translation

struct ContentView: View {
    
    @State private var showTranslation = false
    @State private var sampleText = article
    
    var body: some View {
        VStack {
            Text(sampleText)
                .font(.system(.body, design: .rounded))
                
            Button("Translate") {
                showTranslation.toggle()
            }
            .controlSize(.extraLarge)
            .buttonStyle(.borderedProminent)          
        }
        .padding()
        .translationPresentation(isPresented: $showTranslation, text: article)
    }
}

The app displays some sample text in English with a Translate button placed below it.

Now, when you tap the "Translate" button, a translation overlay appears, displaying the translated text in your desired language. Other than iOS, the Translation API also works on both iPadOS and macOS. Currently, this translation feature cannot be tested in Xcode Preview; you must deploy the app onto a real device for testing.

The .translationPresentation modifier allows you to specify an optional action to be performed when users tap the "Replace with Translation" button. For instance, if you want to replace the original text with the translated text when the button is tapped, you can define this action like this:

.translationPresentation(isPresented: $showTranslation, text: article) { translatedText in
    
    sampleText = translatedText
    
}

Once you specify the action in the modifier, you will see the “Replace with Translation” option in the translation overlay.

Working with the Translation API

For greater control over translations, you can use the Translation API directly instead of relying on the translation overlay. For instance, if your app displays a list of article excerpts and you want to offer translation support, the translation overlay might not be ideal because users would have to select each excerpt individually for translation.

A more efficient solution is to perform a batch translation of all the article excerpts when users tap the "Translate" button. Let’s create a simple demo to see how to work with the Translation API and handle batch translations.

Below is the sample code for creating the UI above:

struct BatchTranslationDemo: View {
    
    @State private var articles = sampleArticles
    
    var body: some View {
        NavigationStack {
            List(articles) { article in
                
                VStack {
                    Text(article.text)
                    
                    if article.translatedText != "" {
                        Text(article.translatedText)
                            .frame(maxWidth: .infinity, alignment: .leading)
                            .padding()
                            .background(Color(.systemGray4))
                    }
                }
                
            }
            .listStyle(.plain)

            .toolbar {
                Button {
                    
                } label: {
                    Label("Translate", systemImage: "translate")
                        .labelStyle(.iconOnly)
                }

            }
        }
        
    }
}

To perform a batch translation, you first need to define a translation configuration that specifies both source and target languages. In the code, you can declare a state variable to hold the configuration like below:

@State private var configuration: TranslationSession.Configuration?

And then, in the closure of the toolbar’s Button, we can instantiate the configuration:

Button {
    
    if configuration == nil {
        configuration = TranslationSession.Configuration(source: .init(identifier: "en-US"), target: .init(identifier: "zh-Hant-TW"))
        return
    }
    
    configuration?.invalidate()
    
} label: {
    Label("Translate", systemImage: "translate")
        .labelStyle(.iconOnly)
}

We specify English as the source language and Traditional Chinese as the target language. If you do not specify the languages, the Translation API will automatically create a default configuration, with iOS determining the source and target languages for you.

To perform translation, you attach the .translationTask modifier to the list view:

List(articles) { article in
	.
	.
	.
}
.translationTask(configuration) { session in
    
    let requests = articles.map { TranslationSession.Request(sourceText: $0.text, clientIdentifier: $0.id.uuidString) }
    
    if let responses = try? await session.translations(from: requests) {
        
        responses.forEach { response in
            updateTranslation(response: response)
        }
    }
}

This modifier initiates a translation task using the specified configuration. Whenever the configuration changes and is not nil, the translation task is executed. Within the closure, we prepare a set of translation requests and use the session's translations(from:) method to perform a batch translation.

If the translation task succeeds, it returns an array of translation responses containing the translated text. We then pass this translated text to the updateTranslation method to display it on screen.

func updateTranslation(response: TranslationSession.Response) {
    
    guard let index = articles.firstIndex(where: { $0.id.uuidString == response.clientIdentifier }) else {
        return
    }
    
    articles[index].translatedText = response.targetText
    
}

Let's deploy the app to a real device for testing. I tested the app on my iPad Air. When you tap the "Translate" button, the app should display additional article excerpts in Traditional Chinese.

Summary

With the new Translation API introduced in iOS 17.4 (and iOS 18), developers can now easily integrate powerful translation features into their apps. This tutorial covers two primary methods for utilizing the API: the simple approach using the .translationPresentation modifier for displaying a translation overlay, and a more flexible approach using the Translation API directly for custom translation solutions.

We demonstrate both approaches in this tutorial. As illustrated in the demo, you can easily add translation capabilities with just a few lines of code. Given the simplicity and power of this API, there’s no reason not to include translation functionality in your apps.

The world of SwiftUI is constantly evolving, with each update pushing the boundaries of app development. With iOS 18, the enhancements are both exciting and significant, set to transform how developers engage with SwiftUI.

This guide aims to explore every new feature and improvement in this version, offering a comprehensive overview of the changes.

The Floating Tab Bar

The Tab view in SwiftUI has been greatly enhanced with the addition of a floating tab bar. This new feature can seamlessly transition into a sidebar, providing users with an intuitive way to access the full functionality of an app.

On iPad, users can now tap a sidebar button on the tab bar to transform the tab bar into sidebar. For developers, it’s just a line of code if you want to support this feature. All you need is to set the tab view style to .sidebarAdaptable:

struct ContentView: View {
    @State var customization = TabViewCustomization()
    
    var body: some View {
        TabView {
            Tab("Home", systemImage: "house.fill") {
                
            }
            
            Tab("Bookmark", systemImage: "bookmark.circle.fill") {
                
            }
            
            Tab("Videos", systemImage: "video.circle.fill") {
                
            }
            
            Tab("Profile", systemImage: "person.crop.circle") {
                
            }
            
            Tab("Settings", systemImage: "gear") {
                
            }
            
        }
        .tint(.yellow)
        .tabViewStyle(.sidebarAdaptable)
        .tabViewCustomization($customization)
    }
}

Once the option is set, users can effortlessly switch between a sidebar and a tab bar, enhancing navigation flexibility. Additionally, the new tab bar offers extensive customization. By attaching the .tabViewCustomization modifier to the Tab view, users can tailor the menu items of the tab bar.

Sheet Presentation Sizing

Sheet presentation sizing is now consistent and streamlined across platforms. By using the .presentationSizing modifier, you can easily create sheets with ideal dimensions using presets such as .form or .page, or even specify custom sizes. Here is a sample:

struct PresentationSizingDemo: View {
    
    @State private var showSheet = false
    
    var body: some View {
        Button {
            showSheet.toggle()
        } label: {
            Text("Show sheet")
        }
        .sheet(isPresented: $showSheet) {
            Text("This is a quick demo of presentation sizing.")
                .presentationSizing(.form)
        }
    }
}

On iPad, the .form preset displays a smaller sheet compared to .page. However, there is no size difference on iPhone.

Color Mesh Gradients

SwiftUI now offers extensive support for colorful mesh gradients. The new MeshGradient feature allows you to create two-dimensional gradients using a grid of positioned colors. By combining control points and colors, you can design a wide variety of gradient effects.

Below shows a couple of gradients created using MeshGradient:

struct ColorMeshDemo: View {
    var body: some View {
        VStack {
            MeshGradient(
                width: 3,
                height: 3,
                points: [
                    .init(0, 0), .init(0.5, 0), .init(1, 0),
                    .init(0, 0.5), .init(0.3, 0.5), .init(1, 0.5),
                    .init(0, 1), .init(0.5, 1), .init(1, 1)
                ],
                colors: [
                    .gray, .purple, .indigo,
                    .orange, .cyan, .blue,
                    .yellow, .green, .teal
                ]
            )
            
            MeshGradient(
                width: 2,
                height: 2,
                points: [
                    .init(0, 0), .init(1, 0),
                    .init(0, 1), .init(1, 1)
                ],
                colors: [
                    .red, .purple,
                    .yellow, .green
                ]
            )
        }
        .ignoresSafeArea()
    }
}

Zoom Transition

SwiftUI now has the built-in support of zoom transition. You can use the .matchedTransitionSource modifier to easily render the zoom transition.

If you're familiar with using matchedGeometryEffect, you'll find matchedTransitionSource quite similar. Below is sample code we wrote to create the zoom transition shown above:

struct ZoomTransitionDemo: View {
    let samplePhotos = (1...20).map { Photo(name: "coffee-\($0)") }
    
    @Namespace() var namespace
    
    var body: some View {
        NavigationStack {
            ScrollView {
                LazyVGrid(columns: [ GridItem(.adaptive(minimum: 150)) ]) {
                    
                    ForEach(samplePhotos) { photo in
                        NavigationLink {
                            Image(photo.name)
                                .resizable()
                                .navigationTransition(.zoom(sourceID: photo.id, in: namespace))
                        } label: {
                            Image(photo.name)
                                .resizable()
                                .scaledToFill()
                                .frame(minWidth: 0, maxWidth: .infinity)
                                .frame(height: 150)
                                .cornerRadius(30.0)
                        }
                        .matchedTransitionSource(id: photo.id, in: namespace)
                        
                    }
                }
            }
        }
        .padding()
    }
}

The matchedTransitionSource modifier is applied to a NavigationLink with a specific photo ID, designating the view as the source of the navigation transition. For the destination view, which is also an Image view, the navigationTransition modifier is used to render the zoom transition.

More Animations for SF Symbols 6

iOS 17 introduced a fantastic collection of expressive animations for SF Symbols. Developers can leverage these animations using the new symbolEffect modifier. iOS 18 pushes the SF Symbols to version 6 with an even wider variety of animated symbols for developers to utilize in their apps.

Here is a sample code snippet for the new rotate animation:

Image(systemName: "ellipsis.message")
            .font(.system(size: 300))
            .symbolRenderingMode(.palette)
            .foregroundStyle(.purple, .gray)
            .symbolEffect(.rotate, value: animate)
            .onTapGesture {
                animate.toggle()
            }

On top of the rotate animation, SF Symbols 6 also provides two other types of animation including .wiggle and .breathe.

Enhancements of SwiftUI Charts

The SwiftUI Charts framework now supports vectorized and function plots. For example, let’s say you want to plot a graph for the following function:

y = x^2

You can use LinePlot to plot the graph like this:

Chart {
    LinePlot(x: "x", y: "y") { x in
        return pow(x, 2)
    }
    .foregroundStyle(.green)
    .lineStyle(.init(lineWidth: 10))
}
.chartXScale(domain: -4...4)
.chartYScale(domain: -4...4)
.chartXAxis {
    AxisMarks(values: .automatic(desiredCount: 10))
}
.chartYAxis {
    AxisMarks(values: .automatic(desiredCount: 10))
}
.chartPlotStyle { plotArea in
    plotArea
        .background(.yellow.opacity(0.02))
}

You can simply provide the function to a LinePlot to graph a function.

More Control of Scroll Views

The new version of SwiftUI delivers a powerful set of new APIs that give developers fine-grained control over their scroll views. The introduction of the onScrollGeometryChange modifier allows you to keep track with the state of scroll views. This new capability enables you to efficiently react to changes in the scroll view's content offsets, content size, and other scroll-related properties.

Here's a sample code snippet that demonstrates how you can use this modifier to display a "Scroll to Top" button after the user has scrolled down a list:

struct ScrollViewDemo: View {
    
    let samplePhotos = (1...20).map { Photo(name: "coffee-\($0)") }
    
    @State private var showScrollToTop = false
    
    var body: some View {
        ScrollView {
            VStack {
                ForEach(samplePhotos) { photo in
                    Image(photo.name)
                        .resizable()
                        .scaledToFill()
                        .frame(height: 200)
                        .clipShape(RoundedRectangle(cornerRadius: 15))
                }
            }
        }
        .padding(.horizontal)
        .overlay(alignment: .bottom) {
            if showScrollToTop {
                Button("Scroll to top") {
                    
                }
                .controlSize(.extraLarge)
                .buttonStyle(.borderedProminent)
                .tint(.green)
            }
        }
        .onScrollGeometryChange(for: Bool.self) { geometry in
            geometry.contentOffset.y < geometry.contentInsets.bottom + 200
            
        } action: { oldValue, newValue in
            withAnimation {
                showScrollToTop = !newValue
            }
        }

    }
}

The geometry of a scroll view changes frequently while scrolling. We can leverage the onScrollGeometryChange modifier to capture the update and display the “Scroll to top” button accordingly.

SwiftUI also introduces the onScrollVisibilityChange modifier for views within a scroll view. This modifier allows you to detect when a particular view becomes visible and perform specific actions in response.

Suppose we have a Rectangle view at the end of a scroll view and we want to trigger a color change animation only when this view comes into view. We can use the onScrollVisibilityChange modifier to detect when the view becomes visible and when it goes off-screen.

Rectangle()
    .fill(color)
    .frame(height: 100)
    .onScrollVisibilityChange(threshold: 0.9) { visible in
        withAnimation(.linear(duration: 5)) {
            color = visible ? .green : .blue
        }
    }

Widgets in Control Center

You now have the ability to design custom resizable controls, like buttons and toggles, which can be placed in the Control Center or on the lock screen. Controls are a new kind of Widget that that are easy to build with App Intents.

To create a control widget in Control Center, you adopt the ControlWidget protocol and provide the implementation. Here is a sample code provided by Apple:

struct StartPartyControl: ControlWidget {
    var body: some ControlWidgetConfiguration {
        StaticControlConfiguration(
            kind: "com.apple.karaoke_start_party"
        ) {
            ControlWidgetButton(action: StartPartyIntent()) {
                Label("Start the Party!", systemImage: "music.mic")
                Text(PartyManager.shared.nextParty.name)
            }
        }
    }
}

We will further look into control widgets in a separate tutorial.

A new Mix Modifier for Color

You can now blend two different colors to create your desired hue by using the new mix modifier. Here is an example:

VStack {
    Color.purple.mix(with: .green, by: 0.3)
        .frame(height: 100)
    
    Color.purple.mix(with: .green, by: 0.5)
        .frame(height: 100)
    
    Color.purple.mix(with: .green, by: 0.8)
        .frame(height: 100)
}

Simply provide the mix modifier with the color to blend and the blend ratio. SwiftUI will then generate the new color based on these parameters.

Visual Effects for Text

You can now extend SwiftUI Text views with custom rendering effects by adopting the TextRenderer. Here is a sample text renderer:

struct CustomTextRenderer: TextRenderer {
    
    func draw(layout: Text.Layout, in context: inout GraphicsContext) {
        
        for line in layout {
            for (index, slice) in runs.enumerated() {
                context.opacity = (index % 2 == 0) ? 0.4 : 1.0
                context.translateBy(x: 0, y: index % 2 != 0 ? -15 : 15)
                
                context.draw(slice)
            }
        }
    }
}

struct TextAnimationDemo: View {
    var body: some View {
        Text("What's New in SwiftUI")
            .font(.system(size: 100))
            .textRenderer(CustomTextRenderer())
    }
}

By implementing the draw method, you can customize the visual effect of each character.

Summary

The iOS 18 update introduces a host of significant enhancements to SwiftUI. This tutorial offers a concise introduction to some of the new features. For more complex features, we will be creating detailed, standalone tutorials to thoroughly explore their applications and benefits. Be sure to stay tuned for these upcoming in-depth guides.

Previously, we provided a brief introduction to Google Gemini APIs and demonstrated how to build a Q&A application using SwiftUI. You should realize how straightforward it is to integrate Google Gemini and enhance your apps with AI features. We have also developed a demo application to demonstrate how to construct a chatbot app using the AI APIs.

The gemini-pro model discussed in the previous tutorial is limited to generating text from text-based input. However, Google Gemini also offers a multimodal model called gemini-pro-vision, which can generate text descriptions from images. In other words, this model has the capacity to detect and describe objects in an image.

In this tutorial, we will demonstrate how to use Google Gemini APIs for image recognition. This simple app allows users to select an image from their photo library and uses Gemini to describe the contents of the photo.

Before proceeding with this tutorial, please visit Google AI Studio and create your own API key if you haven’t done so already.

Adding Google Generative AI Package in Xcode Projects

Assuming you’ve already created an app project in Xcode, the first step to using Gemini APIs is importing the SDK. To accomplish this, right-click on the project folder in the project navigator and select Add Package Dependencies. In the dialog box, input the following package URL:

https://github.com/google/generative-ai-swift

You can then click on the Add Package button to download and incorporate the GoogleGenerativeAI package into the project.

Next, to store the API key, create a property file named GeneratedAI-Info.plist. In this file, create a key named API_KEY and enter your API key as the value.

To read the API key from the property file, create another Swift file named APIKey.swift. Add the following code to this file:

enum APIKey {
  // Fetch the API key from `GenerativeAI-Info.plist`
  static var `default`: String {

    guard let filePath = Bundle.main.path(forResource: "GenerativeAI-Info", ofType: "plist")
    else {
      fatalError("Couldn't find file 'GenerativeAI-Info.plist'.")
    }

    let plist = NSDictionary(contentsOfFile: filePath)

    guard let value = plist?.object(forKey: "API_KEY") as? String else {
      fatalError("Couldn't find key 'API_KEY' in 'GenerativeAI-Info.plist'.")
    }

    if value.starts(with: "_") {
      fatalError(
        "Follow the instructions at https://ai.google.dev/tutorials/setup to get an API key."
      )
    }

    return value
  }
}

Building the App UI

The user interface is straightforward. It features a button at the bottom of the screen, allowing users to access the built-in Photo library. After a photo is selected, it appears in the image view.

To bring up the built-in Photos library, we use PhotosPicker, which is a native photo picker view for managing photo selections. When presenting the PhotosPicker view, it showcases the photo album in a separate sheet, rendered atop your app’s interface.

First, you need to import the PhotosUI framework in order to use the photo picker view:

import PhotosUI

Next, update the ContentView struct like this to implement the user interface:

struct ContentView: View {

    @State private var selectedItem: PhotosPickerItem?
    @State private var selectedImage: Image?

    var body: some View {
        VStack {

            if let selectedImage {
                selectedImage
                    .resizable()
                    .scaledToFit()
                    .clipShape(RoundedRectangle(cornerRadius: 20.0))
            } else {

                Image(systemName: "photo")
                    .imageScale(.large)
                    .foregroundStyle(.gray)
                    .frame(maxWidth: .infinity, maxHeight: .infinity)
                    .background(Color(.systemGray6))
                    .clipShape(RoundedRectangle(cornerRadius: 20.0))
            }

            Spacer()

            PhotosPicker(selection: $selectedItem, matching: .images) {
                Label("Select Photo", systemImage: "photo")
                    .frame(maxWidth: .infinity)
                    .bold()
                    .padding()
                    .foregroundStyle(.white)
                    .background(.indigo)
                    .clipShape(RoundedRectangle(cornerRadius: 20.0))
            }
        }
        .padding(.horizontal)
        .onChange(of: selectedItem) { oldItem, newItem in
            Task {
                if let image = try? await newItem?.loadTransferable(type: Image.self) {
                    selectedImage = image
                }
            }
        }
    }
}

To use the PhotosPicker view, we declare a state variable to store the photo selection and then instantiate a PhotosPicker view by passing the binding to the state variable. The matching parameter allows you to specify the asset type to display.

When a photo is selected, the photo picker automatically closes, storing the chosen photo in the selectedItem variable of type PhotosPickerItem. The loadTransferable(type:completionHandler:) method can be used to load the image. By attaching the onChange modifier, you can monitor updates to the selectedItem variable. If there is a change, we invoke the loadTransferable method to load the asset data and save the image to the selectedImage variable.

Because selectedImage is a state variable, SwiftUI automatically detects when its content changes and displays the image on the screen.

Image Analysis and Object Recognition

Having selected an image, the next step is to use the Gemini APIs to perform image analysis and generate a text description from the image.

Before using the APIs, insert the following statement at the very beginning of ContentView.swift to import the framework:

import GoogleGenerativeAI

Next, declare a model property to hold the AI model:

let model = GenerativeModel(name: "gemini-pro-vision", apiKey: APIKey.default)

For image analysis, we utilize the gemini-pro-vision model provided by Google Gemini. Then, we declare two state variables: one for storing the generated text and another for tracking the analysis status.

@State private var analyzedResult: String?
@State private var isAnalyzing: Bool = false

Next, create a new function named analyze() to perform image analysis:

@MainActor func analyze() {

    self.analyzedResult = nil
    self.isAnalyzing.toggle()

    // Convert Image to UIImage
    let imageRenderer = ImageRenderer(content: selectedImage)
    imageRenderer.scale = 1.0

    guard let uiImage = imageRenderer.uiImage else {
        return
    }

    let prompt = "Describe the image and explain what the objects found in the image"

    Task {
        do {
            let response = try await model.generateContent(prompt, uiImage)

            if let text = response.text {
                print("Response: \(text)")
                self.analyzedResult = text
                self.isAnalyzing.toggle()
            }
        } catch {
            print(error.localizedDescription)
        }
    }
}

Before using the model’s API, we need to convert the image view into an UIImage. We then invoke the generateContent method with the image and a predefined prompt, asking Google Gemini to describe the image and identify the objects within it.

When the response arrives, we extract the text description and assign it to the analyzedResult variable.

Next, insert the following code and place it above the Spacer() view:

ScrollView {
    Text(analyzedResult ?? (isAnalyzing ? "Analyzing..." : "Select a photo to get started"))
        .font(.system(.title2, design: .rounded))
}
.padding()
.frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .leading)
.background(Color(.systemGray6))
.clipShape(RoundedRectangle(cornerRadius: 20.0))

This scroll view displays the text generated by Gemini. Optionally, you can add an overlay modifier to the selectedImage view. This will display a progress view while an image analysis is being performed.

.overlay {

    if isAnalyzing {
        RoundedRectangle(cornerRadius: 20.0)
            .fill(.black)
            .opacity(0.5)

        ProgressView()
            .tint(.white)
    }
}

After implementing all the changes, the preview pane should now be displaying a newly designed user interface. This interface comprises of the selected image, the image description area, and a button to select photos from the photo library. This is what you should see in your preview pane if all the steps have been followed and executed correctly.

Finally, insert a line of code in the onChange modifier to call the analyze() method after the selectedImage. That’s all! You can now test the app in the preview pane. Click on the Select Photo button and choose a photo from the library. The app will then send the selected photo to Google Gemini for analysis and display the generated text in the scroll view.

Summary

The tutorial demonstrates how to build an AI image recognition app using Google Gemini APIs and SwiftUI. The app allows users to select an image from their photo library and uses Gemini to describe the contents of the photo.

From the code we have just worked on, you can see that it only requires a few lines to prompt Google Gemini to generate text from an image. Although this demo illustrates the process using a single image, the API actually supports multiple images. For further details on how it functions, please refer to the official documentation.