Skip to content

[FirebaseAI] Add usage of Grounding with Google Search #1724

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 13 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions firebaseai/ChatExample/Models/ChatMessage.swift
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.

import FirebaseAI
import Foundation

enum Participant {
Expand All @@ -22,12 +23,19 @@ enum Participant {
struct ChatMessage: Identifiable, Equatable {
let id = UUID().uuidString
var message: String
var groundingMetadata: GroundingMetadata?
let participant: Participant
var pending = false

static func pending(participant: Participant) -> ChatMessage {
Self(message: "", participant: participant, pending: true)
}

// TODO(andrewheard): Add Equatable conformance to GroundingMetadata and remove this
static func == (lhs: ChatMessage, rhs: ChatMessage) -> Bool {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This equality operator is only here because GroundingMetadata doesn't conform to Equatable. Since it's a rather complex type, I suggest simplifying this == implementation to only check equality of the id attribute. This is sufficient, since each chat message is uniquely identified by its id attribute.

Let's also call this out:

  // For chat messages, ID-based equality is appropriate since each message should be unique
  static func == (lhs: ChatMessage, rhs: ChatMessage) -> Bool {
    lhs.id == rhs.id
  }

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this implementation only compares the id's, won't the view not update if we update a ChatMessage field like text (for example, in a case where we are streaming a chat message, and updating the text field as each chunk arrives)?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think @dlarocque is right. As another example, if the equality check doesn't include pending then the following won't trigger:

.onChange(of: viewModel.messages, perform: { newValue in
if viewModel.hasError {
// wait for a short moment to make sure we can actually scroll to the bottom
DispatchQueue.main.asyncAfter(deadline: .now() + 0.05) {
withAnimation {
scrollViewProxy.scrollTo("errorView", anchor: .bottom)
}
focusedField = .message
}
} else {
guard let lastMessage = viewModel.messages.last else { return }
// wait for a short moment to make sure we can actually scroll to the bottom
DispatchQueue.main.asyncAfter(deadline: .now() + 0.05) {
withAnimation {
scrollViewProxy.scrollTo(lastMessage.id, anchor: .bottom)
}
focusedField = .message
}
}
})

This results in MessageContentView showing the bouncing dots forever:

if message.pending {
BouncingDots()
} else {

How about we add // TODO(andrewheard): Add Equatable conformance to GroundingMetadata and remove this instead? Although GroundingMetadata is quite a complex type, all the leaf nodes are basic types so it's quite easy to make it conform and Apple actually recommends doing so:

Conforming to the Equatable and Hashable protocols is straightforward and makes it easier to use your own types in Swift. It’s a good idea for all your custom model types to conform.
-- https://developer.apple.com/documentation/swift/adopting-common-protocols

I think it's something we should consider for our APIs going forward.

Copy link
Contributor

@peterfriese peterfriese Jul 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point - I think I got too used to how this works in the new world (with the Observable macro), where this does indeed work.

So we've got three options:

  1. Go with a slightly more complete (but still incomplete) == implementation
  2. Make sure GroundingMetadata conforms to Equatable
  3. Migrate to the Observation framework

Since not everybody will be able to use the new Observation framework, we should probably implement (2) in the short term, which also doesn't hurt once we're able to migrate to the Observation framework.

lhs.id == rhs.id && lhs.message == rhs.message && lhs.participant == rhs.participant && lhs
.pending == rhs.pending
}
}

extension ChatMessage {
Expand Down
17 changes: 12 additions & 5 deletions firebaseai/ChatExample/Screens/ConversationScreen.swift
Original file line number Diff line number Diff line change
Expand Up @@ -18,15 +18,22 @@ import SwiftUI

struct ConversationScreen: View {
let firebaseService: FirebaseAI
let title: String
@StateObject var viewModel: ConversationViewModel

@State
private var userPrompt = ""

init(firebaseService: FirebaseAI) {
init(firebaseService: FirebaseAI, title: String, searchGroundingEnabled: Bool = false) {
let model = firebaseService.generativeModel(
modelName: "gemini-2.0-flash-001",
tools: searchGroundingEnabled ? [.googleSearch()] : []
)
self.title = title
self.firebaseService = firebaseService
_viewModel =
StateObject(wrappedValue: ConversationViewModel(firebaseService: firebaseService))
StateObject(wrappedValue: ConversationViewModel(firebaseService: firebaseService,
model: model))
}

enum FocusedField: Hashable {
Expand Down Expand Up @@ -88,7 +95,7 @@ struct ConversationScreen: View {
}
}
}
.navigationTitle("Chat example")
.navigationTitle(title)
.onAppear {
focusedField = .message
}
Expand Down Expand Up @@ -123,7 +130,7 @@ struct ConversationScreen_Previews: PreviewProvider {
.firebaseAI()) // Example service init

var body: some View {
ConversationScreen(firebaseService: FirebaseAI.firebaseAI())
ConversationScreen(firebaseService: FirebaseAI.firebaseAI(), title: "Chat sample")
.onAppear {
viewModel.messages = ChatMessage.samples
}
Expand All @@ -132,7 +139,7 @@ struct ConversationScreen_Previews: PreviewProvider {

static var previews: some View {
NavigationStack {
ConversationScreen(firebaseService: FirebaseAI.firebaseAI())
ConversationScreen(firebaseService: FirebaseAI.firebaseAI(), title: "Chat sample")
}
}
}
25 changes: 22 additions & 3 deletions firebaseai/ChatExample/ViewModels/ConversationViewModel.swift
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,15 @@ class ConversationViewModel: ObservableObject {

private var chatTask: Task<Void, Never>?

init(firebaseService: FirebaseAI) {
model = firebaseService.generativeModel(modelName: "gemini-2.0-flash-001")
chat = model.startChat()
init(firebaseService: FirebaseAI, model: GenerativeModel? = nil) {
if let model {
self.model = model
} else {
self.model = firebaseService.generativeModel(
modelName: "gemini-2.0-flash-001"
)
}
chat = self.model.startChat()
}

func sendMessage(_ text: String, streaming: Bool = true) async {
Expand Down Expand Up @@ -85,7 +91,14 @@ class ConversationViewModel: ObservableObject {
if let text = chunk.text {
messages[messages.count - 1].message += text
}

if let candidate = chunk.candidates.first {
if let groundingMetadata = candidate.groundingMetadata {
self.messages[self.messages.count - 1].groundingMetadata = groundingMetadata
}
}
}

} catch {
self.error = error
print(error.localizedDescription)
Expand Down Expand Up @@ -119,6 +132,12 @@ class ConversationViewModel: ObservableObject {
// replace pending message with backend response
messages[messages.count - 1].message = responseText
messages[messages.count - 1].pending = false

if let candidate = response?.candidates.first {
if let groundingMetadata = candidate.groundingMetadata {
self.messages[self.messages.count - 1].groundingMetadata = groundingMetadata
}
}
}
} catch {
self.error = error
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

import SwiftUI
import WebKit

/// A view that renders Google Search suggestions with links that allow users
/// to view the search results in the device's default browser.
/// This is added to the bottom of chat messages containing results grounded
/// in Google Search.
struct GoogleSearchSuggestionView: UIViewRepresentable {
let htmlString: String

// This Coordinator class will act as the web view's navigation delegate.
class Coordinator: NSObject, WKNavigationDelegate {
func webView(_ webView: WKWebView,
decidePolicyFor navigationAction: WKNavigationAction,
decisionHandler: @escaping (WKNavigationActionPolicy) -> Void) {
// Check if the navigation was triggered by a user clicking a link.
if navigationAction.navigationType == .linkActivated {
if let url = navigationAction.request.url {
// Open the URL in the system's default browser (e.g., Safari).
UIApplication.shared.open(url)
}
// Cancel the navigation inside our small web view.
decisionHandler(.cancel)
return
}
// For all other navigation types (like the initial HTML load), allow it.
decisionHandler(.allow)
}
}

func makeCoordinator() -> Coordinator {
Coordinator()
}

func makeUIView(context: Context) -> WKWebView {
let webView = WKWebView()
webView.isOpaque = false
webView.backgroundColor = .clear
webView.scrollView.backgroundColor = .clear
webView.scrollView.isScrollEnabled = false
// Set the coordinator as the navigation delegate.
webView.navigationDelegate = context.coordinator
return webView
}

func updateUIView(_ uiView: WKWebView, context: Context) {
// The renderedContent is an HTML snippet with CSS.
// For it to render correctly, we wrap it in a basic HTML document structure.
let fullHTML = """
<!DOCTYPE html>
<html>
<head>
<meta name='viewport' content='width=device-width, initial-scale=1.0, user-scalable=no'>
<style>
body { margin: 0; padding: 0; }
</style>
</head>
<body>
\(htmlString)
</body>
</html>
"""
uiView.loadHTMLString(fullHTML, baseURL: nil)
}
}
81 changes: 81 additions & 0 deletions firebaseai/ChatExample/Views/Grounding/GroundedResponseView.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
// Copyright 2023 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

import FirebaseAI
import SwiftUI

/// A view that displays a chat message that is grounded in Google Search.
struct GroundedResponseView: View {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it might be better to enhance ResponseTextView to provide a slot for an auxiliary view.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can do that in a follow-up PR.

var message: ChatMessage
var groundingMetadata: GroundingMetadata

var body: some View {
// We can only display a response grounded in Google Search if the searchEntrypoint is non-nil.
let isCompliant = (groundingMetadata.groundingChunks.isEmpty || groundingMetadata
.searchEntryPoint != nil)
if isCompliant {
HStack(alignment: .top, spacing: 8) {
VStack(alignment: .leading, spacing: 8) {
// Message text
ResponseTextView(message: message)

if !groundingMetadata.groundingChunks.isEmpty {
Divider()
// Source links
ForEach(0 ..< groundingMetadata.groundingChunks.count, id: \.self) { index in
if let webChunk = groundingMetadata.groundingChunks[index].web {
SourceLinkView(
title: webChunk.title ?? "Untitled Source",
uri: webChunk.uri
)
}
}
}
// Search suggestions
if let searchEntryPoint = groundingMetadata.searchEntryPoint {
Divider()
GoogleSearchSuggestionView(htmlString: searchEntryPoint.renderedContent)
.frame(height: 44)
.clipShape(RoundedRectangle(cornerRadius: 22))
}
}
}
.frame(maxWidth: .infinity, alignment: .leading)
}
}
}

/// A view for a single, clickable source link.
struct SourceLinkView: View {
let title: String
let uri: String?

var body: some View {
if let uri, let url = URL(string: uri) {
Link(destination: url) {
HStack(spacing: 4) {
Image(systemName: "link")
.font(.caption)
.foregroundColor(.secondary)
Text(title)
.font(.footnote)
.underline()
.lineLimit(1)
.multilineTextAlignment(.leading)
}
}
.buttonStyle(.plain)
}
}
}
53 changes: 34 additions & 19 deletions firebaseai/ChatExample/Views/MessageView.swift
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@

import MarkdownUI
import SwiftUI
import FirebaseAI

struct RoundedCorner: Shape {
var radius: CGFloat = .infinity
Expand Down Expand Up @@ -42,29 +43,43 @@ struct MessageContentView: View {
if message.pending {
BouncingDots()
} else {
Markdown(message.message)
.markdownTextStyle {
FontFamilyVariant(.normal)
FontSize(.em(0.85))
ForegroundColor(message.participant == .system ? Color(UIColor.label) : .white)
}
.markdownBlockStyle(\.codeBlock) { configuration in
configuration.label
.relativeLineSpacing(.em(0.25))
.markdownTextStyle {
FontFamilyVariant(.monospaced)
FontSize(.em(0.85))
ForegroundColor(Color(.label))
}
.padding()
.background(Color(.secondarySystemBackground))
.clipShape(RoundedRectangle(cornerRadius: 8))
.markdownMargin(top: .zero, bottom: .em(0.8))
}
// Grounded Response
if let groundingMetadata = message.groundingMetadata {
GroundedResponseView(message: message, groundingMetadata: groundingMetadata)
} else {
// Non-grounded response
ResponseTextView(message: message)
}
}
}
}

struct ResponseTextView: View {
var message: ChatMessage

var body: some View {
Markdown(message.message)
.markdownTextStyle {
FontFamilyVariant(.normal)
FontSize(.em(0.85))
ForegroundColor(message.participant == .system ? Color(UIColor.label) : .white)
}
.markdownBlockStyle(\.codeBlock) { configuration in
configuration.label
.relativeLineSpacing(.em(0.25))
.markdownTextStyle {
FontFamilyVariant(.monospaced)
FontSize(.em(0.85))
ForegroundColor(Color(.label))
}
.padding()
.background(Color(.secondarySystemBackground))
.clipShape(RoundedRectangle(cornerRadius: 8))
.markdownMargin(top: .zero, bottom: .em(0.8))
}
}
}

struct MessageView: View {
var message: ChatMessage

Expand Down
Loading