Skip to content

[Vertex AI] Test SDK with v1 API instead of v1beta #14345

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion FirebaseVertexAI/Sources/Constants.swift
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ import Foundation
/// Constants associated with the Vertex AI for Firebase SDK.
enum Constants {
/// The Vertex AI backend endpoint URL.
static let baseURL = "https://firebasevertexai.googleapis.com"
static let baseURL = "https://staging-firebasevertexai.sandbox.googleapis.com"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This change updates the base URL to a staging environment. Ensure that this is intentional and that the staging environment is appropriate for the intended testing. If this is only for testing purposes, consider adding a comment explaining the use of the staging URL.


/// The base reverse-DNS name for `NSError` or `CustomNSError` error domains.
///
Expand Down
2 changes: 1 addition & 1 deletion FirebaseVertexAI/Sources/GenerativeAIRequest.swift
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ public struct RequestOptions {
let timeout: TimeInterval

/// The API version to use in requests to the backend.
let apiVersion = "v1beta"
let apiVersion = "v1"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This change updates the API version to v1. Verify that all necessary API endpoints and data structures are compatible with this version.


/// Initializes a request options object.
///
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ final class IntegrationTests: XCTestCase {

// MARK: - Generate Content

func testGenerateContent() async throws {
func testGenerateContent_text() async throws {
let prompt = "Where is Google headquarters located? Answer with the city name only."

let response = try await model.generateContent(prompt)
Expand All @@ -75,6 +75,31 @@ final class IntegrationTests: XCTestCase {
XCTAssertEqual(text, "Mountain View")
}

func testGenerateContent_image_fileData_public() async throws {
let storageRef = storage.reference(withPath: "vertexai/public/green.png")
let fileData = FileDataPart(uri: storageRef.gsURI, mimeType: "image/png")

let response = try await model.generateContent(fileData, "What color is this?")

XCTAssertNotNil(response.text)
}

func testGenerateContent_imageData_requiresAuth_wrongUser_permissionDenied() async throws {
let userID = "3MjEzU6JIobWvHdCYHicnDMcPpQ2"
let storageRef = storage.reference(withPath: "vertexai/authenticated/user/\(userID)/pink.webp")

let fileData = FileDataPart(uri: storageRef.gsURI, mimeType: "image/webp")

do {
let response = try await model.generateContent(fileData, "What color is this?")
XCTFail("Expected to throw an error, got response: \(response)")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Enhance the XCTFail message to provide more context about the expected vs. actual results. For example, include the error that was caught.

Suggested change
XCTFail("Expected to throw an error, got response: \(response)")
XCTFail("Expected to throw an error, got response: \(response), error: \(error)")

} catch {
let errorDescription = String(describing: error)
XCTAssertTrue(errorDescription.contains("403"))
XCTAssertTrue(errorDescription.contains("The caller does not have permission"))
}
}

func testGenerateContent_appCheckNotConfigured_shouldFail() async throws {
let app = try FirebaseApp.defaultNamedCopy(name: TestAppCheckProviderFactory.notConfiguredName)
addTeardownBlock { await app.delete() }
Expand All @@ -83,13 +108,63 @@ final class IntegrationTests: XCTestCase {
let prompt = "Where is Google headquarters located? Answer with the city name only."

do {
_ = try await model.generateContent(prompt)
XCTFail("Expected a Firebase App Check error; none thrown.")
let response = try await model.generateContent(prompt)
XCTFail("Expected a Firebase App Check error, got response: \(response)")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Enhance the XCTFail message to provide more context about the expected vs. actual results. For example, include the error that was caught.

Suggested change
XCTFail("Expected a Firebase App Check error, got response: \(response)")
XCTFail("Expected a Firebase App Check error, got response: \(response), error: \(error)")

} catch let GenerateContentError.internalError(error) {
XCTAssertTrue(String(describing: error).contains("Firebase App Check token is invalid"))
}
}

// MARK: - Generate Content Streaming

func testGenerateContentStream_text() async throws {
let prompt = "Where is Google headquarters located? Answer with the city name only."

var text = ""
let contentStream = try model.generateContentStream(prompt)
for try await chunk in contentStream {
text += try XCTUnwrap(chunk.text)
}

text = text.trimmingCharacters(in: .whitespacesAndNewlines)
XCTAssertEqual(text, "Mountain View")
}

func testGenerateContentStream_image_fileData_public() async throws {
let storageRef = storage.reference(withPath: "vertexai/public/green.png")
let fileData = FileDataPart(uri: storageRef.gsURI, mimeType: "image/png")

var text = ""
let contentStream = try model.generateContentStream(fileData, "What color is this?")
for try await chunk in contentStream {
text += try XCTUnwrap(chunk.text)
}

text = text.trimmingCharacters(in: .whitespacesAndNewlines)
XCTAssertFalse(text.isEmpty)
}

func testGenContentStream_image_fileData_requiresAuth_wrongUser_permissionDenied() async throws {
let userID = "3MjEzU6JIobWvHdCYHicnDMcPpQ2"
let storageRef = storage.reference(withPath: "vertexai/authenticated/user/\(userID)/pink.webp")

let fileData = FileDataPart(uri: storageRef.gsURI, mimeType: "image/webp")

let contentStream = try model.generateContentStream(fileData, "What color is this?")

do {
var responses = [GenerateContentResponse]()
for try await response in contentStream {
responses.append(response)
}
XCTFail("Expected to throw an error, got response(s): \(responses)")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Enhance the XCTFail message to provide more context about the expected vs. actual results. For example, include the error that was caught.

Suggested change
XCTFail("Expected to throw an error, got response(s): \(responses)")
XCTFail("Expected to throw an error, got response(s): \(responses), error: \(error)")

} catch {
let errorDescription = String(describing: error)
XCTAssertTrue(errorDescription.contains("403"))
XCTAssertTrue(errorDescription.contains("The caller does not have permission"))
}
}

// MARK: - Count Tokens

func testCountTokens_text() async throws {
Expand Down Expand Up @@ -166,8 +241,8 @@ final class IntegrationTests: XCTestCase {
let fileData = FileDataPart(uri: storageRef.gsURI, mimeType: "image/webp")

do {
_ = try await model.countTokens(fileData)
XCTFail("Expected to throw an error.")
let response = try await model.countTokens(fileData)
XCTFail("Expected to throw an error, got response: \(response)")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Enhance the XCTFail message to provide more context about the expected vs. actual results. For example, include the error that was caught.

Suggested change
XCTFail("Expected to throw an error, got response: \(response)")
XCTFail("Expected to throw an error, got response: \(response), error: \(error)")

} catch {
let errorDescription = String(describing: error)
XCTAssertTrue(errorDescription.contains("403"))
Expand Down Expand Up @@ -229,8 +304,8 @@ final class IntegrationTests: XCTestCase {
let prompt = "Why is the sky blue?"

do {
_ = try await model.countTokens(prompt)
XCTFail("Expected a Firebase App Check error; none thrown.")
let response = try await model.countTokens(prompt)
XCTFail("Expected a Firebase App Check error, got response: \(response)")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Enhance the XCTFail message to provide more context about the expected vs. actual results. For example, include the error that was caught.

Suggested change
XCTFail("Expected a Firebase App Check error, got response: \(response)")
XCTFail("Expected a Firebase App Check error, got response: \(response), error: \(error)")

} catch {
XCTAssertTrue(String(describing: error).contains("Firebase App Check token is invalid"))
}
Expand Down
Loading