Skip to content

checkScreen does not register any changes on iOS using Browserstack #1038

@BosseKarat

Description

@BosseKarat

Environment (please complete the following information):

  • Node.js version: 20.*
  • NPM version: 10.1.0
    "@wdio/browserstack-service": "^9.20.0",
    "@wdio/cli": "^9.20.0",
    "@wdio/local-runner": "^9.20.0",
    "@wdio/mocha-framework": "^9.20.0",
    "@wdio/spec-reporter": "^9.20.0",
    "@wdio/visual-service": "^9.0.0",

Config of WebdriverIO + @wdio/visual-service

capabilities: [
    {
      "bstack:options": {
        osVersion: "18",
        deviceName: 'iPad 9th',
        projectName: "ProjectName",
        buildName: "BuildName",
        sessionName: "SessionName",
        appiumVersion: "2.15.0",
        realMobile: "true",
        debug: "true",
        networkLogs: "true",
        consoleLogs: "verbose",
      },
      browserName: "safari",
      platformName: "iOS",
    } as Capabilities.BrowserStackCapabilities,

capabilities: [
    {
      "bstack:options": {
        osVersion: "26",
        deviceName: "iPhone 15",
        projectName: "ProjectName",
        buildName: "BuildName",
        sessionName: "SessionName",
        appiumVersion: "2.15.0",
        realMobile: "true",
        debug: "true",
        networkLogs: "true",
        consoleLogs: "verbose",
      },
      browserName: "safari",
      platformName: "iOS",
    } as Capabilities.BrowserStackCapabilities,

Describe the bug
checkScreen on iOS devices does not notice any changes at all. I have tried to manually modify the baseline images but checkScreen always marks the tests as passed.

To Reproduce
Steps to reproduce the behavior:

Run this test

    it('checkscreen test', async function () {
        await browser.url('https://webdriver.io/docs/api');

        await expect(
            await browser.checkScreen('A TEST CHECKSCREEN', {
            })
        ).toBeLessThanOrEqual(0.1);
    });

Modify the baseline to trigger the test to fail. You can even modify the url to a completely different page, the test will still pass.

The diff folder is empty, actual image and baseline image should differ but the test is still marked as passed.

Expected behavior
checkScreen should detect differences when comparing actual and baseline image.

Log
I tried scanning the output for any additional information but I could not find anything related to this problem. No errors or anamolies

Additional context
I have had this issue which you fixed in v.6.2.1. I don't know if you can access this thread on discord but this is the duscussion we had last time: https://discord.com/channels/1097401827202445382/1295716403906740254/1295716735336579155

And this was your solution:

Image

Here is the PR that you are referencing: #568

I have tried setting enableLegacyScreenshotMethod: true and 'appium:nativeWebScreenshot': true/false (false is default I think) but nothing helped. I have also tried v.8.0.4, no luck there.

As always, thank you for this amazing tool. Let me know if I can assist you in any way :)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions