Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: pin Firefox version temporarily to 134 until artifacts tar.bz2 are available again (or a fix is provided) #30112

Merged
merged 2 commits into from
Feb 5, 2025

Conversation

seaona
Copy link
Contributor

@seaona seaona commented Feb 4, 2025

Description

Firefox specs started failing with the error:

<failure message="Unable to obtain browser driver.
        For more information on how to install drivers see
        https://www.selenium.dev/documentation/webdriver/troubleshooting/errors/driver_location/. Error: Error executing command for /home/circleci/project/node_modules/selenium-webdriver/bin/linux/selenium-manager with --browser,firefox,--output,json,--browser-path,/opt/firefox/firefox: Unsuccessful response (404 Not Found) for URL https://ftp.mozilla.org/pub/firefox/releases/135.0/linux-x86_64/en-US/firefox-135.0.tar.bz2
  (Ran on CircleCI Node 0 of 24, Job test-e2e-firefox)" type="Error"><![CDATA[Error: Unable to obtain browser driver.

The problem is that the artifact ending with tar.bz2 is not in present in the version 135.0, whereas it was always in previous versions, which we relied on.

https://ftp.mozilla.org/pub/firefox/releases/135.0/linux-x86_64/en-US/

image

From @itsyoboieltr :
It seems like firefox changed the way their releases work, so selenium cannot get the tar.

Context: https://consensys.slack.com/archives/CTQAGKY5V/p1738664570174839

Open in GitHub Codespaces

Related issues

Fixes:

Manual testing steps

I re-enabled FF specs and tested this in another branch:

  1. Check ci, how FF tests started running normally here

I canceled the job once I saw that the browser was successfully downloaded and specs started running

Screenshots/Recordings

Screenshot from 2025-02-05 08-23-28

Pre-merge author checklist

Pre-merge reviewer checklist

  • I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

Copy link
Contributor

github-actions bot commented Feb 4, 2025

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

@metamaskbot metamaskbot added the team-qa QA team label Feb 4, 2025
@seaona seaona marked this pull request as ready for review February 4, 2025 18:01
@seaona seaona requested a review from a team as a code owner February 4, 2025 18:01
@seaona seaona changed the title test: disable FF specs test: disable FF specs temporarily Feb 4, 2025
itsyoboieltr
itsyoboieltr previously approved these changes Feb 4, 2025
hjetpoluru
hjetpoluru previously approved these changes Feb 4, 2025
@metamaskbot
Copy link
Collaborator

Builds ready [23b6ff6]
Page Load Metrics (2040 ± 193 ms)
PlatformPageMetricMin (ms)Max (ms)Average (ms)StandardDeviation (ms)MarginOfError (ms)
ChromeHomefirstPaint50830001995526252
domContentLoaded150929481988403193
load152129612040401193
domInteractive27194543718
backgroundConnect9179564421
firstReactRender16103512612
getState4183333919
initialActions012131
loadScripts110922401478333160
setupStore790222010
uiStartup169133072428457220
Bundle size diffs
  • background: 0 Bytes (0.00%)
  • ui: 0 Bytes (0.00%)
  • common: 0 Bytes (0.00%)

DDDDDanica
DDDDDanica previously approved these changes Feb 4, 2025
@DDDDDanica
Copy link
Contributor

@MetaMask/policy-reviewers There's no fundamental changes in policy, it is just for disabling e2e tests in ff as a temporary solution. Thanks for reviewing 🙏🏻

chloeYue
chloeYue previously approved these changes Feb 5, 2025
<!--
Please submit this PR as a draft initially.
Do not mark it as "Ready for review" until the template has been
completely filled out, and PR status checks have passed at least once.
-->

## **Description**

<!--
Write a short description of the changes included in this pull request,
also include relevant motivation and context. Have in mind the following
questions:
1. What is the reason for the change?
2. What is the improvement/solution?
-->

[![Open in GitHub
Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/MetaMask/metamask-extension/pull/30122?quickstart=1)

## **Related issues**

Fixes:

## **Manual testing steps**

1. Go to this page...
2.
3.

## **Screenshots/Recordings**

<!-- If applicable, add screenshots and/or recordings to visualize the
before and after of your change. -->

### **Before**

<!-- [screenshots/recordings] -->

### **After**

<!-- [screenshots/recordings] -->

## **Pre-merge author checklist**

- [ ] I've followed [MetaMask Contributor
Docs](https://github.com/MetaMask/contributor-docs) and [MetaMask
Extension Coding
Standards](https://github.com/MetaMask/metamask-extension/blob/main/.github/guidelines/CODING_GUIDELINES.md).
- [ ] I've completed the PR template to the best of my ability
- [ ] I’ve included tests if applicable
- [ ] I’ve documented my code using [JSDoc](https://jsdoc.app/) format
if applicable
- [ ] I’ve applied the right labels on the PR (see [labeling
guidelines](https://github.com/MetaMask/metamask-extension/blob/main/.github/guidelines/LABELING_GUIDELINES.md)).
Not required for external contributors.

## **Pre-merge reviewer checklist**

- [ ] I've manually tested the PR (e.g. pull and build branch, run the
app, test code being changed).
- [ ] I confirm that this PR addresses all acceptance criteria described
in the ticket it closes and includes the necessary testing evidence such
as recordings and or screenshots.
@seaona seaona changed the title test: disable FF specs temporarily test: pin Firefox version temporarily to 134 until artifacts tar.bz2 are available again (or a fix is provided) Feb 5, 2025
Copy link
Contributor

@chloeYue chloeYue left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@metamaskbot
Copy link
Collaborator

Builds ready [fd17af7]
Page Load Metrics (1742 ± 54 ms)
PlatformPageMetricMin (ms)Max (ms)Average (ms)StandardDeviation (ms)MarginOfError (ms)
ChromeHomefirstPaint14732021175112158
domContentLoaded14622010171511957
load14712020174211354
domInteractive257935147
backgroundConnect116233189
firstReactRender16104402713
getState55715178
initialActions01000
loadScripts1056151212509948
setupStore86919199
uiStartup16982470200217885
Bundle size diffs
  • background: 0 Bytes (0.00%)
  • ui: 0 Bytes (0.00%)
  • common: 0 Bytes (0.00%)

@seaona seaona added this pull request to the merge queue Feb 5, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Feb 5, 2025
@seaona seaona added this pull request to the merge queue Feb 5, 2025
Merged via the queue into main with commit 8835cf5 Feb 5, 2025
96 checks passed
@seaona seaona deleted the ff-disable-ci branch February 5, 2025 10:05
@github-actions github-actions bot locked and limited conversation to collaborators Feb 5, 2025
@metamaskbot metamaskbot added the release-12.13.0 Issue or pull request that will be included in release 12.13.0 label Feb 5, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
release-12.13.0 Issue or pull request that will be included in release 12.13.0 team-qa QA team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants