Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor GitLab integration #1116

Merged
merged 29 commits into from
Nov 19, 2024
Merged

Refactor GitLab integration #1116

merged 29 commits into from
Nov 19, 2024

Conversation

Ndpnt
Copy link
Member

@Ndpnt Ndpnt commented Nov 6, 2024

Thanks @alessandrozago to the work done in #1098 to support GitLab as an alternative to GitHub for both issue reporting and dataset publishing. 🎉

This PR refactors the initial implementation to reduce the code footprint needed to support GitLab and improve overall consistency across the codebase.

@MattiSG, could you please review these changes first? Once reviewed, we’ll ask @alessandrozago to perform additional testing with GitLab to confirm everything works as expected and offer any feedback.

@Ndpnt Ndpnt requested a review from MattiSG November 6, 2024 15:26
Copy link
Member

@MattiSG MattiSG left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed the disjunction code and the metadata. I didn't review the GitHub and GitLab detailed code and didn't test it.

CHANGELOG.md Outdated Show resolved Hide resolved
CHANGELOG.md Show resolved Hide resolved
scripts/dataset/publish/index.js Show resolved Hide resolved
@Ndpnt Ndpnt force-pushed the add-gitlab-functionalities-update branch from e99a4de to e0c4cbf Compare November 7, 2024 16:04
@Ndpnt Ndpnt force-pushed the add-gitlab-functionalities-update branch from e0c4cbf to aed17f3 Compare November 7, 2024 16:06
@Ndpnt Ndpnt requested a review from MattiSG November 7, 2024 16:07
@alessandrozago
Copy link
Contributor

alessandrozago commented Nov 8, 2024

Dear all, thank you for your interest in this GitLab integration.
We would like to ask to restore the GitLab URL setting in the .env configuration file, since now the URL is hardcoded (mainly OTA_ENGINE_GITLAB_BASE_URL, while OTA_ENGINE_GITLAB**_API**_BASE_URL could be built by simply adding the '/api/v4' hardcoded).
This is useful because GitLab allows to set up custom instances, with different URLs/domains. This also our real use case with the repository: https://code.europa.eu/p2b .
Would you agree in putting the setting back in the code?
Please let us know. Thank you

My mistake, I saw that this was moved into the json configuration file, with "baseURL" and "apiBaseURL". Please ignore this comment, thank you.

@Ndpnt
Copy link
Member Author

Ndpnt commented Nov 12, 2024

Related documentation update OpenTermsArchive/docs#147

@Ndpnt
Copy link
Member Author

Ndpnt commented Nov 12, 2024

@alessandrozago Could you please confirm that it works as expected with GitLab?

@alessandrozago
Copy link
Contributor

@alessandrozago Could you please confirm that it works as expected with GitLab?

Hi @Ndpnt , we did some preliminary testing on the version of commit aed17f3, and the only issue we found at the moment is that the code still references the old parameter "versionsRepositoryURLGitLab" instead of the generic "versionsRepositoryURL".
This occurs in:

  • scripts/dataset/publish/gitlab/index.js:23 and :104
    I just saw the updated documentation that mentions only one parameter "versionsRepositoryURL".
    Please let me know if I should notify this issue in any other way.

When fixed we are available to test it again with also the latest changes from today's commits. Thank you

@Ndpnt
Copy link
Member Author

Ndpnt commented Nov 12, 2024

@alessandrozago Could you please confirm that it works as expected with GitLab?

Hi @Ndpnt , we did some preliminary testing on the version of commit aed17f3, and the only issue we found at the moment is that the code still references the old parameter "versionsRepositoryURLGitLab" instead of the generic "versionsRepositoryURL". This occurs in:

  • scripts/dataset/publish/gitlab/index.js:23 and :104
    I just saw the updated documentation that mentions only one parameter "versionsRepositoryURL".
    Please let me know if I should notify this issue in any other way.

When fixed we are available to test it again with also the latest changes from today's commits. Thank you

Thank you for your feedback! I’ve addressed the issue regarding the old parameter "versionsRepositoryURLGitLab" in the code. The references have been updated to use the generic "versionsRepositoryURL" as per the latest documentation.

Feel free to test again with today's latest commits. Let me know if anything else comes up!


expect(() => {
Reporter.validateConfiguration(repositories);
}).to.throw('Required configuration key "reporter.repositories.declarations" was not found; issues on the declarations repository cannot be created');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I recommend checking against just a significant subset of the string rather than the entire error message, to maximise tests maintainability.

}

/**
* Support for legacy config format where reporter configuration was nested under "githubIssues"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* Support for legacy config format where reporter configuration was nested under "githubIssues"
* Support for legacy config format where reporter configuration was nested under `githubIssues`

@Ndpnt
Copy link
Member Author

Ndpnt commented Nov 13, 2024

@alessandrozago Could you also review the documentation update PR to ensure it’s clear and reflects the recent GitLab updates?

@alessandrozago
Copy link
Contributor

@alessandrozago Could you please confirm that it works as expected with GitLab?

Hi @Ndpnt , we did some preliminary testing on the version of commit aed17f3, and the only issue we found at the moment is that the code still references the old parameter "versionsRepositoryURLGitLab" instead of the generic "versionsRepositoryURL". This occurs in:

  • scripts/dataset/publish/gitlab/index.js:23 and :104
    I just saw the updated documentation that mentions only one parameter "versionsRepositoryURL".
    Please let me know if I should notify this issue in any other way.

When fixed we are available to test it again with also the latest changes from today's commits. Thank you

Thank you for your feedback! I’ve addressed the issue regarding the old parameter "versionsRepositoryURLGitLab" in the code. The references have been updated to use the generic "versionsRepositoryURL" as per the latest documentation.

Feel free to test again with today's latest commits. Let me know if anything else comes up!

Hi @Ndpnt , thanks for the fix on the parameter.
We have tested the latest version of the code ( 0bdd637 ), and it seems to run correctly. We found one error analyzing the logs:
"Error in "Reporter" plugin: TypeError: this.reporter.clearCache is not a function" on /src/reporter/index.js:91 .
The cache was only introduced for github in OTA version 2.3.3, and is not present in the gitlab code we wrote. Aside from the error in the logs, everything else seems to work as expected.

Another small thing is that I left a logger line on debug level with no real indication for the final user ( /src/reporter/gitlab/index.js:265 ). I can remove it on my fork (or change the message to something more clear) but this pull request does not seem to be linked, so it should not update the code. Please let me know if I should do some action to help on this.

We did not find any other issues that can impact the functionalities. So, aside from the error on the cache mentioned above, on our side it is ok to proceed. Please let us know if you would like any other action from our side.

@Ndpnt
Copy link
Member Author

Ndpnt commented Nov 13, 2024

Thanks @alessandrozago for the valuable tests!

@Ndpnt Ndpnt requested a review from MattiSG November 13, 2024 16:35
scripts/dataset/publish/index.js Outdated Show resolved Hide resolved
Co-authored-by: Matti Schneider <[email protected]>
@Ndpnt Ndpnt merged commit 7c65a48 into main Nov 19, 2024
8 checks passed
@Ndpnt Ndpnt deleted the add-gitlab-functionalities-update branch November 19, 2024 08:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants