Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENH] Add docker build and test workflow WIP #18

Closed
wants to merge 24 commits into from

Conversation

viniciusdc
Copy link
Contributor

@viniciusdc viniciusdc commented Aug 4, 2022

closes #2

What does this implement/fix?

Note that these jobs only run for the changed files to avoid executing unnecessary actions

This pull request adds a new test workflow for new PR, where we will be able to:

  • Test the current conda environments before building it inside the docker images. this will help identify any errors before attempting the build (which is faster)
  • Lints the Docker code syntax for each image
  • Cache environment builds and Docker layers to avoid fresh builds on each change
  • Add in docker Pytest specialized workflow for each image (in progress)

Put an x in the boxes that apply

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds a feature)
  • Breaking change (fix or feature that would cause existing features to not work as expected)
  • Documentation Update
  • Code style update (formatting, renaming)
  • Refactoring (no functional changes, no API changes)
  • Build related changes
  • Other (please describe):

Testing

  • Did you test the pull request locally?
  • Did you add new tests? (wip)

Documentation

I will add some docs regarding this workflow once I run some preliminary tests.

Access-centered content checklist

Text styling

  • The content is written with plain language (where relevant).
  • If there are headers, they use the proper header tags (with only one level-one header: H1 or # in markdown).
  • All links describe where they link to (for example, check the Nebari website).
  • This content adheres to the Nebari style guides.

Non-text content

  • All content is represented as text (for example, images need alt text, and videos need captions or descriptive transcripts).
  • If there are emojis, there are not more than three in a row.
  • Don't use flashing GIFs or videos.
  • If the content were to be read as plain text, it still makes sense, and no information is missing.

Any other comments?

This is a workflow diagram of this new pipeline (which needs to be updated):

Blank diagram (1)

fix actions call

fix typ in req.txt

trigger ci

add missing shell spec

fix paths to actions

change image tag

change image tag again

change image tag again

change image tag again
fix dockerfile path error

missed -f file specification

update docker build context

undo context path changes
improve echo msg and update linting path

enh echo msg and update linting path

update step names

add build docker step back

trigger env builds with trivial changes
fix typo

mamba req shell specs

mamba req shell specs

test env caching

test conda as extra spec
test conda build caching

rm hash from conda-build cache key

trivial undo, to trigger cached build

trivial undo, to trigger cached build

test if caching is working

bump action version

trivial undo, to trigger cached build

update caching call

test if caching is working

invert cache order

test if caching is working

mv path-filter for conda builds to main
Try new docker caching

test saved layer cache

test saved layer cache

test saved layer cache

test saved layer cache

update docker build caching keys

list caches for branch

test saved layer cache

fix stringToken error

test saved layer cache

fix stringToken error

test saved layer cache

test saved layer cache

test saved layer cache

test saved layer cache
@viniciusdc
Copy link
Contributor Author

viniciusdc commented Aug 4, 2022

I will squash the tests and trivial commits later on, to avoid unnecessarily committing history

@viniciusdc
Copy link
Contributor Author

Also, the main reason behind adopting the Local registry during image building was based on this preliminary time comparison research.

@pavithraes pavithraes added the status: stale 🥖 This issue or PR has gone quiet for a long time label May 22, 2023
@pavithraes
Copy link
Member

@viniciusdc Shall we consider closing this PR? We can always re-open at a later date when we want to pick it back up. :)

@trallard
Copy link
Member

yes let's close for now and we can pick this up later

@pavithraes pavithraes closed this May 23, 2023
@Adam-D-Lewis Adam-D-Lewis deleted the fix-02-docker-testing branch June 29, 2023 21:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: CI/CD 👷🏽‍♀️ area: testing ✅ Items related to testing status: stale 🥖 This issue or PR has gone quiet for a long time type: enhancement 💅🏼 New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[DEV] - Add tests to the Docker images
3 participants