Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

.gitlab-ci.yml #134

Open
wants to merge 133 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
133 commits
Select commit Hold shift + click to select a range
b7be785
Add option ci_https_proxy and use it to define .gitlab-ci.yml so that…
dHannasch Aug 30, 2019
9f519ad
Take proxy PEM as a GitLab File rather than require it to be included…
dHannasch Oct 17, 2019
73adfad
merge in master
dHannasch Oct 17, 2019
1303620
Rename repo_hosting to repo_hosting_domain.
dHannasch Oct 17, 2019
cd6ba4d
ETC_ENVIRONMENT_LOCATION and SERVERS_TO_WHITELIST_FOR_SSH
dHannasch Oct 17, 2019
14f4819
comment
dHannasch Oct 17, 2019
c98606e
Allow PROXY_CA_PEM to be the certificate rather than a file name.
dHannasch Oct 18, 2019
d598c05
No need to know at the time you create the repo whether you'll need a…
dHannasch Oct 18, 2019
3111389
The quotes are very important here.
dHannasch Oct 18, 2019
aaa088d
Load proxy variables first so we can apk add openssh-client.
dHannasch Oct 18, 2019
67062a7
This potentially leads to a problem if we need SSH to access the ETC_…
dHannasch Oct 18, 2019
b65b488
Uh, that should have been ||.
dHannasch Oct 18, 2019
1fdf614
Use python -m for sphinx and pytest to enable tox --sitepackages.
dHannasch Oct 21, 2019
dddf644
We need to pip install before building the docs to use autodoc.
dHannasch Oct 25, 2019
41008ab
double-check that bundled.pem exists
dHannasch Nov 26, 2019
6c1eaf4
Include PWD so that REQUESTS_CA_BUNDLE can still be found from anothe…
dHannasch Nov 26, 2019
0e906d4
image: dahanna/python:alpine-miniconda-3.7-git-tox
dHannasch Dec 2, 2019
4266a7a
meta.yaml
dHannasch Dec 2, 2019
e59ae98
get_version_for_conda_meta_yaml ought to be able to do something more…
dHannasch Dec 2, 2019
f44ee46
flake8
dHannasch Dec 2, 2019
3ed9980
cookiecutter.distribution_name
dHannasch Dec 2, 2019
b4e2a52
flake8
dHannasch Dec 2, 2019
334ddd6
include conda.recipe/meta.yaml
dHannasch Dec 3, 2019
e459487
pip install --upgrade sphinx
dHannasch Jan 30, 2020
7c1d076
|| curl --verbose --output environment.sh
dHannasch Feb 3, 2020
0dde59d
If someone changes to using a Docker image that does not have conda, …
dHannasch Mar 5, 2020
8169406
pip install --upgrade sphinx
dHannasch Mar 13, 2020
8124d34
python3 --version
dHannasch Apr 7, 2020
461b13c
pip install --upgrade sphinx
dHannasch Apr 13, 2020
85e727d
allow not available
dHannasch Jun 12, 2020
8f41b69
include all in MANIFEST
dHannasch Jun 18, 2020
0d0c21e
skip test-env if it's not found
dHannasch Jul 2, 2020
275cad4
If test-env not found, skip building conda package.
dHannasch Jul 2, 2020
23897f0
Add a couple of commands to quickly print the operating system the te…
dHannasch Jul 4, 2020
8171f8f
Rename to SSH_PRIVATE_DEPLOY_KEY for clarity.
dHannasch Jul 5, 2020
e633b8f
source environment.sh || . ./environment.sh
dHannasch Jul 5, 2020
70beeb3
Ignore any changes to PATH from a loaded environment file.
dHannasch Jul 5, 2020
855d837
skip installing openssh
dHannasch Jul 6, 2020
ea39cd1
install tox if don't already have it
dHannasch Jul 9, 2020
bee53e5
install tox by default but don't need it for docs
dHannasch Jul 9, 2020
d4fdb13
There is no need to devote a valuable cookiecutter variable just to t…
dHannasch Jul 23, 2020
7d13acb
Make get_version_for_conda_meta_yaml appear in setup.py depending on …
dHannasch Jul 24, 2020
14c72b6
Allow tox --sitepackages.
dHannasch Jul 24, 2020
e92874d
timing
dHannasch Jul 24, 2020
cca5678
Specify dependencies for test job.
dHannasch Jul 24, 2020
36e9198
Time building the conda channel.
dHannasch Jul 24, 2020
cf19725
build docker image only if variable set
dHannasch Jul 24, 2020
52103ae
include .dockerignore Dockerfile
dHannasch Jul 24, 2020
a12ce56
Add option to specify whether the docs require the package installed.
dHannasch Jul 24, 2020
b49d815
conda package
dHannasch Jul 24, 2020
6b81816
ignore more things
dHannasch Jul 24, 2020
9bda191
only build if var is present
dHannasch Jul 24, 2020
0f77982
without != null it also checks for nonempty
dHannasch Jul 24, 2020
853e77a
print warning
dHannasch Jul 24, 2020
8e51e4d
try --proxy off
dHannasch Jul 24, 2020
a3e3554
Merging TensorFlow-style assembler.py for Dockerfiles into testenv bo…
dHannasch Jul 24, 2020
692e35b
manually run assembler
dHannasch Jul 24, 2020
bace2f8
smuggle in key as FTP_PROXY
dHannasch Jul 25, 2020
bc2760e
include them in MANIFEST for now
dHannasch Jul 25, 2020
95728e0
flake8
dHannasch Jul 25, 2020
8945a42
delete line
dHannasch Jul 25, 2020
812b937
CMD
dHannasch Jul 25, 2020
508843d
comment
dHannasch Jul 25, 2020
68ed902
EXPOSE is not strictly necessary
dHannasch Jul 25, 2020
f6e7786
keep build_with_kaniko in separate file for ease of keeping in sync s…
dHannasch Jul 25, 2020
e093e42
MANIFEST
dHannasch Jul 25, 2020
83f9e07
move pygments upgrade higher
dHannasch Jul 26, 2020
df5f111
nss
dHannasch Jul 28, 2020
544fbba
wget --version
dHannasch Jul 28, 2020
f189959
skip if apk not found
dHannasch Jul 28, 2020
65c3f06
skip apk
dHannasch Jul 28, 2020
b43b97f
ls -l /home/david/.pki/nssdb/
dHannasch Jul 28, 2020
6b3ff2e
no --help?
dHannasch Jul 28, 2020
93bf30e
time kaniko executor
dHannasch Jul 29, 2020
76413ab
view help
dHannasch Jul 29, 2020
f1c8d27
Move get_version_for_conda_meta_yaml to get out of the way of the C e…
dHannasch Aug 6, 2020
f3ad127
No idea why there's a stray extra c_extension_support
dHannasch Aug 6, 2020
b93a447
extra space for flake8
dHannasch Aug 6, 2020
7f7d8d7
echo -e is not supported on all platforms, so we have to enter and ex…
dHannasch Aug 9, 2020
ab0a797
Hmm...maybe keep assembler.py, but have assembler.py safe to call if …
dHannasch Aug 9, 2020
cd2d000
isort
dHannasch Aug 9, 2020
f96eb8e
tags: docker
dHannasch Aug 16, 2020
f51831e
sed -i -e 's/https/http/' /etc/apk/repositories
dHannasch Sep 24, 2020
2f4932a
image: fkrull/multi-python:bionic
dHannasch Oct 13, 2020
daf1a2e
For GitLab Pages, the artifact path *must* be public.
dHannasch Oct 13, 2020
0165986
fix DeprecationWarning
dHannasch Oct 15, 2020
abcd4ab
upload-to-PyPI
dHannasch Oct 16, 2020
10d2d20
Use PYPI_URL
dHannasch Oct 16, 2020
34ca778
start trying to use deploy key to test gitlab-ci
dHannasch Oct 17, 2020
9702c1e
gitlab-ci-yml-components/for-python
dHannasch Oct 19, 2020
a269c76
safe_sed
dHannasch Oct 19, 2020
c02c859
multiple lines?
dHannasch Oct 19, 2020
a5657d7
multiple lines?
dHannasch Oct 19, 2020
36bca32
give up on safe_sed for now
dHannasch Oct 19, 2020
55ee4f3
colon?
dHannasch Oct 19, 2020
8a53b22
echo multiple times
dHannasch Oct 19, 2020
6c7e6e4
printf?
dHannasch Oct 19, 2020
88d5873
print \n
dHannasch Oct 19, 2020
be6b2cd
cookiecut
dHannasch Oct 19, 2020
de0fa70
git commit
dHannasch Oct 19, 2020
d1b1141
give up on safe_sed for now
dHannasch Oct 19, 2020
230aa42
git diff
dHannasch Oct 19, 2020
dbb42f0
git reset --hard
dHannasch Oct 19, 2020
70cd00c
git clone
dHannasch Oct 19, 2020
d480bf6
git diff
dHannasch Oct 19, 2020
6bccb6a
GITLAB_USER_LOGIN
dHannasch Oct 19, 2020
e1d0e8c
echo does not take \n
dHannasch Oct 19, 2020
1f91bbe
keep py27
dHannasch Oct 19, 2020
fb16594
\/
dHannasch Oct 19, 2020
288aba0
pypy3 before pypy
dHannasch Oct 19, 2020
0cd014a
make branch of same name
dHannasch Oct 19, 2020
8e949ba
already in python-nameless
dHannasch Oct 19, 2020
042dd3a
something about being already in python-nameless makes the gits inter…
dHannasch Oct 19, 2020
d0f1a5c
cd back into python-nameless
dHannasch Oct 19, 2020
3461546
git status
dHannasch Oct 19, 2020
9b50f8a
git push
dHannasch Oct 19, 2020
5202bf3
quote colon
dHannasch Oct 19, 2020
4175754
can we checkout without screwing up git?
dHannasch Oct 19, 2020
52f3c6d
we cannot be sure that branch already exists
dHannasch Oct 19, 2020
d74761b
--allow-empty
dHannasch Oct 19, 2020
b678cc4
Set to verbose before running the opaque shell script.
dHannasch Nov 1, 2020
b2132fe
Allow environment.sh to already be present.
dHannasch Nov 1, 2020
a1078cb
Pass BASE_IMAGE only if provided.
dHannasch Nov 2, 2020
4092064
Bail out on errors. Prevents strange failure modes.
ionelmc Nov 3, 2020
de2cdaf
Add even more error checking.
ionelmc Nov 3, 2020
7bf3ca6
Allow this to fail cause it's a cleanup thing.
ionelmc Nov 3, 2020
de8230d
Merge branch 'master' into gitlab-ci-yml
dHannasch Nov 3, 2020
3d3ec2e
If the conda-channel is not built, don't just give a 404, put a dummy…
dHannasch Nov 5, 2020
7b737da
conda not built every time
dHannasch Nov 5, 2020
9bcc749
The new version of miniconda makes empty channels that are pretty siz…
dHannasch Nov 6, 2020
55ba0af
No need to apt-get update if ssh is already installed.
dHannasch Nov 18, 2020
4dc1171
blank line
dHannasch Nov 18, 2020
e3c40ae
Add hostname -i to no_proxy
dHannasch Nov 20, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 73 additions & 0 deletions .gitlab-ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@

image: dahanna/python-alpine-package:tox-alpine

# https://docs.gitlab.com/ee/ci/yaml/includes.html#re-using-a-before_script-template
include:
- project: 'gitlab-ci-yml-components/for-python'
file: '.before_script.yml'


test:
tags:
- docker
stage: test
script:
# If using an image that does not include tox, we will
# need to pip install tox here.
- pip install --upgrade wheel setuptools tox cookiecutter bumpversion

- git --version || echo "git is not installed."
- python --version
- python2 --version || echo "python2 is not installed."
- virtualenv --version || echo "virtualenv is not installed."
- pip --version
# When testing locally, we might not want to set tox sitepackages=true,
# because the local machine might have all kinds of weird things in the
# environment. But for continuous integration, we do want sitepackages=true,
# because it allows us to use a Docker image with some packages already
# installed to accelerate testing.
- tox --version
- uname --all
- lsb_release --all || echo "lsb_release is not supported on this host."

- printf " repo_hosting_domain" >> gitlab.cookiecutterrc
- 'printf ":" >> gitlab.cookiecutterrc'
- printf " '$repo_hosting_domain'\n" >> gitlab.cookiecutterrc
- echo "might be able to extract it from $CI_SERVER_URL"
- 'printf " repo_username:" >> gitlab.cookiecutterrc'
- echo " '$GITLAB_USER_LOGIN'" >> gitlab.cookiecutterrc
- cat gitlab.cookiecutterrc
# - git clone -b $CI_COMMIT_BRANCH --single-branch $PYTHON_NAMELESS_REPO
# we cannot be sure that branch already exists
- git clone $PYTHON_NAMELESS_REPO
- cd python-nameless
- git checkout $CI_COMMIT_BRANCH || git checkout -b $CI_COMMIT_BRANCH
- cd ..
- cookiecutter --help
- cookiecutter --overwrite-if-exists --no-input --config-file=gitlab.cookiecutterrc .
- cd python-nameless
- 'sed --in-place "s/fkrull\/multi-python:bionic/dahanna\/python-alpine-package:tox-alpine/" .gitlab-ci.yml'
- git status
- git diff
- git status
- git add --all
- git config user.email "[email protected]"
- git config user.name "Your Name"
- git commit --allow-empty -m "Regenerated from cookiecutter."
- git push -u origin $CI_COMMIT_BRANCH
- sed --help
- |
for name in py35 py36 py27 pypy3 pypy; do
for env in $name ${name}-cover ${name}-nocov; do
sed --in-place "s/,$env,/,/" tox.ini
sed --in-place "s/$env,//" tox.ini
sed --in-place "s/,$env//" tox.ini
done
done
- git diff
- git reset --hard
- start_tox=$(date +%s)
# - tox --sitepackages
- echo "tox tests took $(( $(date +%s) - start_tox)) seconds"
- echo "Everything after pulling the Docker image took $(( $(date +%s) - right_after_pull_docker_image)) seconds total"

15 changes: 15 additions & 0 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -347,6 +347,21 @@ You will be asked for these fields:
<https://pypi.org/project/sphinx_py3doc_enhanced_theme>` for a responsive theme based on
the Python 3 documentation.

* - ``docs_require_package``
- .. code:: python

"no"
- Set to ``"yes"`` if you want to enable using your package when building documentation.
Leave as ``"no"`` if you want your docs to build as quickly as possible, to show live updates.
You *can* still use doctests and specify skipif on all of them so that doctests will be run by
tox but passed through when just building the documentation.

Your documentation will absolutely require your package if, for example, your package produces
visualizations that you want to include in the documentation. Since those are large binary files,
you don't want to add them to git. So you'll generate them on-the-fly when the documentation is
built.
You should also set this to ``"yes"`` if you want to use GitLab Pages to host your conda package.

* - ``sphinx_doctest``
- .. code:: python

Expand Down
1 change: 1 addition & 0 deletions cookiecutter.json
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@
"codeclimate" : ["no", "yes"],
"sphinx_docs": ["yes", "no"],
"sphinx_theme": ["sphinx-rtd-theme", "sphinx-py3doc-enhanced-theme"],
"docs_require_package": ["no", "yes"],
"sphinx_doctest": ["no", "yes"],
"sphinx_docs_hosting": "https://{{ cookiecutter.repo_name|replace('.', '') }}.readthedocs.io/",
"travis": ["yes", "no"],
Expand Down
42 changes: 42 additions & 0 deletions gitlab.cookiecutterrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# This file exists so you can easily regenerate your project.
#
# `cookiepatcher` is a convenient shim around `cookiecutter`
# for regenerating projects (it will generate a .cookiecutterrc
# automatically for any template). To use it:
#
# pip install cookiepatcher
# cookiepatcher gh:ionelmc/cookiecutter-pylibrary project-path
#
# See:
# https://pypi.org/project/cookiepatcher
#
# Alternatively, you can run:
#
# cookiecutter --overwrite-if-exists --config-file=project-path/.cookiecutterrc gh:ionelmc/cookiecutter-pylibrary

default_context:

_template: 'cookiecutter-pylibrary'
allow_tests_inside_package: 'yes'
appveyor: 'no'
codeclimate: 'no'
codecov: 'no'
command_line_interface: 'click'
coveralls: 'no'
email: '[email protected]'
full_name: 'David A. Hannasch'
landscape: 'no'
license: 'no'
linter: 'flake8'
pre_commit: 'yes'
pypi_badge: 'no'
pypi_disable_upload: 'no'
repo_hosting: 'other domain not listed'
requiresio: 'no'
scrutinizer: 'no'
setup_py_uses_setuptools_scm: 'yes'
sphinx_docs: 'yes'
sphinx_docs_hosting: 'https://www.google.com'
travis: 'no'
travis_osx: 'no'
website: 'https://www.google.com'
11 changes: 11 additions & 0 deletions hooks/post_gen_project.py
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,16 @@ def unlink_if_exists(path):
unlink_if_exists('.travis.yml')
{% endif %}

{%- if 'gitlab' not in cookiecutter.repo_hosting_domain %}
os.unlink('.gitlab-ci.yml')
os.unlink('.before_script.yml')
os.unlink('.build_with_kaniko.yml')
os.unlink('.dockerignore')
os.unlink('dockerfiles/slim.Dockerfile')
os.unlink('dockerfiles/test.Dockerfile')
os.unlink('conda.recipe/meta.yaml')
{% endif %}

{%- if cookiecutter.repo_hosting == 'no' %}
os.unlink('CONTRIBUTING.rst')
{% endif %}
Expand Down Expand Up @@ -130,6 +140,7 @@ def unlink_if_exists(path):
subprocess.check_call([sys.executable, '-mtox', '-e', 'bootstrap'])
except Exception:
subprocess.check_call([sys.executable, join('ci', 'bootstrap.py')])
subprocess.check_call([sys.executable, join('ci', 'assembler.py')])

print("""
################################################################################
Expand Down
Loading