Error while setting up docker development environment

[daraul]

Following the Development Guidelines I was able to setup the Docker image easily, but the development image just gives me this error:

    ERROR: Command errored out with exit status 1:
     command: /usr/local/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/zipline/setup.py'"'"'; __file__='"'"'/zipline/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
         cwd: /zipline/
    Complete output (12 lines):
    running develop
    running egg_info
    writing requirements to zipline.egg-info/requires.txt
    writing entry points to zipline.egg-info/entry_points.txt
    writing top-level names to zipline.egg-info/top_level.txt
    writing zipline.egg-info/PKG-INFO
    writing dependency_links to zipline.egg-info/dependency_links.txt
    reading manifest file 'zipline.egg-info/SOURCES.txt'
    writing manifest file 'zipline.egg-info/SOURCES.txt'
    running build_ext
    cythoning zipline/assets/_assets.pyx to zipline/assets/_assets.c
    error: /zipline/.eggs/Cython-0.29.17-py3.5-linux-x86_64.egg/Cython/Utility/ObjectHandling.c

[samatix]

@daraul ,

I've just tried it again on the latest build and I didn't have any issue.

This said I would recommend changing the Dockerfile from at line 55 and put this instead as the system at RUN pip install -e . reinstalls different packages.

WORKDIR /ta-lib
ADD ./etc/requirements.in /requirements.in

RUN pip install 'numpy>=1.11.1,<2.0.0' \
  && pip install 'scipy>=0.17.1,<1.0.0' \
  && pip install 'pandas>=0.18.1,<=0.22' \
  && pip install -r /requirements.in \
  && ./configure --prefix=/usr \
  && make \
  && make install \
  && pip install TA-Lib \
  && pip install matplotlib \
  && pip install jupyter

If this works for you, I'll make a PR

[daraul]

Apologies for the super late reply.

To be clear, @samatix and I was able to build the Dockerfile, but according to the docker section of the development guidelines I should also build Dockerfile-dev, but that gives me this error:

Step 3/4 : RUN pip install -r etc/requirements_dev.txt -r etc/requirements_blaze.txt
 ---> Running in 67dd1e1c3c82
ERROR: Could not open requirements file: [Errno 2] No such file or directory: 'etc/requirements_dev.txt'
The command '/bin/sh -c pip install -r etc/requirements_dev.txt -r etc/requirements_blaze.txt' returned a non-zero code: 1

I have a PR that fixes that, but even with that fix I get this error:

Step 4/4 : RUN find . -type f -name '*.c' -exec rm {} + && pip install -e .[all]
 ---> Running in 2e33e436c41f
Obtaining file:///zipline
Requirement already satisfied: pip>=7.1.0 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (20.1)
Requirement already satisfied: setuptools>18.0 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (46.3.1)
Requirement already satisfied: Logbook>=0.12.5 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (1.5.3)
Requirement already satisfied: click>=4.0.0 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (7.1.2)
Requirement already satisfied: six>=1.10.0 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (1.14.0)
Requirement already satisfied: contextlib2>=0.4.0 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (0.6.0.post1)
Requirement already satisfied: python-interface>=1.5.3 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (1.6.0)
Requirement already satisfied: multipledispatch>=0.6.0 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (0.6.0)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (0.10.0)
Requirement already satisfied: numpy>=1.11.3 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (1.18.4)
Requirement already satisfied: pandas<=0.22,>=0.18.1 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (0.22.0)
Requirement already satisfied: pandas-datareader>=0.2.1 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (0.8.1)
Requirement already satisfied: scipy>=0.17.1 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (0.19.1)
Requirement already satisfied: patsy>=0.4.0 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (0.5.1)
Requirement already satisfied: statsmodels>=0.6.1 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (0.11.1)
Requirement already satisfied: empyrical>=0.5.0 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (0.5.3)
Requirement already satisfied: pytz>=2018.5 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (2020.1)
Requirement already satisfied: iso3166>=0.9 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (1.0.1)
Requirement already satisfied: trading-calendars>=1.6.1 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (1.11.7)
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (2.23.0)
Requirement already satisfied: sqlalchemy>=1.0.8 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (1.3.17)
Requirement already satisfied: alembic>=0.7.7 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (1.4.2)
Requirement already satisfied: bcolz>=0.12.1 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (1.2.1)
Requirement already satisfied: h5py>=2.7.1 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (2.10.0)
Requirement already satisfied: tables>=3.4.3 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (3.6.1)
Requirement already satisfied: lru-dict>=1.1.4 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (1.1.6)
Requirement already satisfied: intervaltree>=2.1.0 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (3.0.2)
Requirement already satisfied: bottleneck>=1.0.0 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (1.3.2)
Requirement already satisfied: networkx<2.0,>=1.9.1 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (1.11)
Requirement already satisfied: numexpr>=2.6.1 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (2.7.1)
Requirement already satisfied: iso4217>=1.6.20180829 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (1.6.20180829)
Requirement already satisfied: coverage>=4.0.3 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (5.1)
Requirement already satisfied: nose>=1.3.7 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (1.3.7)
Requirement already satisfied: nose-parameterized>=0.5.0 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (0.6.0)
Requirement already satisfied: nose-ignore-docstring>=0.2 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (0.2)
Requirement already satisfied: nose-timer>=0.5.0 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (1.0.0)
Requirement already satisfied: mock>=2.0.0 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (3.0.5)
Requirement already satisfied: testfixtures>=4.1.2 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (6.14.1)
Requirement already satisfied: flake8>=3.3.0 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (3.8.1)
Requirement already satisfied: matplotlib>=1.5.3 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (3.0.3)
Requirement already satisfied: responses>=0.9.0 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (0.10.14)
Requirement already satisfied: pip-tools>=4.3.0 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (5.1.2)
Requirement already satisfied: TA-Lib>=0.4.9 in /usr/local/lib/python3.5/site-packages (from zipline==1.3.0+608.g0b7a2945.dirty) (0.4.18)
Requirement already satisfied: python-dateutil>=2 in /usr/local/lib/python3.5/site-packages (from pandas<=0.22,>=0.18.1->zipline==1.3.0+608.g0b7a2945.dirty) (2.8.1)
Requirement already satisfied: lxml in /usr/local/lib/python3.5/site-packages (from pandas-datareader>=0.2.1->zipline==1.3.0+608.g0b7a2945.dirty) (4.5.0)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.5/site-packages (from requests>=2.9.1->zipline==1.3.0+608.g0b7a2945.dirty) (2.9)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.5/site-packages (from requests>=2.9.1->zipline==1.3.0+608.g0b7a2945.dirty) (2020.4.5.1)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.5/site-packages (from requests>=2.9.1->zipline==1.3.0+608.g0b7a2945.dirty) (3.0.4)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.5/site-packages (from requests>=2.9.1->zipline==1.3.0+608.g0b7a2945.dirty) (1.25.9)
Requirement already satisfied: python-editor>=0.3 in /usr/local/lib/python3.5/site-packages (from alembic>=0.7.7->zipline==1.3.0+608.g0b7a2945.dirty) (1.0.4)
Requirement already satisfied: Mako in /usr/local/lib/python3.5/site-packages (from alembic>=0.7.7->zipline==1.3.0+608.g0b7a2945.dirty) (1.1.2)
Requirement already satisfied: sortedcontainers<3.0,>=2.0 in /usr/local/lib/python3.5/site-packages (from intervaltree>=2.1.0->zipline==1.3.0+608.g0b7a2945.dirty) (2.1.0)
Requirement already satisfied: decorator>=3.4.0 in /usr/local/lib/python3.5/site-packages (from networkx<2.0,>=1.9.1->zipline==1.3.0+608.g0b7a2945.dirty) (4.4.2)
Requirement already satisfied: pyflakes<2.3.0,>=2.2.0 in /usr/local/lib/python3.5/site-packages (from flake8>=3.3.0->zipline==1.3.0+608.g0b7a2945.dirty) (2.2.0)
Requirement already satisfied: pycodestyle<2.7.0,>=2.6.0a1 in /usr/local/lib/python3.5/site-packages (from flake8>=3.3.0->zipline==1.3.0+608.g0b7a2945.dirty) (2.6.0)
Requirement already satisfied: mccabe<0.7.0,>=0.6.0 in /usr/local/lib/python3.5/site-packages (from flake8>=3.3.0->zipline==1.3.0+608.g0b7a2945.dirty) (0.6.1)
Requirement already satisfied: importlib-metadata; python_version < "3.8" in /usr/local/lib/python3.5/site-packages (from flake8>=3.3.0->zipline==1.3.0+608.g0b7a2945.dirty) (1.6.0)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.5/site-packages (from matplotlib>=1.5.3->zipline==1.3.0+608.g0b7a2945.dirty) (2.4.7)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.5/site-packages (from matplotlib>=1.5.3->zipline==1.3.0+608.g0b7a2945.dirty) (0.10.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.5/site-packages (from matplotlib>=1.5.3->zipline==1.3.0+608.g0b7a2945.dirty) (1.1.0)
Requirement already satisfied: MarkupSafe>=0.9.2 in /usr/local/lib/python3.5/site-packages (from Mako->alembic>=0.7.7->zipline==1.3.0+608.g0b7a2945.dirty) (1.1.1)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.5/site-packages (from importlib-metadata; python_version < "3.8"->flake8>=3.3.0->zipline==1.3.0+608.g0b7a2945.dirty) (1.2.0)
Installing collected packages: zipline
  Attempting uninstall: zipline
    Found existing installation: zipline 1.3.0+608.g0b7a2945.dirty
    Uninstalling zipline-1.3.0+608.g0b7a2945.dirty:
      Successfully uninstalled zipline-1.3.0+608.g0b7a2945.dirty
  Running setup.py develop for zipline
    ERROR: Command errored out with exit status 1:
     command: /usr/local/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/zipline/setup.py'"'"'; __file__='"'"'/zipline/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
         cwd: /zipline/
    Complete output (12 lines):
    running develop
    running egg_info
    writing entry points to zipline.egg-info/entry_points.txt
    writing requirements to zipline.egg-info/requires.txt
    writing zipline.egg-info/PKG-INFO
    writing top-level names to zipline.egg-info/top_level.txt
    writing dependency_links to zipline.egg-info/dependency_links.txt
    reading manifest file 'zipline.egg-info/SOURCES.txt'
    writing manifest file 'zipline.egg-info/SOURCES.txt'
    running build_ext
    cythoning zipline/assets/_assets.pyx to zipline/assets/_assets.c
    error: /zipline/.eggs/Cython-0.29.17-py3.5-linux-x86_64.egg/Cython/Utility/ObjectHandling.c
    ----------------------------------------
  Rolling back uninstall of zipline
  Moving to /usr/local/bin/zipline
   from /tmp/pip-uninstall-y0j3f6s0/zipline
  Moving to /usr/local/lib/python3.5/site-packages/zipline.egg-link
   from /tmp/pip-uninstall-ibms9_7a/zipline.egg-link
ERROR: Command errored out with exit status 1: /usr/local/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/zipline/setup.py'"'"'; __file__='"'"'/zipline/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.
The command '/bin/sh -c find . -type f -name '*.c' -exec rm {} + && pip install -e .[all]' returned a non-zero code: 1

I'm not sure how to address that.

[juehai]

@daraul please try my PR, I just had the same error when I built the dev image.

[daraul]

That works, @juehai! I'll close my PR in favor of yours, since it'd be redundant.