WindowsのDocker環境で python開発環境構築(pandas /matplotlib)

Windwosでローカル環境を汚さないPython開発環境の構築をします。環境構築にはDockerを使います。

以下のパッケージもインストールします。

  • numpy
  • scipy
  • scikit-learn
  • matplotlib
  • pandas

実際に環境を作るまでに、matplotlibのインストールで躓き、Dockerfileのビルドに失敗しましたが、下記の対策(後述)をすることでビルドが正常終了しました。

まずはビルドに成功したDockerfileを作成します。

Dockerfile

FROM alpine:3.7

RUN apk --update-cache \
    add musl \
    linux-headers \
    gcc \
    g++ \
    make \
    gfortran \
    openblas-dev \
    python3 \
    python3-dev \
    freetype-dev

RUN pip3 install --upgrade pip setuptools

# ADD pip_requirements.txt /tmp/pip_requirements.txt

RUN pip3 install numpy \
    scipy  \
    scikit-learn \ 
    matplotlib \ 
    'pandas<0.21.0'

#RUN \
# python3 -m pip install -r /tmp/pip_requirements.txt --no-cache-dir


ADD ./app /usr/src/app
WORKDIR /usr/src/app

以下のDockerファイルを格納したフォルダで、Dockerのビルドを行います。

Dockerfileビルド実行

docker build -t py-fx-pdf .
Step 1/5 : FROM alpine:3.7
 ---> 6d1ef012b567
Step 2/5 : RUN apk --update-cache     add musl     linux-headers     gcc     g++     make     gfortran     openblas-dev     python3     python3-dev     freetype-dev RUN pip3 install --upgrade pip setuptools
 ---> Running in 09616915f754
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/community/x86_64/APKINDEX.tar.gz
ERROR: unsatisfiable constraints:
  RUN (missing):
    required by: world[RUN]
  install (missing):
    required by: world[install]
  pip (missing):
    required by: world[pip]
  pip3 (missing):
    required by: world[pip3]
  setuptools (missing):
    required by: world[setuptools]
The command '/bin/sh -c apk --update-cache     add musl     linux-headers     gcc     g++     make     gfortran     openblas-dev     python3     python3-dev     freetype-dev RUN pip3 install --upgrade pip setuptools' returned a non-zero code: 5

D:\Git\bitbucket\py-fx-pdf>docker build -t py-fx-pdf .
Sending build context to Docker daemon  3.584kB
Step 1/6 : FROM alpine:3.7
 ---> 6d1ef012b567
Step 2/6 : RUN apk --update-cache     add musl     linux-headers     gcc     g++     make     gfortran     openblas-dev     python3     python3-dev     freetype-dev
 ---> Running in eded9e1fd3da
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/community/x86_64/APKINDEX.tar.gz
(1/40) Installing libbz2 (1.0.6-r6)
(2/40) Installing libpng (1.6.34-r1)
(3/40) Installing freetype (2.8.1-r4)
(4/40) Installing pkgconf (1.3.10-r0)
(5/40) Installing zlib-dev (1.2.11-r1)
(6/40) Installing libpng-dev (1.6.34-r1)
(7/40) Installing freetype-dev (2.8.1-r4)
(8/40) Installing libgcc (6.4.0-r5)
(9/40) Installing libstdc++ (6.4.0-r5)
(10/40) Installing binutils-libs (2.30-r1)
(11/40) Installing binutils (2.30-r1)
(12/40) Installing gmp (6.1.2-r1)
(13/40) Installing isl (0.18-r0)
(14/40) Installing libgomp (6.4.0-r5)
(15/40) Installing libatomic (6.4.0-r5)
(16/40) Installing mpfr3 (3.1.5-r1)
(17/40) Installing mpc1 (1.0.3-r1)
(18/40) Installing gcc (6.4.0-r5)
(19/40) Installing musl-dev (1.1.18-r3)
(20/40) Installing libc-dev (0.7.1-r0)
(21/40) Installing g++ (6.4.0-r5)
(22/40) Installing libquadmath (6.4.0-r5)
(23/40) Installing libgfortran (6.4.0-r5)
(24/40) Installing gfortran (6.4.0-r5)
(25/40) Installing linux-headers (4.4.6-r2)
(26/40) Installing make (4.2.1-r0)
(27/40) Installing openblas-ilp64 (0.2.19-r3)
(28/40) Installing openblas (0.2.19-r3)
(29/40) Installing openblas-dev (0.2.19-r3)
(30/40) Installing expat (2.2.5-r0)
(31/40) Installing libffi (3.2.1-r4)
(32/40) Installing gdbm (1.13-r1)
(33/40) Installing xz-libs (5.2.3-r1)
(34/40) Installing ncurses-terminfo-base (6.0_p20171125-r1)
(35/40) Installing ncurses-terminfo (6.0_p20171125-r1)
(36/40) Installing ncurses-libs (6.0_p20171125-r1)
(37/40) Installing readline (7.0.003-r0)
(38/40) Installing sqlite-libs (3.25.3-r0)
(39/40) Installing python3 (3.6.8-r0)
(40/40) Installing python3-dev (3.6.8-r0)
Executing busybox-1.27.2-r11.trigger
OK: 467 MiB in 53 packages
Removing intermediate container eded9e1fd3da
 ---> e3a11839783d
Step 3/6 : RUN pip3 install --upgrade pip setuptools
 ---> Running in 42e99c94dd9d
Collecting pip
  Downloading https://files.pythonhosted.org/packages/f9/fb/863012b13912709c13cf5cfdbfb304fa6c727659d6290438e1a88df9d848/pip-19.1-py2.py3-none-any.whl (1.4MB)
Collecting setuptools
  Downloading https://files.pythonhosted.org/packages/ec/51/f45cea425fd5cb0b0380f5b0f048ebc1da5b417e48d304838c02d6288a1e/setuptools-41.0.1-py2.py3-none-any.whl (575kB)
Installing collected packages: pip, setuptools
  Found existing installation: pip 18.1
    Uninstalling pip-18.1:
      Successfully uninstalled pip-18.1
  Found existing installation: setuptools 40.6.2
    Uninstalling setuptools-40.6.2:
      Successfully uninstalled setuptools-40.6.2
Successfully installed pip-19.1 setuptools-41.0.1
Removing intermediate container 42e99c94dd9d
 ---> 8fe1e99b27d9
Step 4/6 : RUN pip3 install numpy     scipy      scikit-learn     matplotlib     'pandas<0.21.0'
 ---> Running in c21ce10fc810
Collecting numpy
  Downloading https://files.pythonhosted.org/packages/93/48/956b9dcdddfcedb1705839280e02cbfeb2861ed5d7f59241210530867d5b/numpy-1.16.3.zip (5.1MB)
Collecting scipy
  Downloading https://files.pythonhosted.org/packages/a9/b4/5598a706697d1e2929eaf7fe68898ef4bea76e4950b9efbe1ef396b8813a/scipy-1.2.1.tar.gz (23.1MB)
Collecting scikit-learn
  Downloading https://files.pythonhosted.org/packages/f1/cb/3297656b9a3cce0cb60c691d568225fc025d47d3eb668b3c480211801a52/scikit-learn-0.20.3.tar.gz (11.8MB)
Collecting matplotlib
  Downloading https://files.pythonhosted.org/packages/26/04/8b381d5b166508cc258632b225adbafec49bbe69aa9a4fa1f1b461428313/matplotlib-3.0.3.tar.gz (36.6MB)
Collecting pandas<0.21.0
  Downloading https://files.pythonhosted.org/packages/ee/aa/90c06f249cf4408fa75135ad0df7d64c09cf74c9870733862491ed5f3a50/pandas-0.20.3.tar.gz (10.4MB)
Collecting cycler>=0.10 (from matplotlib)
  Downloading https://files.pythonhosted.org/packages/f7/d2/e07d3ebb2bd7af696440ce7e754c59dd546ffe1bbe732c8ab68b9c834e61/cycler-0.10.0-py2.py3-none-any.whl
Collecting kiwisolver>=1.0.1 (from matplotlib)
  Downloading https://files.pythonhosted.org/packages/16/e7/df58eb8868d183223692d2a62529a594f6414964a3ae93548467b146a24d/kiwisolver-1.1.0.tar.gz
Collecting pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 (from matplotlib)
  Downloading https://files.pythonhosted.org/packages/dd/d9/3ec19e966301a6e25769976999bd7bbe552016f0d32b577dc9d63d2e0c49/pyparsing-2.4.0-py2.py3-none-any.whl (62kB)
Collecting python-dateutil>=2.1 (from matplotlib)
  Downloading https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl (226kB)
Collecting pytz>=2011k (from pandas<0.21.0)
  Downloading https://files.pythonhosted.org/packages/3d/73/fe30c2daaaa0713420d0382b16fbb761409f532c56bdcc514bf7b6262bb6/pytz-2019.1-py2.py3-none-any.whl (510kB)
Collecting six (from cycler>=0.10->matplotlib)
  Downloading https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Requirement already satisfied: setuptools in /usr/lib/python3.6/site-packages (from kiwisolver>=1.0.1->matplotlib) (41.0.1)
Installing collected packages: numpy, scipy, scikit-learn, six, cycler, kiwisolver, pyparsing, python-dateutil, matplotlib, pytz, pandas
  Running setup.py install for numpy: started
    Running setup.py install for numpy: still running...
    Running setup.py install for numpy: still running...
    Running setup.py install for numpy: finished with status 'done'
  Running setup.py install for scipy: started
    Running setup.py install for scipy: still running...
    Running setup.py install for scipy: still running...
    Running setup.py install for scipy: still running...
    Running setup.py install for scipy: still running...
    Running setup.py install for scipy: still running...
    Running setup.py install for scipy: still running...
    Running setup.py install for scipy: still running...
    Running setup.py install for scipy: still running...
    Running setup.py install for scipy: still running...
    Running setup.py install for scipy: still running...
    Running setup.py install for scipy: still running...
    Running setup.py install for scipy: still running...
    Running setup.py install for scipy: finished with status 'done'
  Running setup.py install for scikit-learn: started
    Running setup.py install for scikit-learn: still running...
    Running setup.py install for scikit-learn: still running...
    Running setup.py install for scikit-learn: still running...
    Running setup.py install for scikit-learn: finished with status 'done'
  Running setup.py install for kiwisolver: started
    Running setup.py install for kiwisolver: finished with status 'done'
  Running setup.py install for matplotlib: started
    Running setup.py install for matplotlib: still running...
    Running setup.py install for matplotlib: finished with status 'done'
  Running setup.py install for pandas: started
    Running setup.py install for pandas: still running...
    Running setup.py install for pandas: still running...
    Running setup.py install for pandas: still running...
    Running setup.py install for pandas: still running...
    Running setup.py install for pandas: still running...
    Running setup.py install for pandas: finished with status 'done'
Successfully installed cycler-0.10.0 kiwisolver-1.1.0 matplotlib-3.0.3 numpy-1.16.3 pandas-0.20.3 pyparsing-2.4.0 python-dateutil-2.8.0 pytz-2019.1 scikit-learn-0.20.3 scipy-1.2.1 six-1.12.0
Removing intermediate container c21ce10fc810
 ---> 5ab83f06cdc9
Step 5/6 : ADD ./app /usr/src/app
ADD failed: stat /var/lib/docker/tmp/docker-builder011564994/app: no such file or directory

D:\Git\bitbucket\py-fx-pdf>

ハマったポイント

事象1 matplotlibのインストールでエラー

Downloading https://files.pythonhosted.org/packages/26/04/8b381d5b166508cc258632b225adbafec49bbe69aa9a4fa1f1b461428313/matplotlib-3.0.3.tar.gz (36.6MB)
ERROR: Complete output from command python setup.py egg_info:
ERROR: ============================================================================
Edit setup.cfg to change the build options
BUILDING MATPLOTLIB matplotlib: yes [3.0.3] python: yes [3.6.8 (default, Apr 22 2019, 10:24:07) [GCC 6.4.0]] platform: yes [linux] REQUIRED DEPENDENCIES AND EXTENSIONS numpy: yes [not found. pip may install it below.] install_requires: yes [handled by setuptools] libagg: yes [pkg-config information for 'libagg' could not be found. Using local copy.] freetype: no [The C/C++ header for freetype2 (ft2build.h) could not be found. You may need to install the development package.] png: no [pkg-config information for 'libpng' could not be found.] qhull: yes [pkg-config information for 'libqhull' could not be found. Using local copy.] OPTIONAL SUBPACKAGES sample_data: yes [installing] toolkits: yes [installing] tests: no [skipping due to configuration] toolkits_tests: no [skipping due to configuration] OPTIONAL BACKEND EXTENSIONS agg: yes [installing] tkagg: yes [installing; run-time loading from Python Tcl / Tk] macosx: no [Mac OS-X only] windowing: no [Microsoft Windows only] OPTIONAL PACKAGE DATA dlls: no [skipping due to configuration] ============================================================================ * The following required packages can not be built: * freetype, png ----------------------------------------
ERROR: Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-x7nt15d8/matplotlib

対応

setuptoolのバージョンアップをDockerfileに追加したらうまく動きました。

apkインストールにfreetype-devを追加

RUN pip3 install --upgrade pip setuptools
RUN apk  add freetype-dev

全部のインストール/ビルドに1時間弱くらいかかりました。

作成したDockerイメージでコンテナの起動

docker image ls

REPOSITORY TAG IMAGE ID CREATED SIZE
5ab83f06cdc9 38 minutes ago 841MB
480945cf63f2 2 hours ago 468MB
1cbd3f840117 2 hours ago 465MB
e06bb564ee93 2 hours ago 86MB

本当はコンテナに名前を付けておくといいのですが、忘れたのでタグ名で起動します。

docker container run -it 5ab83f06cdc9
 / #
/ # ls
bin dev etc home lib media mnt proc root run sbin srv sys tmp usr var
/ #

About: ken


コメントを残す

メールアドレスが公開されることはありません。 * が付いている欄は必須項目です

このサイトはスパムを低減するために Akismet を使っています。コメントデータの処理方法の詳細はこちらをご覧ください