以前,我们仅将内部pip存储库用于源代码分发。展望未来,我们还希望托管轮子来完成两件事:
为(本地)开发人员机器和Alpine Docker环境提供我们自己的代码
为没有Alpine车轮的包裹创建车轮
不幸的是,使用不同库构建的轮子共享相同的工件名称,而第二个被pip存储库拒绝:
docker-compose.yml
version: '3'
services:
build-alpine:
build: alpine
image: build-alpine-wheels
volumes:
- $PWD/cython:/build
working_dir: /build
command: sh -c 'python setup.py bdist_wheel && twine upload --repository-url http://pypi:8080 -u admin -p admin dist/*'
build-debian:
build: debian
image: build-debian-wheels
volumes:
- $PWD/cython-debian:/build
working_dir: /build
command: bash -c 'sleep 10s && python setup.py bdist_wheel && twine upload --repository-url http://pypi:8080 -u admin -p admin dist/*'
pypi:
image: stevearc/pypicloud:1.0.2
volumes:
- $PWD/pypi:/etc/pypicloud/
alpine-test:
image: build-alpine-wheels
depends_on:
- build-alpine
command: sh -c 'while ping -c1 build-alpine &>/dev/null; do sleep 1; done; echo "build container finished" && pip install -i http://pypi:8080/pypi --trusted-host pypi cython && cython --version'
debian-test:
image: python:3.6
depends_on:
- build-debian
command: bash -c 'while ping -c1 build-debian &>/dev/null; do sleep 1; done; echo "build container finished" && pip install -i http://pypi:8080/pypi --trusted-host pypi cython && cython --version'
高山/ Dockerfile
FROM python:3.6-alpine
RUN apk add --update --no-cache build-base
RUN pip install --upgrade pip
RUN pip install twine
debian / Docker文件
FROM python:3.6-slim
RUN apt-get update && apt-get install -y \
build-essential \
&& rm -rf /var/lib/apt/lists/*
RUN pip install --upgrade pip
RUN pip install twine
pypi / config.ini
[app:main]
use = egg:pypicloud
pyramid.reload_templates = False
pyramid.debug_authorization = false
pyramid.debug_notfound = false
pyramid.debug_routematch = false
pyramid.default_locale_name = en
pypi.default_read =
everyone
pypi.default_write =
everyone
pypi.storage = file
storage.dir = %(here)s/packages
db.url = sqlite:///%(here)s/db.sqlite
auth.admins =
admin
user.admin = $6$rounds=535000$sFuRqMc5PbRccW1J$OBCsn8szlBwr4yPP243JPqomapgInRCUavv/p/UErt7I5FG4O6IGSHkH6H7ZPlrMXO1I8p5LYCQQxthgWZtxe1
# For beaker
session.encrypt_key = s0ETvuGG9Z8c6lK23Asxse4QyuVCsI2/NvGiNvvYl8E=
session.validate_key = fJvHQieaa0g3XsdgMF5ypE4pUf2tPpkbjueLQAAHN/k=
session.secure = False
session.invalidate_corrupt = true
###
# wsgi server configuration
###
[uwsgi]
paste = config:%p
paste-logger = %p
master = true
processes = 20
reload-mercy = 15
worker-reload-mercy = 15
max-requests = 1000
enable-threads = true
http = 0.0.0.0:8080
virtualenv = /env
###
# logging configuration
# http://docs.pylonsproject.org/projects/pyramid/en/latest/narr/logging.html
###
[loggers]
keys = root, botocore, pypicloud
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = INFO
handlers = console
[logger_pypicloud]
level = DEBUG
qualname = pypicloud
handlers =
[logger_botocore]
level = WARN
qualname = botocore
handlers =
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[formatter_generic]
format = %(levelname)s %(asctime)s [%(name)s] %(message)s
设置和执行
git clone https://github.com/cython/cython
git clone https://github.com/cython/cython cython-debian
docker-compose build
docker-compose up
最后,我希望两个测试容器都能够执行
cython --version
。适用于Alpine容器:alpine-test_1 | Collecting cython
alpine-test_1 | Downloading http://pypi:8080/api/package/cython/Cython-0.29.12-cp36-cp36m-linux_x86_64.whl (5.0MB)
alpine-test_1 | Installing collected packages: cython
alpine-test_1 | Successfully installed cython-0.29.12
alpine-test_1 | Cython version 0.29.12
但是不适用于Debian容器:
debian-test_1 | Downloading http://pypi:8080/api/package/cython/Cython-0.29.12-cp36-cp36m-linux_x86_64.whl (5.0MB)
debian-test_1 | Installing collected packages: cython
debian-test_1 | Successfully installed cython-0.29.12
debian-test_1 | Traceback (most recent call last):
debian-test_1 | File "/usr/local/bin/cython", line 6, in <module>
debian-test_1 | from Cython.Compiler.Main import setuptools_main
debian-test_1 | File "/usr/local/lib/python3.6/site-packages/Cython/Compiler/Main.py", line 28, in <module>
debian-test_1 | from .Scanning import PyrexScanner, FileSourceDescriptor
debian-test_1 | ImportError: libc.musl-x86_64.so.1: cannot open shared object file: No such file or directory
我发现这两种环境都试图拉动轮子特别令人好奇,因为有各种各样的软件包无法与Alpine配合使用(例如Pandas),在这种情况下,pip可以直接用于源代码分发。我想我在这方面也一定做错了。
因此,现在我想知道如何创建这些轮子,以便对于软件包的每个版本,可以在pip存储库中使用两个不同的轮子,并使pip自动下载并安装正确的轮子。
最佳答案
我建议完全不使用Alpine,通过多阶段构建(https://pythonspeed.com/articles/smaller-python-docker-images/)可以获得几乎一样小的图像,而musl不仅意味着没有二进制轮子。人们由于肌肉缺陷而产生了很多生产错误(Python崩溃,时间戳格式问题-请参见https://pythonspeed.com/articles/base-image-python-docker-images/以获得参考)。
大多数已知的musl链接都已修复,但是它的区别足够大,以至于仅获得较小的100MB图像似乎就不值得冒生产风险(更不用说您非常昂贵的开发者时间了!)。
关于python - 如何在同一pip存储库中维护glibc和libmusl Python轮子?,我们在Stack Overflow上找到一个类似的问题:https://stackoverflow.com/questions/57217763/