python模块导入 - 相对路径问题

问题描述

我正在用 python 2.7 开发我自己的模块.它位于 ~/Development/.../myModule 而不是 /usr/lib/python2.7/dist-packages 或 /usr/lib/python2.7/站点包.内部结构为:

I'm developing my own module in python 2.7. It resides in ~/Development/.../myModule instead of /usr/lib/python2.7/dist-packages or /usr/lib/python2.7/site-packages. The internal structure is:

/project-root-dir
  /server
    __init__.py
    service.py
    http.py
  /client
    __init__.py
    client.py

client/client.py 包含 PyCachedClient 类.我遇到了导入问题:

client/client.py includes PyCachedClient class. I'm having import problems:

project-root-dir$ python
Python 2.7.2+ (default, Jul 20 2012, 22:12:53)
[GCC 4.6.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from server import http
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "server/http.py", line 9, in <module>
    from client import PyCachedClient
ImportError: cannot import name PyCachedClient

我没有将 PythonPath 设置为包含我的 project-root-dir，因此当 server.http 尝试包含 client.PyCachedClient 时，它会尝试从相对路径加载它并失败.我的问题是 - 我应该如何以良好的 Pythonic 方式设置所有路径/设置?我知道每次打开控制台并尝试运行我的服务器时，我都可以在 shell 中运行 export PYTHONPATH=... ，但我想这不是最好的方法.如果我的模块是通过 PyPi(或类似的东西)安装的，我会将它安装在 /usr/lib/python... 路径中，它会自动加载.

I didn't set PythonPath to include my project-root-dir, therefore when server.http tries to include client.PyCachedClient, it tries to load it from relative path and fails. My question is - how should I set all paths/settings in a good, pythonic way? I know I can run export PYTHONPATH=... in shell each time I open a console and try to run my server, but I guess it's not the best way. If my module was installed via PyPi (or something similar), I'd have it installed in /usr/lib/python... path and it'd be loaded automatically.

我希望得到有关 Python 模块开发最佳实践的提示.

I'd appreciate tips on best practices in python module development.

推荐答案

我的 Python 开发工作流程

这是开发 Python 包的基本过程，其中包含了我认为社区中的最佳实践.它是基本的——如果你真的很认真地开发 Python 包，还有更多的东西，每个人都有自己的偏好，但它应该作为一个模板来开始，然后更多地了解所涉及的部分.基本步骤是:

My Python development workflow

This is a basic process to develop Python packages that incorporates what I believe to be the best practices in the community. It's basic - if you're really serious about developing Python packages, there still a bit more to it, and everyone has their own preferences, but it should serve as a template to get started and then learn more about the pieces involved. The basic steps are:

• 使用 virtualenv 进行隔离
• setuptools 用于创建可安装包和管理依赖项
• python setup.py develop 在开发模式下安装该包

• Use virtualenv for isolation
• setuptools for creating a installable package and manage dependencies
• python setup.py develop to install that package in development mode

首先，我建议使用 virtualenv 来获得隔离用于开发包的环境.在开发过程中，您需要安装、升级、降级和卸载包的依赖项，而您不希望

First, I would recommend using virtualenv to get an isolated environment to develop your package(s) in. During development, you will need to install, upgrade, downgrade and uninstall dependencies of your package, and you don't want

• 你的开发依赖会污染你系统范围的site-packages
• 您系统范围的站点包来影响您的开发环境
• 版本冲突

污染你的系统范围的 site-packages 是不好的，因为你在那里安装的任何包都将可用于你安装的所有使用系统 Python 的 Python 应用程序，即使你只需要该依赖项你的小项目.而且它刚刚安装在一个新版本中，该版本覆盖了系统范围的 site-packages 中的版本，并且与依赖它的 ${important_app} 不兼容.你懂的.

Polluting your system-wide site-packages is bad, because any package you install there will be available to all Python applications you installed that use the system Python, even though you just needed that dependency for your small project. And it was just installed in a new version that overrode the one in the system wide site-packages, and is incompatible with ${important_app} that depends on it. You get the idea.

让你的系统范围的 site-packages 影响你的开发环境是不好的，因为也许你的项目依赖于你已经在系统 Python 的 site-packages 中获得的模块.因此，您忘记正确声明您的项目依赖于该模块，但一切正常，因为它始终存在于您的本地开发框中.直到你发布你的包并且人们尝试安装它，或者将它推送到生产环境等等......在干净的环境中开发迫使你正确声明你的依赖项.

Having your system wide site-packages influence your development environment is bad, because maybe your project depends on a module you already got in the system Python's site-packages. So you forget to properly declare that your project depends on that module, but everything works because it's always there on your local development box. Until you release your package and people try to install it, or push it to production, etc... Developing in a clean environment forces you to properly declare your dependencies.

因此，virtualenv 是一个具有自己的 Python 解释器和模块搜索路径的隔离环境.它基于您之前安装的 Python 安装，但与它隔离.

So, a virtualenv is an isolated environment with its own Python interpreter and module search path. It's based on a Python installation you previously installed, but isolated from it.

要创建 virtualenv，请使用 easy_install 或 pip 将 virtualenv 包安装到系统范围的 Python 中:

To create a virtualenv, install the virtualenv package by installing it to your system wide Python using easy_install or pip:

sudo pip install virtualenv

请注意，这将是唯一次您以 root 身份(使用 sudo)安装某些内容到您的全局站点包中.此后的所有内容都将发生在您将要创建的 virtualenv 中.

Notice this will be the only time you install something as root (using sudo), into your global site-packages. Everything after this will happen inside the virtualenv you're about to create.

现在创建一个用于开发包的 virtualenv:

Now create a virtualenv for developing your package:

cd ~/pyprojects
virtualenv --no-site-packages foobar-env

这将创建一个目录树~/pyprojects/foobar-env，这是你的虚拟环境.

This will create a directory tree ~/pyprojects/foobar-env, which is your virtualenv.

要激活virtualenv，将cd放入其中，sourcebin/activate script:

To activate the virtualenv, cd into it and source the bin/activate script:

~/pyprojects $ cd foobar-env/
~/pyprojects/foobar-env $ . bin/activate
(foobar-env) ~/pyprojects/foobar-env $

注意前导点.，它是source shell 命令的简写.还要注意提示是如何变化的: (foobar-env) 表示您在激活的 virtualenv 内部(并且始终需要隔离才能工作).因此，每次打开新的终端选项卡或 SSH 会话等时，请激活您的 env.

Note the leading dot ., that's shorthand for the source shell command. Also note how the prompt changes: (foobar-env) means your inside the activated virtualenv (and always will need to be for the isolation to work). So activate your env every time you open a new terminal tab or SSH session etc..

如果您现在在该激活的环境中运行 python，它实际上将使用 ~/pyprojects/foobar-env/bin/python 作为解释器，并带有自己的 site-packages 和隔离模块搜索路径.

If you now run python in that activated env, it will actually use ~/pyprojects/foobar-env/bin/python as the interpreter, with its own site-packages and isolated module search path.

现在创建您的包.基本上，您需要一个带有 setup.py 的 setuptools 包来正确声明包的元数据和依赖项.您可以按照 setuptools 文档 自行完成此操作，或使用 粘贴模板.要使用 Paster 模板，请将 PasteScript 安装到您的 virtualenv 中:

Now for creating your package. Basically you'll want a setuptools package with a setup.py to properly declare your package's metadata and dependencies. You can do this on your own by by following the setuptools documentation, or create a package skeletion using Paster templates. To use Paster templates, install PasteScript into your virtualenv:

pip install PasteScript

让我们为我们的新包创建一个源目录以保持井井有条(也许您想将项目拆分为多个包，或者稍后使用源中的依赖项):

Let's create a source directory for our new package to keep things organized (maybe you'll want to split up your project into several packages, or later use dependencies from source):

mkdir src
cd src/

现在要创建您的包，请执行

Now for creating your package, do

paster create -t basic_package foobar

并回答交互界面中的所有问题.大多数都是可选的，只需按 ENTER 即可保留默认值.

and answer all the questions in the interactive interface. Most are optional and can simply be left at the default by pressing ENTER.

这将创建一个名为 foobar 的包(或更准确地说，是一个 setuptools 发行版).这是

This will create a package (or more precisely, a setuptools distribution) called foobar. This is the name that

• 人们会使用 easy_install 或 pip install foobar
来安装你的包
• 在 setup.py
中其他包将使用的名称依赖于您的包
• 它将在 PyPi 上调用什么

在内部，您几乎总是创建一个名为相同的 Python 包(如具有 __init__.py 的目录).这不是必需的，顶级 Python 包的名称可以是任何有效的包名称，但将其命名为与发行版相同的常见约定.这就是为什么将两者分开很重要但并不总是容易的原因.因为顶级 python 包名称是什么

Inside, you almost always create a Python package (as in "a directory with an __init__.py) that's called the same. That's not required, the name of the top level Python package can be any valid package name, but it's a common convention to name it the same as the distribution. And that's why it's important, but not always easy, to keep the two apart. Because the top level python package name is what

• 人们(或您)将使用 import foobar 或 from foobar import baz
导入您的包

• people (or you) will use to import your package using import foobar or from foobar import baz

因此，如果您使用了粘贴模板，它已经为您创建了该目录:

So if you used the paster template, it will already have created that directory for you:

cd foobar/foobar/

现在创建您的代码:

vim models.py

models.py

class Page(object):
    """A dumb object wrapping a webpage.
    """

    def __init__(self, content, url):
        self.content = content
        self.original_url = url

    def __repr__(self):
        return "<Page retrieved from '%s' (%s bytes)>" % (self.original_url, len(self.content))

和使用 models.py 的同一目录中的 client.py:

And a client.py in the same directory that uses models.py:

client.py

import requests
from foobar.models import Page

url = 'http://www.stackoverflow.com'

response = requests.get(url)
page = Page(response.content, url)

print page

在setup.py中声明对requests模块的依赖:

Declare the dependency on the requests module in setup.py:

  install_requires=[
      # -*- Extra requirements: -*-
      'setuptools',
      'requests',
  ],

版本控制

src/foobar/ 是您现在要置于版本控制之下的目录:

Version control

src/foobar/ is the directory you'll now want to put under version control:

cd src/foobar/
git init
vim .gitignore

.gitignore

*.egg-info
*.py[co]

git add .
git commit -m 'Create initial package structure.

将您的包安装为开发蛋

现在是在开发模式下安装包的时候了:

Installing your package as a development egg

Now it's time to install your package in development mode:

python setup.py develop

这将安装requests 依赖项和您的 包作为开发蛋.因此，它已链接到您的 virtualenv 的站点包中，但仍位于 src/foobar 中，您可以在其中进行更改并使它们立即在 virtualenv 中处于活动状态，而无需重新安装您的包.

This will install the requests dependency and your package as a development egg. So it's linked into your virtualenv's site-packages, but still lives at src/foobar where you can make changes and have them be immediately active in the virtualenv without re-installing your package.

现在对于您的原始问题，使用相对路径导入:我的建议是，不要这样做.既然您已经有了合适的 setuptools 包，它已安装并可导入，那么您当前的工作目录就不再重要了.只需执行 from foobar.models import Page 或类似操作，声明该对象所在的完全限定名称.这使您的源代码更具可读性和可发现性，对您自己和其他阅读您代码的人来说都是如此.

Now for your original question, importing using relative paths: My advice is, don't do it. Now that you've got a proper setuptools package, that's installed and importable, your current working directory shouldn't matter any more. Just do from foobar.models import Page or similar, declaring the fully qualified name where that object lives. That makes your source code much more readable and discoverable, for yourself and other people that read your code.

您现在可以通过在激活的 virtualenv 中的任何位置执行 python client.py 来运行您的代码.python src/foobar/foobar/client.py 工作正常，您的包已正确安装，您的工作目录不再重要.

You can now run your code by doing python client.py from anywhere inside your activated virtualenv. python src/foobar/foobar/client.py works just as fine, your package is properly installed and your working directory doesn't matter any more.

如果您想更进一步，您甚至可以为您的 CLI 脚本创建一个 setuptools 入口点.这将在您的 virtualenv 中创建一个 bin/something 脚本，您可以从 shell 运行该脚本.

If you want to go one step further, you can even create a setuptools entry point for your CLI scripts. This will create a bin/something script in your virtualenv that you can run from the shell.

setup.py

  entry_points='''
  # -*- Entry points: -*-
  [console_scripts]
  run-fooobar = foobar.main:run_foobar
  ''',

client.py

def run_client():
    # ...

main.py

from foobar.client import run_client

def run_foobar():
    run_client()

重新安装您的软件包以激活入口点:

Re-install your package to activate the entry point:

python setup.py develop

就这样，bin/run-foo.

一旦你(或其他人)在 virtualenv 之外真实地安装了你的包，入口点将在 /usr/local/bin/run-foo 或类似的地方，它会自动在 $PATH 中.

Once you (or someone else) installs your package for real, outside the virtualenv, the entry point will be in /usr/local/bin/run-foo or somewhere simiar, where it will automatically be in $PATH.

• 创建包的版本并上传 PyPi，例如使用 zest.releaser
• 保留变更日志并对您的包进行版本控制
• 了解声明依赖项
• 了解 distribute、distutils、setuptools 和 distutils2 之间的区别

推荐阅读:

• Hitchhiker 包装指南
• pip 食谱

这篇关于python模块导入 - 相对路径问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！