安装gymnasium，利用经典的环境进行RL

gym是强化学习的一个经典环境，已经更新成了gymnasium
基本的安装按照https://gymnasium.farama.org/environments/atari/adventure/

pip install gymnasium
pip install gymnasium[atari]
pip install gymnasium[all]

不同的环境需要如下配置：

• box-2d

pip install gymnasium[box2d]

• mujoco

pip install gymnasium[mujoco]

• atari

pip install gymnasium[accept-rom-license]
pip install ale-py

import gymnasium as gym
import ale_py

gym.register_envs(ale_py)  # unnecessary but helpful for IDEs

env = gym.make('ALE/Breakout-v5', render_mode="human")  # remove render_mode in training
obs, info = env.reset()
episode_over = False
while not episode_over:
    action = policy(obs)  # to implement - use `env.action_space.sample()` for a random policy
    obs, reward, terminated, truncated, info = env.step(action)

    episode_over = terminated or truncated
env.close()

问题1[Failed building wheel for box2d-py]

  Building wheel for box2d-py (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [16 lines of output]
      Using setuptools (version 69.5.1).
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-38
      creating build/lib.linux-x86_64-cpython-38/Box2D
      copying library/Box2D/Box2D.py -> build/lib.linux-x86_64-cpython-38/Box2D
      copying library/Box2D/__init__.py -> build/lib.linux-x86_64-cpython-38/Box2D
      creating build/lib.linux-x86_64-cpython-38/Box2D/b2
      copying library/Box2D/b2/__init__.py -> build/lib.linux-x86_64-cpython-38/Box2D/b2
      running build_ext
      building 'Box2D._Box2D' extension
      swigging Box2D/Box2D.i to Box2D/Box2D_wrap.cpp
      swig -python -c++ -IBox2D -small -O -includeall -ignoremissing -w201 -globals b2Globals -outdir library/Box2D -keyword -w511 -D_SWIG_KWARGS -o Box2D/Box2D_wrap.cpp Box2D/Box2D.i
      error: command 'swig' failed: No such file or directory
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for box2d-py
  Running setup.py clean for box2d-py
Failed to build box2d-py
ERROR: Could not build wheels for box2d-py, which is required to install pyproject.toml-based projects

https://github.com/openai/gym/issues/3143#issuecomment-1560325924

Box2D is written in C++ and requires swig to bind C++ to Python.
For Windows users:

You can download swig here: https://www.swig.org/download.html
Mac users:

brew install swig
For Linux users:

apt-get install swig3.0
ln -s /usr/bin/swig3.0 /usr/bin/swig

问题2[ERROR: Failed building wheel for mujoco-py]

ModuleNotFoundError: No module named 'lockfile'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for mujoco-py
  Running setup.py clean for mujoco-py
Failed to build mujoco-py
ERROR: Could not build wheels for mujoco-py, which is required to install pyproject.toml-based projects

 sudo apt install libosmesa6-dev libgl1-mesa-glx libglfw3

sudo apt-get install libgl1-mesa-dev libgl1-mesa-glx libosmesa6-dev python3-pip python3-numpy python3-scipy

pyproject.toml

什么是pyproject.toml

pyproject.toml是Python生态系统中一个新兴的标准配置文件,它主要用于以下几个方面:

构建和打包工具配置:
    定义项目的构建系统和依赖关系,如使用 poetry、setuptools 还是 flit 等。
    指定项目的元数据信息,如名称、版本、作者等。

依赖管理:
    声明项目所需的依赖包及其版本要求。
    支持指定依赖包的来源,如 PyPI、Git 仓库等。

构建和发布配置:
    定义项目的构建和发布流程,如打包格式、发布渠道等。
    支持自定义构建和发布脚本。

静态类型检查和代码格式化:
    配置静态类型检查工具,如 mypy。
    设置代码格式化工具,如 black、isort 等。

相比传统的 setup.py 文件,pyproject.toml 提供了更加标准化和灵活的项目配置方式。它逐渐成为 Python 社区的新标准,被越来越多的工具和框架所支持。使用 pyproject.toml 可以帮助开发者更好地管理和分享 Python 项目。