背景
- 针对离线环境存在python版本不匹配，安装三方库缓慢或无外网的情况下，需要制作可移植的解释器及其附带的库文件。
- 由于sys.path是在安装时设置（见下图），直接压缩安装的python解释器及其lib,include文件必然导致不兼容。
- conda虽然内置python解释器但同样需要安装，不够轻便
#sys.py
base_exec_prefix = 'C:\\ProgramData\\Anaconda3'
base_prefix = 'C:\\ProgramData\\Anaconda3'
byteorder = 'little'
copyright = 'Copyright (c) 2001-2019 Python Software Foundation.\nAll Rights Reserved.\n\nCopyright (c) 2000 BeOpen.com.\nAll Rights Reserved.\n\nCopyright (c) 1995-2001 Corporation for National Research Initiatives.\nAll Rights Reserved.\n\nCopyright (c) 1991-1995 Stichting Mathematisch Centrum, Amsterdam.\nAll Rights Reserved.'
dllhandle = 140724640546816
dont_write_bytecode = True
executable = 'C:\\ProgramData\\Anaconda3\\python.exe'
exec_prefix = 'C:\\ProgramData\\Anaconda3'
解决方案
使用virtualenv制作虚拟环境，将原解释器中lib、include与site-packages等不依赖路径文件替换，并修改若干启动项。
virtualenv用于制作虚拟环境，python3.3后的标准库中venv集成了virtualenv一些关键功能，具体区别：
virtualenvis a tool to create isolated Python environments.Since Python3.3, a subset of it has been integrated into the standard library under the venv module. Thevenvmodule does not offer all features of this library, to name just a few more prominent:
- is slower (by not having the
app-dataseed method), - is not as extendable,
- cannot create virtual environments for arbitrarily installed python versions (and automatically discover these),
- is not upgrade-able via pip,
- does not have as rich programmatic API (describe virtual environments without creating them).
virtualenv命令选项
usage: virtualenv [--version] [--with-traceback] [-v | -q] [--discovery {builtin}] [-p py] [--creator {builtin,cpython2-posix}] [--seeder {app-data,pip}] [--no-seed] [--activators comma_separated_list] [--clear] [--system-site-packages]
[--symlinks | --copies] [--download | --no-download] [--extra-search-dir d [d ...]] [--pip version] [--setuptools version] [--wheel version] [--no-pip] [--no-setuptools] [--no-wheel] [--clear-app-data] [--symlink-app-data]
[--prompt prompt] [-h]
dest
optional arguments:
--version display the version of the virtualenv package and it's location, then exit
--with-traceback on failure also display the stacktrace internals of virtualenv (default: False)
-h, --help show this help message and exit
verbosity:
verbosity = verbose - quiet, default INFO, mapping => CRITICAL=0, ERROR=1, WARNING=2, INFO=3, DEBUG=4, NOTSET=5
-v, --verbose increase verbosity (default: 2)
-q, --quiet decrease verbosity (default: 0)
discovery:
discover and provide a target interpreter
--discovery {builtin} interpreter discovery method (default: builtin)
-p py, --python py target interpreter for which to create a virtual (either absolute path or identifier string) (default: /usr/bin/python)
creator:
options for creator builtin
--creator {builtin,cpython2-posix}
create environment via (builtin = cpython2-posix) (default: builtin)
dest directory to create virtualenv at
--clear remove the destination directory if exist before starting (will overwrite files otherwise) (default: False)
--system-site-packages give the virtual environment access to the system site-packages dir (default: False)
--symlinks try to use symlinks rather than copies, when symlinks are not the default for the platform (default: True)
--copies, --always-copy try to use copies rather than symlinks, even when symlinks are the default for the platform (default: False)
seeder:
options for seeder app-data
--seeder {app-data,pip} seed packages install method (default: app-data)
--no-seed, --without-pip do not install seed packages (default: False)
--download pass to enable download of the latest pip/setuptools/wheel from PyPI (default: False)
--no-download, --never-download pass to disable download of the latest pip/setuptools/wheel from PyPI (default: True)
--extra-search-dir d [d ...] a path containing wheels the seeder may also use beside bundled (can be set 1+ times) (default: [])
--pip version pip version to install, bundle for bundled (default: latest)
--setuptools version setuptools version to install, bundle for bundled (default: latest)
--wheel version wheel version to install, bundle for bundled (default: latest)
--no-pip do not install pip (default: False)
--no-setuptools do not install setuptools (default: False)
--no-wheel do not install wheel (default: False)
--clear-app-data clear the app data folder of seed images (/root/.local/share/virtualenv/seed-v1) (default: False)
--symlink-app-data symlink the python packages from the app-data folder (requires seed pip>=19.3) (default: False)
activators:
options for activation scripts
--activators comma_separated_list
activators to generate - default is all supported (default: bash,cshell,fish,powershell,python)
--prompt prompt provides an alternative prompt prefix for this environment (default: None)
操作步骤
1. pip install virtualenv
2. virtualenv venv -p 解释器路径 --system-site-packages --copies
执行命令生成venv文件夹，集成了python的虚拟环境。
具体参数说明见virtualenv命令选项，特别说明下要选择--copies，默认的--symlinks会创建引用链接，移植会导致链接失效不可用
选择--symlinks后的venv/lib/python3.6
(root) [root@host-192-169-100-115 python3.6]# pwd
/home/venv/lib/python3.6
(root) [root@host-192-169-100-115 python3.6]# ls -al
lrwxrwxrwx. 1 root root 58 Mar 5 00:57 abc.py -> /home/runtime/lib/python3.6/abc.py
lrwxrwxrwx. 1 root root 61 Mar 5 00:57 base64.py -> /home/runtime/lib/python3.6/base64.py
lrwxrwxrwx. 1 root root 61 Mar 5 00:57 bisect.py -> /home/runtime/lib/python3.6/bisect.py
3.拷贝原有解释器的lib（lib64）、include等文件（site-packages含在lib内）复制到venv一级目录下
virtualenv的说明中有一段：
Created python virtual environments are usually not self-contained. A complete python packaging is usually made up of thousand of files, so it’s not efficient to install the entire python again into a new folder. Instead virtual environments are mere shells, that contain little within itself, and borrow most from the system python (this is what you installed, when you installed python itself). This does mean that if you upgrade your system python your virtual environments might break, so watch out. The upside of this referring to the system python is that creating virtual environments can be fast.
提到拷贝完整文件对于虚拟环境并没有意义，会借用系统解释器的部分组件，故新版本的virtualenv已经不含lib库，并且脱离lib库的python无法运行。
本需求需要一个完整的python，故手动拷贝。（如果virtualenv有选项更好，可惜并没有找到）
venv目录：
[root@host-192-169-100-115 venv]# ll
total 12
drwxr-xr-x. 2 root root 4096 Mar 5 01:02 bin
drwxr-xr-x. 2 root root 4096 Mar 5 00:57 include
drwxr-xr-x. 3 root root 4096 Mar 5 00:57 lib
drwxr-xr-x. 3 root root 4096 Mar 5 00:57 pyenv.cfg
4.删除venv中目录下的pyenv.cfg文件1
若无该文件可以不关注。
5.修改bin目录下执行脚本的绝对路径
当前有两类文件存在绝对路径：
- bin目录下其他可执行python工具，开头解释器为固定解释器
以pip举例：
#!/usr/bin/python3.6
# -*- coding: utf-8 -*-
import re
import sys
from pip import main
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
sys.exit(main())
修改前：
#!/usr/bin/python3.6
修改后：
#!/usr/bin/env python3.6
- 用于启动虚拟环境的bin/activate文件，其中venv的目录为绝对路径
修改前：
VIRTUAL_ENV="/home/venv"
修改后：
CUR_BIN_DIR=$(cd $(dirname ${BASH_SOURCE:-$0});pwd)
VIRTUAL_ENV="$(dirname $CUR_BIN_DIR)"
验证步骤
- 切换任意目录读取activate中命令（source venv所在文件夹/venv/bin/activate）
[root@SDN-4416-1 venv]# source /home/runtime/venv/bin/activate
- 检查python命令索引
(venv) [root@SDN-4416-1 venv]# which python
/home/runtime/venv/bin/python
- 进入python控制器台检查标准库和三方库路径非借用
(venv) [root@SDN-4416-1 venv]# python
Python 3.7.4 (default, Aug 13 2019, 20:35:49)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.__file__
'/home/runtime/venv/lib/python3.7/os.py'
>>> import requests
>>> requests.__file__
'/home/runtime/venv/lib/python3.7/site-packages/requests/__init__.py'
>>>
延伸阅读
1.删除venv中目录下的pyenv.cfg文件？
文件内容：
[root@SDN-4416-1 venv]# cat pyvenv.cfg
home = /usr
implementation = CPython
version_info = 2.7.5.final.0
virtualenv = 20.0.4
include-system-site-packages = true
base-prefix = /usr
base-exec-prefix = /usr
base-executable = /usr/bin/python
为了解释这个操作的原因，先引出两个问题：
- 虚拟环境下的python可执行文件与系统默认的有什么不同呢？
- Python又是如何找到虚拟环境下的第三方库的呢？
事实上，上面提到的python可执行文件之间并没有什么不同，但是它们所在的位置至关重要。在Python启动的时候，它会获取自身所在的路径。然后这一路径（bin的上一级）被设置到sys.prefix和sys.exec_prefix之中。
sys.prefix
A string giving the site-specific directory prefix where the platform independent Python files are installed; by default, this is the string '/usr/local'. This can be set at build time with the --prefix argument to the configure script. The main collection of Python library modules is installed in the directory prefix/lib/pythonX.Y while the platform independent header files (all except pyconfig.h) are stored in prefix/include/pythonX.Y, where X.Y is the version number of Python, for example 3.2.
Note If a virtual environment is in effect, this value will be changed in site.py to point to the virtual environment. The value for the Python installation will still be available, via base_prefix.
在搜索第三方的site-packages时，搜索目录是sys.prefix所指向的路径下的lib/pythonX.X/site-packages/，其中X.X是Python的版本。
假设python文件所在路径为/Users/michaelherman/python-virtual-environments/env/bin，因此sys.prefix会被设为/Users/michaelherman/python-virtual-environments/env，从而site-packages的路径就变成了/Users/michaelherman/python-virtual-environments/env/lib/pythonX.X/site-packages。最后，这一路径被存储在sys.path数组中，其中包含着所有包的引用来源。
详见：Python虚拟环境详解
再来，关于pyenv.cfg有这样的解释：
PEP-405 Python Virtual Environments
When the Python binary is executed, it attempts to determine its prefix (which it stores in sys.prefix), which is then used to find the standard library and other key files, and by the site module to determine the location of the site-package directories. Currently the prefix is found (assuming PYTHONHOME is not set) by first walking up the filesystem tree looking for a marker file (os.py) that signifies the presence of the standard library, and if none is found, falling back to the build-time prefix hardcoded in the binary.
This PEP proposes to add a new first step to this search. If a pyvenv.cfg file is found either adjacent to the Python executable or one directory above it (if the executable is a symlink, it is not dereferenced), this file is scanned for lines of the form key = value. If a home key is found, this signifies that the Python binary belongs to a virtual environment, and the value of the home key is the directory containing the Python executable used to create this virtual environment.
In this case, prefix-finding continues as normal using the value of the home key as the effective Python binary location, which finds the prefix of the base installation. sys.base_prefix is set to this value, while sys.prefix is set to the directory containing pyvenv.cfg.
(If pyvenv.cfg is not found or does not contain the home key, prefix-finding continues normally, and sys.prefix will be equal to sys.base_prefix.)
Also, sys.base_exec_prefix is added, and handled similarly with regard to sys.exec_prefix. (sys.exec_prefix is the equivalent of sys.prefix, but for platform-specific files; by default it has the same value as sys.prefix.)
The site and sysconfig standard-library modules are modified such that the standard library and header files are found relative to sys.base_prefix / sys.base_exec_prefix, while site-package directories ("purelib" and "platlib", in sysconfig terms) are still found relative to sys.prefix / sys.exec_prefix.
Thus, a Python virtual environment in its simplest form would consist of nothing more than a copy or symlink of the Python binary accompanied by a pyvenv.cfg file and a site-packages directory.
同样可以通过标准site.py找到对pyenv.cfg文件的引用：
#site.py
If a file named "pyvenv.cfg" exists one directory above sys.executable,
sys.prefix and sys.exec_prefix are set to that directory and
it is also checked for site-packages (sys.base_prefix and
sys.base_exec_prefix will always be the "real" prefixes of the Python
installation). If "pyvenv.cfg" (a bootstrap configuration file) contains
the key "include-system-site-packages" set to anything other than "false"
(case-insensitive), the system-level prefixes will still also be
searched for site-packages; otherwise they won't.
也就是说sys.prefix和sys.exec_prefix被设置好后，pyenv.cfg存在时虚拟环境时会继续替换为文件中的前缀,在本需求中没有必要，故删去。
