Conda build framework¶
Building a package requires a recipe. A recipe is flat directory which contains the following files:
- meta.yaml (metadata file)
- build.sh (Unix build script which is executed using bash)
- bld.bat (Windows build script which is executed using cmd)
- run_test.py (optional Python test file)
- patches to the source (optional, see below)
- other resources, which are not included in the source and cannot be generated by the build scripts.
The same recipe should be used to build a package on all platforms.
When building a package, the following steps are invoked:
- read the metadata
- download the source (into a cache)
- extract the source in a source directory
- apply the patches
- create a build environment (build dependencies are installed here)
- run the actual build script. The current working directory is the source directory with environment variables set. The build script installs into the build environment
- do some necessary post processing steps: shebang, rpath, etc.
- add conda metadata to the build environment
- package up the new files in the build environment into a conda package
- test the new conda package: - create a test environment with the package (and its dependencies) - run the test scripts
There are example recipes for many conda packages in the conda-recipes repo.
The meta.yaml file¶
package: name: bsdiff4 # lower case name of package, may contain '-' but no spaces version: 1.1.4 # version of package. Should use the PEP-386 verlib conventions. source: # The source section specifies where the source code of the package is coming # from, it may be coming from a source tarball like: fn: qt-everywhere-opensource-src-4.8.4.tar.gz url: http://filer/src/qt-everywhere-opensource-src-4.8.4.tar.gz md5: 89c5ecba180cae74c66260ac732dc5cb # (optional) # or from git: git_url: email@example.com:ilanschnell/bsdiff4.git git_tag: 1.1.4 # (optional) # also optionally patches may be applied to the source patches: - my.patch # the patch file is expected to be found in the recipe # Note, the source section is optional. If you want to specify a source # location locally, the easiest way is to not specify the source here, but # to just add something like # # cp -r $RECIPE_DIR/../src . # cd src # ... # # in build.sh (and similarly in bld.bat). This assumes the source is # shipped alongside the recipe in src. # The build number should be incremented for new builds of the same version build: # (optional) number: 1 # (optional, defaults to 0) # optional Python entry points entry_points: - bsdiff4 = bsdiff4.cli:main_bsdiff4 - bspatch4 = bsdiff4.cli:main_bspatch4 # the build and runtime requirements: requirements: # (optional) build: - python run: - python test: # (optional) # files which are copied from the recipe into the (temporary) test # directory which are needed during testing files: - test-data.txt # in addition to the run-time requirements, we can specify requirements # needed during testing requires: - nose # commands we want to make sure they work, which are expected to get # installed by the package commands: - bsdiff4 -h - bspatch4 -h # Python imports imports: - bsdiff4 # The script run_test.py will be run automatically if it is part of the # recipe about: # (optional) home: https://github.com/ilanschnell/bsdiff4 license: BSD
Specifying versions in requirements¶
Each element in the list of build and run-time requirements is a match specification, i.e. a string, which (when split by spaces) has 1, 2 or 3 parts:
the first part is always the (exact) name
- the second part refers to the version, and may contain special characters
| means “or”, e.g. 1.0|1. matches either version 1.0 or 1.2
* means (in terms of regex) r'.*'
1.0|1.4* matches 1.0, 1.4, 1.4.1b2, but not 1.2
(when there are 3 parts, the second part has to be the exact version)
the third part is always the (exact) build string
In addition, you can add selectors to each line, which are used as part of a preprocessing stage. Before the yaml file is read, each selector is evaluated, and if it is False, the line that it is on is removed. A selector is of the form [<selector>] at the end of a line.
source: url: http://path/to/unix/source [not win] url: http://path/to/windows/source [win]
A selector is just a valid Python statement, that is executed. The following variables are defined. Unless otherwise stated, the variables are booleans.
|linux||True if the platform is Linux|
|linux32||True if the platform is Linux and the Python architecture is 32-bit|
|linux64||True if the platform is Linux and the Python architecture is 64-bit|
|armv6||True if the platform is Linux and the Python architecture is armv6l|
|osx||True if the platform is OS X|
|unix||True if the platform is Unix (OS X or Linux)|
|win||True if the platform is Windows|
|win32||True if the platform is Windows and the Python architecture is 32-bit|
|win64||True if the platform is Windows and the Python architecture is 64-bit|
|py||The Python version as a two digit string (like '27'). See also the CONDA_PY environment variable below.|
|py3k||True if the Python major version is 3|
|py2k||True if the Python major version is 2|
|py26||True if the Python version is 2.6|
|py27||True if the Python version is 2.7|
|py33||True if the Python version is 3.3|
|np||The NumPy version as a two digit string (like '17'). See also the CONDA_NPY environment variable below.|
Because the selector is any valid Python expression, complicated logic is possible.
source: url: http://path/to/windows/source [win] url: http://path/to/python2/unix/source [unix and py2k] url: http://path/to/python3/unix/source [unix and py3k]
Note that the selectors delete only they line that they are on, so you may need to put the same selector on multiple lines.
source: url: http://path/to/windows/source [win] md5: 30fbf531409a18a48b1be249052e242a [win] url: http://path/to/unix/source [unix] md5: 88510902197cba0d1ab4791e0f41a66e [unix]
Environment variables set during the build process¶
The following environment variables are set, both on Unix (build.sh) and on Windows (bld.bat) during the build process:
|ARCH||either 32 or 64, to specify whether the build is 32-bit or 64-bit. The value depends on the ARCH environment variable, and defaults to the architecture the interpreter running conda was compiled with|
|SRC_DIR||path to where source is unpacked (or cloned)|
|PREFIX||build prefix where build script should install to|
|RECIPE_DIR||directory of recipe|
|PATH||prepended by the build prefix bin directory|
|PYTHON||path to python executable in build prefix (note that python is only installed in the build prefix when it is listed as a build requirement)|
|PY3K||1 when Python 3 is installed in build prefix, else 0|
|STDLIB_DIR||Python standard library location|
|SP_DIR||Python’s site-packages location|
|PY_VER||Python version building against|
|SYS_PYTHON||Python of process which is building|
|SYS_PREFIX||prefix of process which is building, usually root env|
When building “unix-style” packages on Windows, which are then usually statically linked to executables, we do this in a special Library directory under the build prefix. The following environment variables are only defined in Windows:
On Mac OS X, we have:
|OSX_ARCH||i386 or x86_64, depending on Python build||PKG_CONFIG_PATH|
On Linux, we have:
Note that build.sh is run with bash -x -e (the -x makes it echos each command that is run, and the -e makes it exit whenever a command in the script returns nonzero exit status). You can revert this in the script if you need to by using the set command.
Environment variables that affect the build process¶
|CONDA_PY||Should be 26, 27, or 33. This is the Python version used to build the package.|
|CONDA_NPY||Should be either 16 or 17. This is the NumPy version used to build the package.|