Mirroring a PyPI repository (AER 2.32)

Before you begin

You need to have already completed installing and configuring your Anaconda Repository instance. Due to the size of Anaconda Repository, it is important that you have configured a file storage location with sufficient disk space. If necessary please see the instructions for setting the file storage location.

The full PyPI mirror requires approximately 120GB.

The mirror command

To create a PyPI mirror run:

anaconda-server-sync-pypi

This will load all of the packages on pypi.python.org into the ~pypi binstar user account.

Verify that this is working by opening your browser to http://your.anaconda.repository/pypi/~pypi

NOTE: Replace your.anaconda.repository with the actual URL to your Anaconda Repository.

Customizing the mirror

It is possible to customize the mirror behavior by creating a configuration file such as $PREFIX/etc/anaconda-server/mirror/pypi.yaml and using the --mirror-config option:

anaconda-server-sync-pypi --mirror-config /etc/binstar/mirrors/pypi.yaml

The following configuration options are available:

Name Description
user The local user under which the PyPI packages are imported. Default: pypi
whitelist A list of packages to mirror. Only packages listed are mirrored. If the list is empty, all packages are checked. Default: []
blacklist A list of packages to skip. The packages listed are ignored. Default: []
latest_only Only download the latest versions of the packages. Default: false.
remote_url The URL of the PyPI mirror. /pypi is appended to build the XML RPC API URL, /simple for the simple index, and /pypi/{package}/{version}/json for the JSON API. Default: https://pypi.python.org/
xml_rpc_api_url A custom value for XML RPC URL. If this value is present, it takes precedence over the URL built using remote_url. Default: null
simple_index_url A custom value for the simple index URL. If this value is present, it takes precedence over the URL built using remote_url. Default: null
use_xml_rpc Whether to use the XML RPC API as specified by PEP381. If this is set to true, the simple index is used to determine which packages to check. The simple index is also used if the XML RPC API fails for any reason. Default: true
use_serial Whether to use the serial number provided by the XML RPC API. Only packages updated since the last serial saved will be checked. If this is set to false all PyPI packages are going to be checked for updates. Default: true
create_org Create the mirror user as an Organization instead of a regular user account. All superusers are added to the “Owners” group of the organization. Default: false

Example:

whitelist:
  - requests
  - six
  - numpy
  - simplejson
latest_only: true
remote_url: http://pypimirror.local/
use_xml_rpc: false

Configure pip

To configure pip to use this new mirror you must edit your ~/.pip/pip.conf file:

[global]
index-url = http://your.anaconda.repo:<port>/pypi/~pypi/simple

NOTE: Replace your.anaconda.repo with the actual URL to your Anaconda Repository.