"pip -t": A simple and transparent alternative to virtualenv
Often, virtualenv is overkill for the basic task of installing project dependencies and
keeping them isolated. We present a simple alternative consisting of:
./.pip to your
pip install -t .pip to install modules locally
python from your project's root directory
Which version would you like today?
Installing dependencies is a required step for almost any Python application.
Each Python app depends on a different set of libraries,
and to be sure that it behaves as expected, the best thing is to install exactly the right version of each library.
The standard practice is to ship Python projects with a
requirements.txt file. This file lists the libraries
that the project depends on, and a version number for each one. If present, installing the dependencies
is as easy as:
$ pip install -r requirements.txt
So far so good! The problems start when using two or more projects with conflicting dependencies. Let's suppose
project A only works with library X version 0.1, and project B uses the same library X, but only works with version 0.2.
By default, pip installs libraries globally into the Python interpreter's library path. This means that issuing the
$ pip install X==0.2
command will make X version 0.2 available in every
Python instance, overwriting version 0.1 if it was
previously installed. Switching between project A and project B would require reinstalling the right version of X each time,
which is time-consuming and inconvenient.
An island in the sun
One popular solution to this commonly encountered problem is virtual environments. The virtualenv
framework allows you to create isolated Python environments. The dependencies for each project are kept
separate from each other. However, some users find virtualenv complicated to use, so packages like virtualenvwrapper
and autoenv extend its functionality in an attempt to make things easier. Other solutions include
Anaconda environments in the Anaconda Python distribution, and
pyvenv which is baked into the Python standard
library starting from Python 3.3.
Though these are great tools, we have always felt that they represent a rather heavy and complicated toolset for what
should essentially be a very simple task.
provide the easy, reliable and powerful
package management capabilities which it feels like Python is missing. The key to their success? Both tools
download a copy of the right versions of the right libraries, by default placing them in a special folder directly within
your project's directory. The downloaded libraries remain local to only that project, meaning that you automatically avoid
the issues described above.
As it turns out, there's a simple way of replicating the npm/Bower approach for Python packages, involving
these easy steps:
./.pip to your
-t .pip to install your libraries locally.
Then, simply execute your code from within your project directory and forget about
The trick works because
./.pip is a relative path. As a result, if you run
~/dev/project_a/.pip gets included in that Python instance's
library path. If you run
gets included instead. This works on all major platforms: Linux, Mac and Windows.
The folder name
.pip is arbitrary of course — for example, one could choose to name the folder
libs instead. However,
.pip is quick to type, and the initial
. hides the folder by default on Linux/Mac.
Step 1: Set the PYTHONPATH
The following command will permanently set the PYTHONPATH for standard terminal sessions:
After that, either restart your terminal or do
$ echo 'export PYTHONPATH="./.pip:$PYTHONPATH"' >> ~/.bash_profile
$ source .bash_profile to make sure the PYTHONPATH
is loaded into the current session. Depending on your platform, you might want to use
Go to Control Panel > System and Security > System > Change Settings > Advanced > Environment Variables,
and add/edit the PYTHONPATH variable either to your user variables or system variables, setting it to
.\.pip;(...other paths...). Then restart your command prompt.
If you prefer to change the PYTHONPATH only temporarily for the duration of your terminal session, you can also do
$ export PYTHONPATH=./.pip on Mac/Linux or
> set PYTHONPATH=.\.pip on Windows.
On Mac/Linux, you can even set the PYTHONPATH just for the duration of a Python session:
$ PYTHONPATH=./.pip python main.py.
Step 2: Install packages with "pip -t"
Now that we have set the PYTHONPATH, the only thing left to do is to install our packages into the right
location using pip. For this, we use the
--target switch to indicate the directory
into which pip should install the packages:
$ cd project_a
project_a$ pip install requests==2.7.0 -t .pip
>>> import requests
Now let's do the same for another project with another version:
$ cd project_b
project_b$ pip install requests==2.6.0 -t .pip
>>> import requests
This works equally well when using a
$ pip install -r requirements.txt -t .pip
Different Python interpreters
You can easily run your program with different Python interpreters as follows:
$ /path/to/python main.py
However, there's an issue if you are switching between Python 2 and 3 and you are using packages that don't have
a single code base, i.e. they compile their source code during installation using
2to3. In that case, you
would have to introduce something like
.pip3 and add this path in front of your PYTHONPATH when running
things with Python 3.
If you happen to have packages that have been installed globally using
easy_install, you've got the
issue that easy_install prepends the path to those libraries to your
gives them priority over whatever you have in
.pip. The solution here is to get rid of global
easy_install installations. You can easily check if something hijacks your .pip setup by running
import sys;sys.path within a Python session. If there are paths in front of
./.pip, then you might need to clean up things first.