So you want a specific Python version on a server where you don't have root

Notes from 3/11/15

Updated 3/30/2016 (for Python 3!)

This is a simple HOWTO for getting a Python development environment up and running with virtualenv and virtualenvwrapper.

The HOWTO is written for social scientists who need to do their work from the command line, for example on a server provided by their parent institution.

Before moving on to more complicated things, like installing Anaconda or getting yourself set up to manage different virtual environments with a tool like Docker, try this. And if this doesn't get it done for you, then the exercise will make it apparent precisly why you need Anaconda or Docker.

The following instructions were written for a machine running SElinux, but they ought to work on any *nix machine. They will also work fine from the OS X Terminal, so you could (and should!) follow this procedure to get multiple versions of Python running on your Mac.

1. Project setup

a. Development environment

11/3/14

Install Python locally (for instance, following these instructions): http://stackoverflow.com/questions/1534210/use-different-python-version-with-virtualenv

For Python 3.5.1, this involves:

mkdir ~/src
wget https://www.python.org/ftp/python/3.5.1/Python-3.5.1.tar.xz
tar -xf Python-3.5.1.tar.xz
cd Python-3.5.1
mkdir ~/.localpython
./configure --prefix=/home/username/.localpython
make
make install

Since we're logging in remotely every time, set environment variables in ~/.bash_profile. This is what I add to my ~/.bash_profile file:

PATH="/home/username/.local/bin:/home/username/.localpython/bin:$PATH"
PYTHONHOME="/home/username/.localpython"
export PYTHONHOME
export WORKON_HOME=$HOME/.virtualenvs
export PROJECT_HOME=$HOME/projects

After you have made these adjustments to your .bash_profile, you can activate them by typing:

source ~/.bash_profile

Now it's time to install virtualenv and virtualenvwrapper. We'll do that using pip.

pip is installed automatically with Python 3.

Now install virtualenv, virtualenvwrapper, and also a new ipython because why not.

pip install virtualenv virtualenvwrapper ipython

Virtualenvwrapper will put a file "virtualenvwrapper.sh" in ./localpython/bin. Add this to the end of your .bash_profile now:

source /home/username/.localpython/bin/virtualenvwrapper.sh

You can source your .bash_profile again at this time, or just type the same line into the command line.

Finally, remember to create the projects directory described in your bash_profile:

mkdir projects

b. Project environment

11/3/2014

Make sure to run commands from inside the working directory for that project:

mkproject project_name should create a new env and send you to that project directory.

What have we done?

  1. Installed a "local" version of Python.

  2. You configured your account such that when you run Python as your username, you use that version of Python, not the version that is installed on the server. This means you can use whatever version carries the tools you want. Importantly, this means you can also easily install whatever Python packages you want into your local Python by using "pip install package_name."

  3. You installed Virtualenv, which is another tool for managing Python environments. Say you need one set of tools for your current project, but it turns out that later you work on another project that needs, for example, an older version of the same package, or a later version of the same package. If you just had your local Python, you'd be screwed -- always uninstalling and reinstalling. With Virtualenv, you compartmentalize your local Python into additional versions of Python.

  4. When you typed mkproject project_name, you created a virtualenv called project_name and started working in that environment. You'd see (project_name) at the front of your bash prompt, letting you know that's the environment you're in. If, in that environment, you typed pip install package_name, you would install package_name -- but only in that environment! This means that you can have many different projects on the same machine, even if their Python package requirements conflict with one another.

2. Using the new tools

To create a new project using a specific version of Python, use:

mkproject <project_name> -p ~/.localpython/bin/<symlink for Python executable>

To start working in a given project, type workon project_name.

When you are done working in that project environment, type deactivate.