Peer Pressure

TiddlyWeb DreamHost Experiments

These are rough notes for installing TiddlyWeb on a DreamHost hosted virtual domain. This is a fairly technical dump of some things I tried. It is not yet a straight up cookbook because there are some issues to be resolved before it can be. It might not look like it from all this text, but all of this could be compressed down to a very short shell script. That will come soon(ish).

I’ll get the big issues out of the way first. As I was winnowing down the steps involved to get things to go, it became clear that the Python (version 2.5) installed on the host was not dealing well with namespaces packages. This is unfortunate because that’s how we’ve started packaging tiddlywebplugins. The bug revealed itself as each subsequent plugin clobbering the contents of the tiddlywebplugins directory. So this process requires a custom built and installed Python. That’s covered below.

The second issue is that pip, during the install process, would like to do some renaming of temporary directory structures in. On large servers /tmp is often on another filesystem and the code pip uses explodes under those conditions. There’s a workaround by setting TMPDIR in the environment. That’s covered below too.

Step Zero: Install a New Python

Installing a modern Python keeps the namespace issues described above at bay. It needs to be done first, because what we do later requires the good Pythonic base.

I cribbed from Ryan Kanno to get Python built and installed. My slightly modified way to go is below. For the Python source URL, use whatever is under the “Source Distribution” quick links for Python 2.x.x on http://www.python.org/

$ mkdir opt
$ mkdir downloads
$ cd downloads
$ wget http://www.python.org/ftp/python/2.6.4/Python-2.6.4.tar.bz2
$ tar jxf Python-2.6.4.tar
$ cd Python-2.6.4
$ ./configure --prefix=$HOME/opt/ --enable-unicode=ucs4
$ make # and assuming all goes well
$ make install

In $HOME/opt/bin there should be a python. Check its version:

$ cd
$ opt/bin/python -V
Python 2.6.4

Step One: Install and Setup VirtualEnv

virtualenv makes it easier to manage Python modules on a host for which you do not have root. It’s likely since we’ve installed our own Python that we wouldn’t need this step, but I’m hoping DreamHost will fix their Python, and Step Zero will go away and we can start counting from One.

For this step I cribbed from the DreamHost wiki. In these instructions we’ll use the Python we built. For the URL for virtualenv, copy the tarball link at the end of http://pypi.python.org/pypi/virtualenv

The following makes your home directory the root of your virtualenv. If you want to use many virtualenvs, this is probably not what what you want to do. But if you just want a single nice Python environment, this’ll do.

$ cd ~downloads
$ wget http://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.4.3.tar.gz
$ tar zxvf virtualenv-1.4.3.tar.gz
$ ~/opt/bin/python virtualenv-1.4.3/virtualenv.py $HOME
$ echo "source bin/activate" >> ~/.bash_profile # activate updates your path nicely
$ cd
$ . .bash_profile # get those path changes

Now the python command in $HOME/bin is your main and active Python.

Step Two: Install TiddlyWeb

This is all automagic. The TMPDIR expression works around the pip problem described above.

$ cd
$ mkdir tmp
$ TMPDIR=$HOME/tmp pip -E . install -U tiddlywebwiki

A fair number of things will be installed from http://pypi.python.org/. When it is done, when you run:

$ ls bin

You should see twanager and twinstance in there.

Step Three: Create Your Instance

Now we will create our TiddlyWeb instance, which will do outside any web exposed areas. We don’t want to accidentally leak direct access to the store. You can name your instance whatever you like. We’ll call this one aleph.

$ cd # go home
$ twinstance aleph
$ cd aleph
$ ls
store  tiddlywebconfig.py

Now we will configure our instance so it is smart enough to run. We need to set two things in tiddlywebwconfig.py: server_prefix and server_host. Set server_prefix to ‘/index.cgi’ for now. Set server_host according to your hostname. If it is aleph.example.com then it will be:

'server_host': {
    'scheme': 'http',
    'host': 'aleph.example.com',
    'port': '80',
}

(Make these changes using your favorite editor on tiddlywebconfig.py.)

Step Four: Make it Go as CGI

In the DreamHost setup I was using, the web server’s document root is in playground/public. I’m not familiar enough with DreamHost to know if this is normal. I guess there can be several per user, so you’ll have to map the following to your own setup.

$ cd ~/playground/public
$ wget http://github.com/tiddlyweb/tiddlyweb/raw/master/index.cgi
$ chmod 755 index.cgi

Edit index.cgi to make 3 changes:

1. Change the first line to the full path of your Python: #!/home/your username/bin/python.
2. Set tiddlywebconfig_dir to your instance directory: '/home/your username/aleph'.
3. Just to be sure change, os.environ[‘PYTHON_EGG_CACHE’] to: '/home/your username/tmp''. You created thistmp` directory earlier up the page.

Go to /index.cgi/ at your host. For example: http://aleph.example.com/index.cgi/. That trailing / is required (for now). To see the default wiki try your version of http://aleph.example.com/index.cgi/recipes/default/tiddlers.wiki.

Note that because DreamHost runs the CGI script as your user (via suexec), there are no weird permissions modifications you need to make here: new files are created just fine.

If everything has gone well up to now, that should do it. What follows in Step Five is a bonus.

Step Five: Make it Go with Passenger

Passenger is a tool for hosting Rails, Rack and WSGI applications on Apache (amongst other servers). DreamHost has made it pretty easy to use for WSGI things. To use it, change to the directory above your public directory and install some more stuff:

$ cd ~/playground
$ wget http://github.com/tiddlyweb/tiddlyweb/raw/master/apache.py
$ mv apache.py passenger_wsgi.py

Edit passenger_wsgi.py in a similar way to how index.cgi was edited above:

1. Uncomment the two lines containing INTERP. Change INTERP to the path to your Python. In our examples that is /home/your username/bin/python (replace your username with the actual username).
2. Set dirname to '/home/your username/aleph', or wherever your tiddlywebconfig.py is.
3. Just to be sure change, os.environ[‘PYTHON_EGG_CACHE’] to: '/home/your username/tmp''. You created thistmp` directory earlier up the page.

(Observant readers will note there’s a fair amount of conceptual and actual duplication between apache.py and index.cgi. This is a known bug and things are converging towards fixes.)

Go back to the instance directory and edit tiddlywebconfig.py to comment out server_prefix. Then visit the web, this time without index.cgi: http://aleph.example.com/ should give links to recipes and bags. Click around and you’ll eventually be in a TiddlyWiki.

Down the Road

Eventually all this can probably be compressed into two steps, something similar to the following. Still waiting on additional experimental results.

$ wget http://tiddlyweb.example.com/quick_install.sh
$ sh install.sh domain.example.com

And once we get to that point, it’s reasonable to start thinking about things like:

$ scp tiddlyweb-installer.cgi my.host.example.com:public_html
$ open http://my.host.example.com/tiddlyweb-installer.cgi

It was a bit of a silly night, so somewhere in there this video was created: