Ever since I stumbled across http://lowendbox.com, I've been like a kid in a candy shop; from the 256MB/1vCPU for $1.65/mo., to the 2048MB/4vCPU for $7.00/mo. my current CHUNKHOST seems expensive by comparison. But since every platform doesn't support every operating system (OpenVZ for example can't support openSUSE 12.3 or Ubuntu 13.04 like it can on the more expensive Xen or KVM - something about the kernel level though I've read conflicting reports) you have to be prepared to make some sacrifices and be familiar with a wide range of distros.
My oldest VPS is running Ubuntu 12.04 LTS, and my newest CentOS 6; but my favorites are running openSUSE 12.3. I found a good article explaining why its DAFT PUNK to run 32-bit on lower end machines. That aside, of the two Dartmouth boxes I have, the one in Frankfurt is 64-bit - currently hosting http://newtonkansas.us, and my xenchunk (Ubuntu) is also 64-bit.
Then I found two more. Got one on special for $12/annually, and another I'm going to let drop at the end of this month. This latter one I have been rebuilding with openSUSE 11.4 and incrementally upgrading it versus skipping versions (HINT: I don't recommend skipping versions). When I used to run Solaris and HP/UX at home, I always put Big Brother monitor on there so I could keep an eye on them at work, but with these numbers, I may start deploying BB just to keep track of them all!
I'm currently teaching myself how to build an SMTP relay server. With all these boxes, I think it will be nice to have a private, centralized, mail server.
Lest we forget, I'm also running openSUSE 12.3 on physical server-class hardware and as a vbox vm on my desktop - YES I FIND I HAVE PLENTY OF REASON TO DO SO :P