Although you can achieve this with practically any disto by doing a full install to a USB drive, the problem, especially when you want everything encrypted, is that paging to disk is incredibly slow - and default HD installs do a lot of paging. This route will also thrash away at the I/O on your flash drive greatly reducing it's lifespan.
I have yet to find an ideal solution to this. but here are two distros that come close:
Puppy Linux
Yes, the cute name and clunky gui put me of at first - but it is very well put together technically. It uses a persistent overlay to save all changes into an (optionally encrypted) savefile, which it flushes to disk periodically or on demand. However, although Puppy is now based on Ubuntu, it is heavily modified and stripped down, and includes it's own package management system which I found less than appealing. I was not able to get apt-get running.
Liberte Linux
As far as privacy goes - this is hands down a winner. It randomizes network device mac addresses automatically, and automatically channels all network traffic through Privoxy -> Tor. It has a persistent overlay so files are saved in an encrypted savefile which seems to be flushed to disk in real time. It is based on hardened Gentoo and the firewall blocks all outbound traffic that doesn't go through Privoxy. It even blocks dns lookups that have not been tunneled at the firewall level. The downside is that you cannot install your own software. The persistent overlay does not extend to the general system, and portage is not part of the package (neither are build packages). This is designed as a feature, but in reality really hampers the flexibility of the distro (I will not deny there are security benefits to such an architecture though).
Next on my list to try is Knoppix (I don't know why I didn't think of it before, it was the first linux system that I ever used! now that takes me back...). But apparently it supports a full disk overlay (so software installs will persist) and AES256 encryption of the savefile. But this is just what I've gleaned so far.
[EDIT]
Knoppix
It turns out Knoppix has come a long way since I first tried it. In 6.0 onward it does have full filesystem overlay support for persistence, and gives you the choice to encrypt it with AES256 bit encryption out of the box. Download the latest version, slap it on a flash drive, and customize to your hearts content... I was able to apt-get tor and other privacy tools easily, and they persist across reboots. Also, it has got a lot more lightweight and is based on LXDE now - makes for a very smooth and quick UX. One tip, don't install it on VFAT, use EXT3 instead. VFAT has a maximum 4GB file size which means that your persisted data cannot go over 4GB, it also takes a lot longer to initialize the save file on VFAT than it does on EXT3. I should have known to look to Knoppix first for a live distro.
If its persistent then is it technically not a "live" Linux distro? The solution to your problems with privatix would be to replace the graphics card ( seems strange a light weight linux distro would have a problem with any graphics card ). – Ramhound – 2012-01-18T18:46:47.200
What I mean by persistent is that it should be able to save/keep the changes and applications installed. Privatix does that. – Auxiliary – 2012-01-18T19:08:57.257