For anonymous web surfing, at a minimum, two components are required: a proxy and a browser that doesn’t identify you. At the office, I have both and much more to protect my privacy and provide anonymity. If I have to use a Windows computer at a client’s offices, then temporary measures have to be undertaken.
The simplest solution for this, without using an anonymous VPN, is the JonDo Proxy program that will hide your IP address (Java application) and JonDoFox, a Firefox profile optimized for anonymous and secure web surfing. Using the USB doesn’t leave any traces on the computer for some snoop at the client’s office to uncover. This need Windows as the OS.
For more privacy and anonymity, you can use JonDo/Tor-Secure-Live-DVD, a secure, pre-configured environment for anonymous surfing and more. This has its own OS based on the Debian GNU/Linux OS. The live system contains proxy clients for JonDonym, Tor Onion Router and Mixmaster remailer and much more.
The advantage of the live system is that it is on a DVD, which prevents any other system from writing something dangerous to the DVD.
Using these do not make it impossible to uncover individual users, as there is no such thing as a 100% security, but for most users, this will be adequate for most situations. If you are concerned about this, I suggest you read the surveillance reports on the law enforcement page.