Browser with its own hosts file?

36

8

I have a number of staging and test servers that I need to constantly modify my hosts file to access (they depend on the domain name, so I have to change the hosts file to get them to work).

I find this annoying. I'd like to setup a portable browser of some kind for each kind of site I want to work with. Is there any version of any graphical web browser (including browsers based on the rendering engines of other browsers) that will do this?

This way I can simply launch the instance that's already configured to work with staging if I want to test staging.

Any ideas?

EDIT: The important feature is the ability to run two browsers concurently, both browsing to some-address.com, but with one browser directing the reqests to the server at 1.2.3.4 and the other browser directing the requests to the server at 5.6.7.8.

EDIT: (3/16/14)

This question was asked several years ago, and I still don't know of a decent way to do this. Let me try to explain further, because people seem to misunderstand the problem. It's not something that can be easily fixed just by using a proxy server. I suppose it might be possible to write extensive rules in a proxy server to solve the problems, but that's a huge complex solution to what should be a simple problem.

So to clarify:

There are three web servers. Production, Test, and Development. Each runs a multi-tenant application that customizes itself based on the hostname used to access it. Conceptually, this is similar to a Content Management System or blog engine that allows multiple sites to be run off the same server instance. You just point the hostname to the server, and whichever hostname is used to access it via HTTP 1.1 hostname mapping, is the site it displays.

Any solution which simply rewrites the host file doesn't work because of the requirement to run multiple browser windows at the same time to the different sites, production, development, test, etc... So simply constantly altering the host file doesn't work because it affects all browser instances. If you change the mapping for one browser, then the other browsers are also now mapped to that server so any subsequent requests will go to the wrong server.

All i'm looking for is a way to force this browser instance to use this host file, and that browser instance to use that host file, such as through a command line parameter. I don't even care if I need to install multiple copies of the browser.

This may be doable with firefox portable, since I think it uses its own config files per installed copy, but the problem is that the third party software used to administer the sites does not work with firefox (the sites themselves do, but the admin tools do not, they work fine with chrome and IE and Opera, however.. and I have no access to the source for this to fix it).

Erik Funkenbusch

Posted 2010-12-13T23:10:22.093

Reputation: 927

Most people in your situation run a couple of WinXP or Ubuntu virtual machines with different HOSTS configs. – digitxp – 2010-12-13T23:14:40.203

@digitxp - Yes, I'd already thought of that but i really don't want the overhead of multiple VM's and several entire OS's just to browse a couple of sites. Obviously, if there is no other choice that's what i'll probably have to do.. – Erik Funkenbusch – 2010-12-13T23:18:33.603

If all else fails: Fiddler Web Debugger (Windows) or GlimmerBlocker and Charles Web Debugging Proxy (Mac) can rewrite the requests on the fly...

– Arjan – 2010-12-13T23:30:11.873

@Arjan - Proxies will work to some extent, but I would have to have a seperate proxy instance for each browser instance, or change proxy settings each time which is even more annoying that changing host files. – Erik Funkenbusch – 2010-12-13T23:45:35.807

No, you could then use different domain names in your browsers, and rewrite those names using the proxy. (But I know: it's not what you asked for.) – Arjan – 2010-12-13T23:47:46.237

You want (effectively) a separate hosts file "for each kind of site you want to work with" What does the bit I put in quotes mean? – barlop – 2010-12-15T11:06:49.520

1Perhaps a Web Proxy like squid, would let you redirect in a way that does what you want done from DNS resolution? (of course a web proxy can be configured on a per browser basis) – barlop – 2010-12-15T11:15:50.883

this really needs to be a firefox plugin... would be a nice one! – boomhauer – 2012-05-12T17:04:11.047

It looks to me that you need a browser with its own IP stack. – dotancohen – 2012-11-18T16:03:34.393

If setting up a proxy server that does some rewriting would be feasible, then see Easy way to view a website at a different IP than associated with hostname? (cc. @noloader)

– Arjan – 2014-03-16T08:18:44.593

looking at some of your comments you seem to conflate the concept of a separate proxy for different -instances- of a browser, with the concept of a different proxy for different browsers(where all instances of any one browser might or would use the same proxy). The word "instance" has a specific meaning. – barlop – 2014-03-16T16:09:44.387

@barlop - It's not possible to use a different proxy for different instances of IE. What i'm trying to accomplish is to be able to launch different instances of the browser using a different host/ip mapping. – Erik Funkenbusch – 2014-03-16T20:51:26.873

@ErikFunkenbusch I vaguely recall that firefox has an extension called FoxyProxy that lets you say this site can use this ip. That's quite far from what you're asking, I don't know if that helps. Also it may be worth considering proxifiers and transparent proxies, tsocks is an example it's mentioned here http://kakku.wordpress.com/2007/11/18/proxify-any-application-tsocks-and-proxychains-force-any-program-to-communicate-through-a-socks-https-proxy-or-use-cascading-proxies/ and http://superuser.com/questions/319516/how-to-force-any-program-to-use-socks

– barlop – 2014-03-16T23:51:24.200

@ErikFunkenbusch with firefox maybe if each ff instance is launching in its own profile then it could have its own proxy settings or its own foxyproxy settings even. I notice though that for any of those suggestions, you're not getting a separate hosts file. though harrymc's sandboxie suggestion sounds like it may work. – barlop – 2014-03-16T23:52:41.910

@barlop - I already mentioned that I thought a portable ff could have its own settings for each installed copy. But as I also said, the software doesn't work with FF. So neither would foxyproxy, but they're good choices for others. – Erik Funkenbusch – 2014-03-17T06:17:11.950

@ErikFunkenbusch You say 3rd party programs won't work with FF.. so you think foxyproxy wouldn't work with FF. That is confusion on your part. IE,Chrome, and Opera(now though perhaps not in the past), use the windows proxy settings, while firefox uses its own one. So programs that work by changing the windows ones, won't affect firefox. But FoxyProxy does not change windows settings, it/and it is a firefox extension. FoxyProxy is not for Chrome and IE, it is for Firefox! whether it helps you or not is another matter. And >1 solution w/ FF is better than just 1 w/FF. – barlop – 2014-03-17T10:19:11.590

@ErikFunkenbusch furthermore, do you have any comment on dave parrish's solution of multiple squids, each with its own hosts file, and chrome launched with a command line to give it its own squid web proxy and you specify on the command line, the squid web proxy that has the hosts file you want for that instance of chrome. Furthermore, on the subject of proxifiers and transparent proxies, they wouldn't and couldn't work by changing those windows proxy settings that chrome and IE use. They are designed to also work with software that has no proxy set. – barlop – 2014-03-17T18:17:48.220

Answers

19

Squid solved the problem for me. I think many proxy servers could do the same.

Specifically, I had to install Squid. Then change the squid.conf so that it accessed an alternate hosts file. This is the line you want to change:

#hosts_file /etc/hosts

In my case I used hosts_file /etc/hosts-squid. I created the /etc/hosts-squid file and changed the hosts-squid file so that it would give me access to the new IP addresses that I wanted for the given URL. That was all I needed to do to configure the Squid proxy server.

I then created a command to launch my browser with the proxy settings. I used:

chromium-browser --proxy-server=localhost:3128

Running that command effectively give me a running instance of chromium which can access the domains setup in the hosts-squid file that I created.

I was also able to setup more proxy servers using multiple instances of Squid to access a production, development and local website all on the same computer. Instructions on how to run multiple Squid instances is here. I also wrote a blog post explaining the problem and solution in a little bit more detail.

Dave Parrish

Posted 2010-12-13T23:10:22.093

Reputation: 561

8

I believe that you are asking to setup a virtual environment for each browser that includes its own hosts file.

One solution might be to use Sandboxie to set up a separate sandbox, each containing its own hosts file. The browser need not be installed into the sandbox to use the hosts file, just started from inside that sandbox. Desktop shortcuts can be created for launching the browser from within the sandbox.

The disadvantage will be that the browsers are going to be isolated from the real file system for example when downloading files from the Internet, but Sandboxie includes mechanisms for copying such files outside of the sandbox.

Unfortunately, the feature of running programs in more than one sandbox at the same time is only available in the paid version of Sandboxie (15 euros plus tax for one year license for personal use at home). Feasibility testing can be done using the free version.

For Linux users, similar products are :

harrymc

Posted 2010-12-13T23:10:22.093

Reputation: 306 093

Sorry about the late reply harrymc. Its not clear if the program allows one to provide per-sandboxed system files like %SYSTEM32%/Drivers/etc/hosts on Windows and /etc/hosts on Linux. Could you confirm? I'd prefer the Linux version because I did a quick security evaluation of Sandboxie a couple of years ago on Windows (it failed, and the reports to the author went unanswered). – jww – 2014-03-21T01:06:07.180

if Sandboxie supports per-sandboxed system files like %SYSTEM32%/Drivers/etc/hosts, then the points are yours. The OS was underspecified, and its not fair to you to withhold them if Windows/Sandboxie works. – jww – 2014-03-21T01:14:52.420

AFAIK Sandboxie creates a completely parallel file-system and registry. For Linux a simple sandbox program is arkose or maybe sandfox but I have no experience with them.

– harrymc – 2014-03-21T06:19:57.063

Thanks harrymc. I'm having trouble awarding you the points. I see a box with +50, but I can't click on it. Plus, I thought it was worth 100. Any ideas (or do I need to wander over to Meta)? – jww – 2014-03-21T17:42:52.363

See the Bounty FAQ : Of the bounty of 100, one half was awarded automatically after 7+1 days without designated answer, one half was lost.

– harrymc – 2014-03-21T19:47:44.787

harrymc - that's kind of weird. I'm looking at the "Bounty Grace Period Started - Super User" email dated 20 Mar 2014 14:22:00 +0000. It states the 24 hour grace period has started. I'm told to award the bounty, or risk automatic award. I clearly tried to award within the period. I think something is bent/broken here. – jww – 2014-03-21T22:55:16.030

You could ask about it in Meta Superuser.

– harrymc – 2014-03-21T23:14:06.333

4

You can automate the process of changing host file with firefox + HostAdmin

HostAdmin is a Firefox Addon , that helps you modify the Hosts file , switch domain-ip mapping. HostAdmin can understand your Hosts file via a enhanced Hosts file syntax. In additional, HostAdmin refresh your DNS cache automaticlly whenever you change your Hosts file, even manually.

Example config

#==== Project 1
# 127.0.0.1 localhost1
127.0.0.1 localhost2
127.0.0.1 localhost3
#====
#==== Project 2
# 127.0.0.1 localhost1
# 127.0.0.1 localhost2
# 127.0.0.1 localhost3  

More info at https://addons.mozilla.org/en-US/firefox/addon/hostadmin/

Multiple instance

To run two instance of firefox browser concurently and independently in linux create multiple launchers with following script. Also install the plugin in all firefox.

Firefox 1

#!/bin/bash
th="/home/$USER/tmp/p1"
mkdir -p "$th"
export HOME="$th"
firefox -no-remote

Firefox 2

#!/bin/bash
th="/home/$USER/tmp/p2"
mkdir -p "$th"
export HOME="$th"
firefox -no-remote  

You may use the profile option (firefox -P ) in a browser to use native profiles for different setting and may use concurrently.

totti

Posted 2010-12-13T23:10:22.093

Reputation: 832

1He is asking for 2 browsers using 2 hosts files at the same time. – harrymc – 2014-03-16T07:17:26.640

1

As an aside: see How do I format my posts using Markdown or HTML?

– Arjan – 2014-03-16T08:11:54.270

@harrym c updated. – totti – 2014-03-16T15:47:26.633

2

I know this isn't what you're asking for, but a temporary solution could be to write a batch script to swap the hosts files and then claunch the browser.

At least it would minimize the the work you're doing.

Or you could just name these server something entirely different in the hosts file so you never have to switch it to begin with.

Chris Harrod

Posted 2010-12-13T23:10:22.093

Reputation: 269

I can't rename them because the web applications depend on the host name to render the right content. Also, the reason I wish to do this is that i can run multiple browsers at the same time and have one go to staging and another to dev. You can't do that by swapping hosts files. – Erik Funkenbusch – 2010-12-13T23:26:05.813

@MystereMan: If you are talking about META BASE, just do browser detection server-side. You shouldn't have your own host name in any other URL... – Tamara Wijsman – 2011-02-04T14:10:18.820

@TomWij - No, i'm not talking about META BASE. I'm talking about host based virtual hosting, not IP based. The hostname field determines which site is accessed. – Erik Funkenbusch – 2011-02-04T15:46:42.563

@MystereMan: Then your first comment on this answer doesn't make any sense. As you are talking about "the web application rendering the right content", I don't see what virtual hosts have to do with that? – Tamara Wijsman – 2011-02-04T15:51:55.403

@TomWij - We have a multi-tennant CMS system that hosts multiple sites on a single server, with a single IP. To view any given site, you Must use a hostname, otherwise you only access the default instance. I need to switch between production, staging, and development versions of this server, which means changing the mapping of domain names to their IP addresses on the fly. I can do this in the hosts file, but that changes things globally so i can't open simultaneous copies of staging and production in two different browser instances. – Erik Funkenbusch – 2011-02-04T16:20:34.283

@MystereMan: Hmm, I see. But can't you do this for example: x.com = production; stage.x.com = staging; dev.x.com = development; assign those all on the same IP and on the server forward them to the appropriate site. This way, it doesn't change towards the user (assuming it is a server that is live) and you can access the different stages from an machine without changing stuff or opening different browsers. Furthermore, I don't understand "which means changing the mapping of domain names to their IP addresses" as you said you have a single IP and then it would just be like my example. – Tamara Wijsman – 2011-02-04T18:09:54.920

@TomWij - No, changing the name to stage.x.com... changes the name, and thus the web server will not serve the right site. For example, i have two sites. www.example.com and www.foobar.com. The server will serve the correct based only on the domain name www.example.com, if you change it to stage.example.com, it's a different web site as far as the server is concerned. To make this work, we would have to alter every host configuration on the staging and dev servers, and since wed currently have over 50,000 domain names, all served from a database, that would be a challenge to syncronize. – Erik Funkenbusch – 2011-02-04T18:26:59.373

@TomWij - also, we have some sites like www.example.com, nj.example.com, fl.example.com, etc... it's simply not feasible to keep all this in sync between production, dev, and staging. Plus, there are dozens of internal components that are keyed on the host name, and will fail if they get something they don't expect. Again, all i need to do is have a way to say this instance of the browser uses this ip address for this set of names, and that instance uses that ip address for the names. – Erik Funkenbusch – 2011-02-04T18:30:42.163

@Mystery Man: It would be a single query against the database to add "stage." or "dev." to every host name. As for the code, depends on refactoring... Running three instances of Privoxy and running the browsers against those three instances is a client side solution that would work, but server side would be more useful (especially when working in a team)... – Tamara Wijsman – 2011-02-04T19:21:18.797

1

Privoxy can be used for this purpose.

Tamara Wijsman

Posted 2010-12-13T23:10:22.093

Reputation: 54 163

How, exactly, would Privoxy work for this purpose? It looks like just another local proxy server, none of which work because with most browsers you can't define a proxy per instance.. you can only define the global proxy for all instances – Erik Funkenbusch – 2011-02-04T05:48:18.227

@MystereMan: It would proxy your requests and thus manipulate the requests into other requests, it's fairly easy to do that. You are talking non-sense about the proxy servers: Mirosoft Internet Explorer (GUI), Mozilla Firefox (GUI), Google Chrome (CLI --proxy-server=), Apple Safari (GUI), Opera (GUI) and any other browser (GUI or CLI) allows you to set your own proxy server...

– Tamara Wijsman – 2011-02-04T14:05:20.030

@TomWij - No. IE, Chrome, and Safari all use the global proxy server setting in "Internet Settings". That means, if you change it in one web browser, it changes in all of them. Firefox has its own settings for this, but unfortunately I can't use Firefox because it doesn't work with the CMS we use (everything else does). – Erik Funkenbusch – 2011-02-04T15:44:31.897

@Mystere Man: Incorrect. Stop repeating your thoughts and instead see what configuration each individual browser gives you. If you aren't spending the effort to actually read documentation then you will never get to solve your problem as you are asking something too specific that doesn't apply to the rest of the people. As I said before but you didn't seem to have read: Any browser allows you to set the proxy server for that browser. Google Chrome provides the CLI parameter --proxy-server=.

– Tamara Wijsman – 2011-02-04T15:46:51.227

Well, that helps. But, I know of no way to do this in IE, or Safari. – Erik Funkenbusch – 2011-02-04T16:16:40.870

1Hmm, I might be wrong about Safari as I didn't use that for quite some time. But I think I misunderstood you, why would you want to have a different proxy server in every single browser? I'm getting confused as why something like "x.com = production; stage.x.com = staging; dev.x.com = development" wouldn't just work for you given a single IP and host header redirection on the server... – Tamara Wijsman – 2011-02-04T18:16:06.317

1

Don't know if this is convenient enough for you, but one possible solution would be to install IIS on the machine with the browsers and provide server side scripts (like ASP.net, PHP, Ruby...) that change the hosts file and then redirect to the actual page. In your browsers, you then create a bookmark to that script, for example http://localhost/config_ie.rb which changes the hosts file (requires correct permissions on that file, of course) and then do a 302-redirct to the real page.

Michael Kremser

Posted 2010-12-13T23:10:22.093

Reputation: 55

1The part that most people fail to understand about this is that in order to work effectively, you need to be able to run multiple browsers simultaneously, using different host/ip mappings. For example, I need to have both my production and test and debug instances open in different browser instances. Each one has to use the same hostname (because HTTP 1.1 hostname mapping maps the correct site based on the hostname). If you just change the host file, then every instance of the browsers now maps to the new server, not just the single instance of it. – Erik Funkenbusch – 2014-03-16T20:20:43.960

@Erik Funkenbusch: Thanks for clarification. Well, my remaining ideas: a) One concept would be using something like filesystem virtualisation, which is called "Data Redirection" in Windows and is normally used for UAC to support legacy applications that write data to sensitive paths. Maybe there is some option that allows you to use that for your own purposes. Then you could have a custom hosts file for every process. b) DNS interception locally: Request from "iexplore.exe" etc. could be manipulated according to some list you supply. -- Is there some idea that you like? – Michael Kremser – 2014-03-17T07:17:18.577

1

If a full VM is too much, a container-based solution might do.

If we are talking about linux here, lxc might be a feasible solution.

Apart from that - even a full VM with a minimal X11-linux is not that big (max 2 GB disk) and does not consume that many resources (max. 512 MB RAM) with nowadays hardware...

A Lubuntu VM might do here.

Nils

Posted 2010-12-13T23:10:22.093

Reputation: 347

I find the RAM and the processing are OK. The shared disk is what cripples the system. – jww – 2014-03-21T01:13:15.107

0

One relatively simple solution (but maybe a bit resource heavy) would be to have a few virtual machines running in your host. Each virtual machine having it's own host file.

So you would have one virtual machine for the staging environment, one for prod, etc.

Pete

Posted 2010-12-13T23:10:22.093

Reputation: 370

Perhaps you should read the first two comments under the question. This was addressed 4 years ago. – Erik Funkenbusch – 2014-03-18T08:07:28.510

Actually I did see the comment, but not until after I posted this answer. But I still thinks it makes sense to have it as a separate answer (as opposed to just a comment), as it can help other people with the same type of problem. – Pete – 2014-03-18T09:27:50.433

Two VMs would cripple most workstations due to the shared disk bottleneck. I mildly suffer with one VM open (one is modern Mac Mac Book Pro with dual quad cores; the second is a modern ASUS laptop with a single quad core). Two VMs would be painful. – jww – 2014-03-21T01:11:54.757