How can this website reidentify me even after deleting all of my browser's history and using a VPN?

222

The website dropmail.me is able to successfully reidentify me (and offer my last used temp mail addresses via. "Restore access") despite doing the following:

Delete all my browsers history which includes cache, cookies, website settings, download history, search history, browser history and active logins. Basically everything that can be deleted through the Firefox menu. I'm using Firefox 52 ESR.
Use a VPN (that according to their claims is safe against IPv6 and DNS leaking) that I have not used when I previously visited this website.
Using uBlock Origin and uMatrix

Additional information:

My "identity" must somehow be bound to my current browser profile. When I use a different browser or a new browser profile, the website doesn't reidentify me as the same person. Actually, it is sufficient to use the Firefox addon Priv8 and create a new sandbox to be identified as a different person. This might indicate that there is some kind of storage for websites that can't be accessed or deleted though Firefox. (It's not Flash cookies, the website doesn't use Flash!)
(Update) Other browsers are not affected. Microsoft Edge, after deleting browser history, doesn't allow reidentification. This is a Firefox-only issue!

My questions are:

How on earth are they able to reidentify me? Since their only motivation to reidentify me is to offer access to previously used mail addresses, I don't think they use any "dark" techniques like fingerprinting, but of course it can't be ruled out.
How can I protect from this kind of "super-tracking" used by this website?

manuel

Posted 2017-09-16T15:33:06.977

Reputation: 1 958

6Use incognito mode when you visit. Chrome and IE has it, I'm sure Firefox does too. – Appleoddity – 2017-09-16T15:44:06.840

1Do you login at any point? – LPChip – 2017-09-16T15:47:36.763

5@Appleoddity: Yes, incognito mode helps, but as far as I understand this just prevents websites from storing or reading browser history etc. So when I delete everything this should have the same effect but it doesn't. Maybe a bug in Firefox? – manuel – 2017-09-16T15:49:03.530

1@LPChip: No. I don't – manuel – 2017-09-16T15:49:30.810

I strongly suspect the evil that is evercookie

– Prime – 2017-09-16T20:40:35.853

2@Prime, in this very case it's not. Manuel is right: "Since their only motivation to reidentify me is to offer access to previously used mail addresses, I don't think they use any "dark" techniques" and peeking in the code you'll see they're simply using standard web technology. Firefox is to blame here, in this specific case. – Arjan – 2017-09-17T08:15:09.767

With permanent ever-cookies!

– Chloe – 2017-09-18T03:02:27.617

1If you use a whitelist-only cookie policy (i.e. blocking them by default) then sites can't use indexedb, also preventing such tracking – the8472 – 2017-09-18T21:29:10.613

Even clearing everything still won't protect against fingerprinting

– o11c – 2017-09-19T05:35:06.963

@o11c True, but you cannot use fingerprinting to uniquely identify "new" users, only to re-identify returning users – cat – 2017-09-20T00:24:10.647

Even if you reinstall your entire browser, you may still be identifiable by your installed fonts, plugins, even your connected monitor! There's really very little you can do to avoid that sort of profiling.

– KlaymenDK – 2017-09-20T20:00:54.990

@cat: If they see a fingerprint they haven't seen before, its likely to be a new one. btw https://amiunique.org is one of the many sites to figure out a bit on how unique you are yourself...

– PlasmaHH – 2017-09-21T08:25:46.633

I would try deleting cookies: based on what you are saying about incognito mode helping that would be the most likely culprit. Incognito mode stops the persisting of cookies, amongst other things:) – GMasucci – 2017-09-22T08:04:37.643

1@GMasucci: Did you also read the question and have you noticed that there is already an accepted answer? – manuel – 2017-09-23T10:36:02.370

Answers

253

The website is using IndexedDB, for which MDN writes:

IndexedDB is a way for you to persistently store data inside a user's browser. Because it lets you create web applications with rich query abilities regardless of network availability, your applications can work both online and offline.

Not clearing it sounds like a bug in Firefox indeed, but apparently the developers feel otherwise. Like in March 2015, someone wrote:

But even when you delete all your history information the data from IndexedDB persist.

The right way to delete this data is by going to about:permissions address, look for the domain and pressing the Forget About This Site button.

While about:permissions does not work in my Firefox 55, going into Tools, Page Info, Permissions I get the button "Clear Storage":

Even worse, neither the greyed-out "Use Default: Always Ask" in the above screen capture, nor enabling "Tell you when a website asks to store data for offline use" in settings, Advanced, Network, have any effect to avoid storage:

It seems the following from August 2011 may still apply (where "[only]" is added by me):

By default in Firefox 4, a site can use up to 50MB of IndexedDB storage. [Only] If it tries to use more than 50MB, Firefox will ask the user for permission [...]

In Firefox for mobile devices (Google Android and Nokia Maemo), Firefox will [only] ask for permission if a site tries to use more than 5MB [...]

To disable it altogether, go to about:config and disable dom.indexedDB.enabled. However, beware that such might affect plugins/add-ons as well, which seems to be why some want to remove that option, for which someone noted in May 2016:

Until IndexedDB is handled in the same way as cookies with respect to accepting/clearing and third-party behavior, this pref should exist.

(One may find dom.storage.enabled interesting too...)

Arjan

Posted 2017-09-16T15:33:06.977

Reputation: 29 084

153

Indeed. Wow. That's a big deal. I didn't now there is such a loophole in the browser which is "obsessed with protecting your privacy".

– manuel – 2017-09-16T16:32:50.210

2@manuel, I added an about:config setting and some more references. Not very satisfying... :-( – Arjan – 2017-09-16T17:29:33.197

...but, @manuel, disabling it might affect plugins/add-ons, so beware. I'm out of here ;-) – Arjan – 2017-09-16T17:46:52.417

23Disabling it even breaks the website mentioned before, and probably other websites too. Let's hope Mozilla will have a second look at this. One solution might be to delete this storage after I close my browser and only selected site which I trust can have their permanent storage. (this is the way I handle cookies at the moment) – manuel – 2017-09-16T17:56:44.670

– Bob – 2017-09-16T18:00:00.917

2@manuel I wonder if they have removed all of the Netscape Navigator code from Firefox? – trognanders – 2017-09-17T00:33:17.347

You can view offline storage use using about:preferences#privacy if about:permissions doesn't work for you. – unpythonic – 2017-09-18T04:43:02.480

The german media mentions this topic: https://heise.de/-3835084

– StanE – 2017-09-19T12:21:12.427

27It's always been profoundly stupid that Mozilla treat IndexedDB differently from cookies. It's still blatantly a kind of cookie. The protocol is different, but clearly if I want to block or delete all cookies I don't care about the protocol. Unfortunately, Mozilla's practice is always "add features now, sort out the privacy implications years from now, if ever". @manuel Try wading through about:config some time. There is truly a lot of scary stuff built into Firefox and enabled by default. Mozilla's alleged pro-privacy stance is pure marketing, and nothing more. – Boann – 2017-09-19T13:40:39.333

@BaileyS Probably. That's why Netscape lost the (earlier) browser wars after all and had to rebrand and remarket completely. It was said that they started from scratch, though of course it might be possible they copy-pasted some of the code. But I wouldn't worry that Firefox uses any of Navigator's IndexedDB code, given that it didn't exist at the time :D – Luaan – 2017-09-23T08:46:23.170

This is eye-opening, but could you extend the answer with a brief comment on how other browsers do on IndexedDB privacy? – alexis – 2017-09-24T13:33:28.537

@Arjan Is there a way to reduce the threshold of 50MB for IndexedDB to something like 0MB? That way, Firefox will always ask for permission when a site want to store the data. – RogUE – 2017-09-24T15:45:10.060

1The Mozilla bug report is now marked as resolved and it sounds as though they've actually updated the "clear data" settings to include the indexed db when removing offline data. Not sure which patch/release it will be in, though. – Roddy of the Frozen Peas – 2017-09-25T05:43:26.613

The leonardcamacho.me link is broken. Archive.org has a stubby broken copy at https://web.archive.org/web/20170919135015/https://leonardcamacho.me/how-tot-delete-data-from-indexeddb-on-firefox/

– Max Barraclough – 2018-01-31T11:32:53.677

@RoddyoftheFrozenPeas Good spot. From the Bugzilla page: "Marking the 52.5.0esr build as verified fixed." – Max Barraclough – 2018-05-14T09:02:59.257

As noted by Arjan it's unfortunately easy to leave site-data installed currently. This is improving somewhat with the preference UX redesign in FF57.

For example, under "Privacy and Security" there is now a "Site Data" section:

Clicking the site data "settings" will allow you to remove site-data for a specific origin:

This will remove data stored in IDB, Cache API, etc. It will also remove cookies for the origin:

(Sorry for not making this a comment under Arjan's answer, but I wanted to include these screenshots.)

Disclaimer: I am a Mozilla employee

Ben Kelly

Posted 2017-09-16T15:33:06.977

Reputation: 689

4Any idea if you're also planning to actually prompt the user for permission to store the data to start with, even if it's just a single bit? (I've read somewhere that for third-party sites the setting for "allow third-party cookies" also applies to IndexedDB, but I've not tested that.) The "Offline Web Content and User Data" setting is quite deceiving, I feel, as it apparently does not apply to IndexedDB. – Arjan – 2017-09-17T08:23:46.180

My comment about its "not a nuke everything for this origin" comment was wrong. It does actually delete cookies as well. I'll update the answer to reflect this. – Ben Kelly – 2017-09-17T13:42:31.557

8Its a difficult balance between prompting when appropriate and prompting too much. Right now storage is design around the idea sites can use storage without a prompt, but that the browser is free to delete it under pressure. If the site wants persistent storage then they need a prompt. Storage APIs are disabled in 3rd party iframes when 3rd party cookies are disabled. In the future we may move to "double-keying" the origin based on the top-level window origin (like safari has done) which would further isolate storage. In FF this is called "first party isolation" and comes from TOR project. – Ben Kelly – 2017-09-17T13:50:26.950

Other solutions you can use today: 1. Private browsing mode prevents all storage. 2. Containers isolate storage (https://blog.mozilla.org/firefox/introducing-firefox-multi-account-containers/). Be aware, though, that preventing/isolating storage will not prevent all fingerprinting.

– Ben Kelly – 2017-09-17T14:07:23.497

7@Ben Kelly: It still doesn't cover the use case "allow every website to store everything to maintain functionality but automatically delete storage at browser exit" Right? Private browsing is not a nice solution either since it keeps nothing at all (maybe I still want to keep my browser history or storage for sites I really trust. And no, I don't want to switch between normal and private browsing all the time.) – manuel – 2017-09-17T14:16:43.290

You're correct out "delete on exit" cookie setting does not apply to things like IDB. I filed a bug here https://bugzilla.mozilla.org/show_bug.cgi?id=1400678. I believe our overall permissions UX is being reworked as well, but I'm not sure the kind of storage restrictions being talked about here are included or not. I filed a bug for that as well: https://bugzilla.mozilla.org/show_bug.cgi?id=1400679. We are working to improve these features, but its an incremental process. Sorry for the issues.

– Ben Kelly – 2017-09-17T15:30:38.003

Edit: Please read the comment by Ben Kelly before messing with any files in your profile.

Since there is no solution inside Firefox, one can easily implement a temporary fix for this outside of Firefox. IndexedDB files are stored in the directory <profile>/storage/default. By emptying this folder (for example though a scheduled script) you can recover full control over your data and how long it persists. Since each website is stored in a separate folder, you could even implement a whitelist/blacklist or basically every policy you want, given you have some programming experience.

It's not a nice solution and no excuse for Firefox developers to continue to postpone a proper solution for this. (Bugreports exist for years now!)

And be aware that the data format and location might change over time. For example, in a previous version, all IndexedDB data was stored in a single SQL file.

manuel

Posted 2017-09-16T15:33:06.977

Reputation: 1 958

2Note that this can corrupt your profile for certain sites. Some state (like service worker registrations) are stored outside this directory. Sites can get confused if the storage is removed, but the service worker registration remains. Our new "Site Data" removal UX is shipping in november and is a better solution. Or, just run in private browsing mode which disables all storage. – Ben Kelly – 2017-09-17T14:00:33.517

1@BenKelly "...are stored outside this directory." What does that mean? Stored where? How do we delete that part too? – John1024 – 2017-09-23T06:04:58.777

2We don't support arbitrary changes to the profile directory. If you start manually deleting stuff then don't be surprised if your profile becomes corrupted and sites don't work correctly. I really don't recommend it. Some of the issues raised by the original question here are being fixed as we speak. Also, private browsing, containers, etc have been mentioned as immediate solutions. Please don't hand modify your profile. – Ben Kelly – 2017-09-24T00:28:08.140