How to decode/decipher Mozilla Firefox proprietary .jsonlz4 format? (sessionstore-backups/recovery.jsonlz4)

19

6

I'm trying to get a handle on Mozilla Firefox's proprietary file format .jsonlz4, used, for example, for sessionstore-backups/recovery.jsonlz4, but to no avail.

How do I get back my data, specifically, some long text I've typed in some textareas of a crashed session? It's my data!

cnst

Posted 2018-10-04T00:35:21.313

Reputation: 1 857

10I wouldn't call the format proprietary. Granted, it's custom, not used anywhere outside Mozilla projects, but since the whole of Firefox—including the relevant (de)compression code—is free and open source, this format shouldn't be called proprietary. (P.S. I'm not talking of the branding, which is licensed differently.) – Ruslan – 2018-10-04T09:46:50.780

2@Ruslan, but it is in fact proprietary — just because it's OSS doesn't make it non-proprietary, as there are zero standard tools to look into the content of these files, whereas all other files, even Java's JAR format, can easily be managed with 100% standard non-proprietary tools that are available in ports/packages of every decent UNIX system. OTOH, it is completely non-trivial to actually get back your own data from these .jsonlz4 files. – cnst – 2018-10-06T02:23:26.527

2JsonLZ4 was a bad idea. – neverMind9 – 2019-03-11T23:31:03.897

Answers

19

There's few Google results that actually result in doable solutions, but, as per https://www.reddit.com/r/firefox/comments/2ps6wg/jsonlz4_bookmark_backups/, the following appears to work most reliably:

  • in about:config, toggle the devtools.chrome.enabled setting from the default of false to a value of true

  • open Scratchpad from within Firefox:

    • either with fn+Shift+F4 on a MacBook,
    • or Shift+F4,
    • or through the menu bar through ToolsWeb DeveloperScratchpad
  • in the menu bar within Scratchpad of Firefox, change Environment from Content to Browser (omitting this step would subsequently result in errors like Exception: ReferenceError: OS is not defined at the next step)

  • use code like the following within the Scratchpad of Firefox:

    var file = "/Users/…/sessionstore-backups/recovery.baklz4";
    //OS.File.read(file, { compression: "lz4" }).then(bytes => 
    //  OS.File.writeAtomic(file + ".uncompressed", bytes));
    
    OS.File.read(file, { compression: "lz4" }).then(bytes => {
      OS.File.writeAtomic(file + ".uncompressed.stringify",
        JSON.stringify(JSON.parse(new TextDecoder().decode(bytes)),null,1))
    });
    

    The final parameter to JSON.stringify handles how many spaces would be used at each line; putting 0 causes the whole thing to be printed on a single line, putting 1 splits the lines properly (putting 2 would create too much useless whitespace and increases the size of the file for little benefit)

  • click the Run button

  • run fgrep :textarea /Users/…/sessionstore-backups/recovery.baklz4.uncompressed.stringify from within the Terminal app

cnst

Posted 2018-10-04T00:35:21.313

Reputation: 1 857

This doesn't seem to work in FF 72 on Linux. Scratchpad has been removed, but the Console now supports multi-line mode. However, the OS.File.read(... line gives : "ReferenceError: OS is not defined". – mivk – 2020-01-17T17:47:50.600

@mivk that error is mentioned above — happens due to wrong environment; is there no way to chance the environment? – cnst – 2020-01-17T21:42:10.580

I have not found this "Environment" setting or anything similar in FF 72.0.1. – mivk – 2020-01-17T23:37:38.167

12

Unfortunately, due to a non-standard header, standard tools won't work. There's an open proposal to change that. Apparently the Mozilla header was devised before a standard lz4 frame format existed; it does wrap a standard lz4 block.

That said, the same bug report includes a few alternative methods. I'll list them briefly:

  • Use the dejsonlz4 tool, which includes binary builds for Windows and should be easy to build on *nix
    • lz4json is a similar tool, but relies on an external liblz4 and is somewhat easier to build on *nix but harder on Windows (outside WSL)
  • Use this fairly simple Python script: https://gist.github.com/Tblue/62ff47bef7f894e92ed5 (requires the lz4 package via pip or your package manager) -- the script appears to be python3 but is trivially adaptable to python2
  • There is a webextension available that should be able to open these. NB: while source is available, I have not verified it, and the permissions it requests are a bit concerning (especially the response to concerns)
  • In theory, you should be able to strip the first 8 bytes (e.g. with dd if=original.jsonlz4 of=stripped.lz4 bs=8 skip=1) and that should leave you with a valid lz4 block. Note that this is distinct from a lz4 frame. While most programming languages have libraries that can easily decode a block, finding a prebuilt tool to do so is more difficult, e.g. the liblz4-tool package only accepts the frame format.

Bob

Posted 2018-10-04T00:35:21.313

Reputation: 51 526

1Why was LZ4 necessary in first place? LZ4 is an absolutely moronic idea. – neverMind9 – 2019-06-12T18:18:08.023

BTW, here's the lz4json that compiles cleanly on UNIX®, Mac OS X, MacOS, FreeBSD, OpenBSD and NetBSD — https://github.com/cnst/lz4json.

– cnst – 2019-12-29T06:04:54.007

5

I was able to extract the URLs from the {profile-dir}/sessionstore-backups/recovery.jsonlz4 file using the following free online tool designed expressly for this purpose:

https://www.jeffersonscher.com/ffu/scrounger.html

The same site offers a similar tool for decrypting jsonlz4 files from the {profile-dir}/bookmarkbackups directory.

MikeOnline

Posted 2018-10-04T00:35:21.313

Reputation: 176

2Also just found this, works flawless. – lowtechsun – 2019-02-25T08:35:07.980

1

On UNIX® and UNIX-like systems, like Mac OS X with MacPorts, FreeBSD, OpenBSD or NetBSD with pkgsrc, the following https://github.com/cnst/lz4json fork of lz4json could also be used to compile cleanly out of the box, e.g., on Mac OS X w/ MacPorts:

sudo port install lz4
git clone https://github.com/cnst/lz4json.git
cd lz4json
make
./lz4jsoncat ~/Library/Application\ Support/Firefox/Profiles/CHANGE\
THIS.default/sessionstore-backups/recovery.jsonlz4 \
| python -m json.tool | fgrep :textarea | more

cnst

Posted 2018-10-04T00:35:21.313

Reputation: 1 857