Are there json2 and 2json (like xml2)?

How can I convert JSON file to format suitable for editing with grep/sed/perl oneliners and back? For XML there's xml2 and 2xml. What for JSON?

I expect it to work approximately like this:

input:

{ "status" : "test\nquest\n", 
  "reply": [1,2,3,
     {"key1":true, "key2":3.4, "key3":[[{}]] }]}

output:

{status"test
{status"quest
{status"
{reply[1
{reply[2
{reply[3
{reply[{key1!true
{reply[{key2=3.4
{reply[{key3[[{

Requirements:

Round-tripping should preserve essentials in the JSON;
Multi-line strings should be supported well, like in xml2;
Each line should describe the full path from the root of JSON tree;
Filtering the output using grep should usually preserve validness of format.
Text should be in UTF-8. Multi-line text should take multiple lines.

Is there already such program of I should implement it?

Vi.

Posted 2013-10-01T20:18:26.507

Reputation: 13 705

Cursory googling suggests you should implement it yourself. Or, you know, don't use grep/sed/perl oneliners, and actually parse the JSON using a JSON parser and read/modify it that way. – allquixotic – 2013-10-01T20:59:18.037

The json2 should correctly parse it. Using json2+2json is like using a JSON library. Consider it as shell scripting JSON library. – Vi. – 2013-10-01T21:03:52.683

suggests you should implement it yourself -> Is there a set of JSON input samples to test the program (with empty objects, Unicode trickery, deep nesting, various escape sequences)? I'm not sure I know the entire JSON... – Vi. – 2013-10-01T21:06:52.160

I don't have a readily available list of JSON files to test, but you can find example code that JSON parser library authors have written, since most of them are open source. Check here, here, and here (the latter seems extremely useful, but there's no promise that dataset is exhaustive!).

– allquixotic – 2013-10-02T12:14:29.150

So this json2/2json thing would, what, basically transform a CFG-class language (JSON) down into a regular language that can be sanely parsed by grep's regular expressions? Even if so, it seems like handling arbitrary hash nesting, etc. would be ridiculously painful in the shell, either requiring recursion (potential stack overflow waiting to happen) or some kind of a stack data structure. If you're going to implement a json2/2json program (which may be generally useful), I still have to wonder, why not just write a program in your favorite language using a json parser lib? – allquixotic – 2013-10-02T12:28:26.750

"The transformation is complex. The syntax used by these tools is relatively intuitive, but difficult to describe precisely.". I expect the similar to be for this pair of programs. – Vi. – 2013-10-02T15:00:06.077

"why not just write a program in your favorite language using a json parser lib" -> I want command line/bash access everywhere. I even implemented a FUSE filesystem in shell scripts. You can't easily process XML and JSON using oneliners, including-but-not-limited-to because of you usually need to import things before using XML and JSON libraries. – Vi. – 2013-10-02T15:02:57.823

Often people use some simple regex or like this to extract a piece of data from XML/JSON; just to avoid dealing with any dependencies and to avoid comprehending any abstractions. Just making the program to the same as I doing. This way makes buggy things relying on identation, absence of funny characters, etc. But with xml2 and friends such approach becomes simpler and more stable. – Vi. – 2013-10-02T15:06:47.677

"I want command line/bash access everywhere" Uh, what? So you're saying that you want to be able to program anything in bash, yet, you're relying on a C/C++/Ruby/something program that you're about to write (the non-existant json2 that will soon exist if you succeed) to get your bash code working? To be frank, this sounds very hair-brained. I would just write all the code in Ruby for a task that requires parsing, because even if you manage to implement a json2 utility, you're going to have to distribute it to any system that you end up writing your bash script on... – allquixotic – 2013-10-02T15:16:42.773

Even the xml2 utility is not universally available. It doesn't exist on my Android phone's busybox shell. It doesn't exist on my GNU/Linux laptop out of the box. If you're looking for extreme portability, this isn't going to get it done. If you're able to compile binaries or run scripts in a real programming language on any computer you need to work on, then why not write your actual parsing code in that language? So much easier and more reliable, because it looks like even your json2 utility would hit problems for trying to deal at the wrong level of the Chomsky hierarchy. – allquixotic – 2013-10-02T15:18:22.860

If avoiding dependencies is your big worry, there are embeddable JSON parsers that basically involve copy-pasting code into your source file. They are available for e.g. Ruby, Python, C++. You don't have to have some library package installed on the system, you just include the parser as a part of your own code. And you avoid the fundamental problem of trying to "parse" a higher level of the Chomsky hierarchy (CFG) using a stricter level (Regular language). – allquixotic – 2013-10-02T15:22:59.943

http://chat.stackexchange.com/rooms/118/root-access – allquixotic – 2013-10-02T15:23:27.757

Are there json2 and 2json (like xml2)?

Answers