Bloatware contest: producing 100+ MiB executable

22

3

Create short source code in your favourite compiled language that compiles into a big (not less than 104857600 bytes) executable file. The program must be runnable (assuming 1GB of free memory) and can do anything (preferrable something simple like a hello world).

Usage of non-obvious tricks is encouraged.

Boring example in C:

int a[1024*1024*25] = { 1 };

int main(){}

Bonus points if it can be "explained" why the executable can't be reduced in size (i.e. all bloat is actually used somehow).

Vi.

Posted 2013-12-18T20:35:33.590

Reputation: 2 644

Question was closed 2016-04-19T14:27:25.613

This could stay on-topic if the winning criterion was changed to "largest output file" or something, but that would invalidate the current answers and make this a duplicate of at least one other challenge. See The state of the popularity contest tag

– cat – 2016-04-18T21:37:04.460

@cat, Can you please review all my questions and give recommendataion: shall I continue to invent questions like them or they generally no loger fit here?

– Vi. – 2016-04-18T21:39:29.470

7Statically link ALL the libraries! – marinus – 2013-12-18T20:40:27.267

That's why initially thought about 10+ MiB, but revised to 100+... Or it means all libraries in the system? – Vi. – 2013-12-18T20:45:00.110

Can a HTML file be considered as an executable? – xem – 2013-12-21T19:07:09.027

Unlikely. – Vi. – 2013-12-22T01:29:01.973

Answers

13

OK, here's another one in C, going for the vaguely defined bonus points:

#define a(x) x,x|1,x|2,x|3,x|4,x|5,x|6,x|7
#define b(x) a(x),a(x|8),a(x|16),a(x|24)
#define c(x) b(x),b(x|32),b(x|64),b(x|96)
#define d(x) c(x),c(x|128),c(x|256),c(x|384)
#define e(x) d(x),d(x|512),d(x|4<<8),d(x|6<<8)
#define f(x) e(x),e(x|2048),e(x|4096),e(x|6144)
#define g(x) f(x),f(x|8192),f(x|4<<12),f(x|6<<12)
#define h(x) g(x),g(x|2<<14),g(x|4<<14),g(x|6<<14)
#define i(x) h(x),h(x|2<<16),h(x|4<<16),h(x|6<<16)
#define j(x) i(x),i(x|2<<18),i(x|4<<18),i(x|6<<18)
#define k(x) j(x),j(x|2<<20),j(x|4<<20),j(x|6<<20)
int u,v,z[]={k(0),k(2<<22),k(4<<22),k(6<<22)}
int main(){for(u=v=0;u<1<<25;u++)v|=u!=z[u];return v;}

Basically, at compile time, it build a ascending sequence of integers from 0 to 225 − 1. At runtime, it verifies that the sequence indeed contains the expected values, and if not, returns a non-zero error code.

Ps. If I did my math right, the executable should be over 100 MiB. I'll let you know the exact size once it's done compiling...

Ilmari Karonen

Posted 2013-12-18T20:35:33.590

Reputation: 19 513

1Ps. My attempts to verify the actual size have been (hopefully temporarily) stymied by what I suspect to be a rather unusual GCC error message: virtual memory exhausted: Cannot allocate memory. o_O Will try to tweak options to see if I can get it to compile somehow. – Ilmari Karonen – 2013-12-19T16:34:23.083

12

Aww, I was just picturing you standing on a rolling chair and having a sword fight with another developer.

– Iszi – 2013-12-19T18:16:42.007

Also can't build with clang (ICE) and tcc. – Vi. – 2013-12-20T14:43:58.877

1Turn off all optimization (-O0) to minimize the demands on the compiler, and enabling pipes (-pipe) might or might not help. – dmckee --- ex-moderator kitten – 2013-12-21T02:35:19.917

@dmckee: I've tried -O0. Will try -pipe later, once I'm back at my desktop again. (Ps. If I cut it down to 1/4 of its current size, it does compile, giving an executable a bit over 32 MiB. That suggests that the principle is sound, I just need to find a beefier compile server. Now that it's weekend, I might try it on one of the University servers.) – Ilmari Karonen – 2013-12-21T11:37:20.533

3

That compiling problem reminds of a winning entry of IOCCC who write his own preprocessor to verify that the program was correct: http://www.ioccc.org/2004/vik2.hint

– Christian Semrau – 2013-12-22T10:03:23.687

@IlmariKaronen When I attempt compilation I get the error cc1.exe: out of memory allocating 65536 bytes haha. Perhaps if I had a 64 bit build of GCC it could work. – chbaker0 – 2013-12-30T07:29:09.527

wasnt able to build, too (64bit GCC; 12GB ram with windows;) Got up to 10GB of ram usage and heavily filled pagefile and it didnt finish....) had to restart firefox after due to it being pagefiled and getting laggy – masterX244 – 2014-02-17T19:23:27.130

6

C#

Not sure if this qualifies as short, because the source code ended up being >30k :)

I.e - too big to quote. Here's a slightly shortened version of it

using System.Collections.Generic;
class Program
{
    static void Main()
    {
        var a = new List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<List<int>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>();
    }
}

The code I actually compiled can be found here: http://pastebin.com/R5T3e3J0

This will create a .EXE file of ~45KiB when compiled without optimizations. Compile it again with Ngen.exe (Native Image Generator) and it becomes a whopping 104MiB!

This works due to how the CLR generic type system works. Each and every List<> in the above code will generate a new type declaration (normally through JIT compilation, but Ngen performs AOT compilation). So one type for List< int >, another for List< List< int > >, and so on. So for this code, a total of 5160 different generic lists will be created.

Christian Palmstierna

Posted 2013-12-18T20:35:33.590

Reputation: 211

1What you need is a script that will write your program. – hildred – 2013-12-29T15:28:49.680

It's rather trivial to reduce the size of the program (by removing nesting levels of List). Can it be made so that you can't easily remove the repeating code without affecting program function? – Vi. – 2013-12-29T22:56:13.530

Well of course you can just remove the nesting, but likewise, in the C example you could just remove a few #defines and make the program smaller.

The way I interpreted the requirement that it can't be reduced in size, was that it can't be optimized away. If you're allowed to just modify the source code I don't quite see the point. :) – Christian Palmstierna – 2013-12-30T09:58:04.337

1Although it should be noted that this can probably be optimized away, since the variable a is never used. – Christian Palmstierna – 2013-12-30T10:00:12.847

4

COBOL

   ID DIVISION. 
   PROGRAM-ID. BLOAT. 
   ENVIRONMENT DIVISION. 
   DATA DIVISION. 
   WORKING-STORAGE SECTION. 
   01  THE-TEST-STRINGS. 
       05  FILLER OCCURS 11584 TIMES. 
           10  TEST-STRING          PIC X(11584). 
   LOCAL-STORAGE SECTION. 
   01  FIRST-TIME-FLAG              PIC X VALUE "Y". 
   01  DISP-BEFORE-STRING     COMP  PIC 9(8). 
   01  LOOP-COUNTER           COMP  PIC 9(8). 
   01  START-STRING. 
       05  FILLER OCCURS 0 TO 11584 TIMES 
           DEPENDING ON DISP-BEFORE-STRING. 
           10  FILLER               PIC X. 
       05  THE-SUBSTRING            PIC X(12). 
   01  INITIAL-STRING               PIC X(12) 
                                     VALUE "HELLO WORLD!".
   LINKAGE SECTION. 
   01  STRING-PARAMETER             PIC X(11584). 
   01  THE-RESULT                   PIC X. 
   PROCEDURE DIVISION USING 
                                    STRING-PARAMETER 
                                    THE-RESULT 
                                    . 

       IF FIRST-TIME-FLAG = "Y" 
           PERFORM                  SET-UP-STRINGS 
       END-IF 
       PERFORM 
         VARYING                    LOOP-COUNTER 
         FROM                       1 
           BY                       1 
         UNTIL                      LOOP-COUNTER 
           GREATER THAN 11584 
         OR STRING-PARAMETER 
             EQUAL TO               TEST-STRING 
                                        ( LOOP-COUNTER ) 
       END-PERFORM 
       IF STRING-PARAMETER 
         EQUAL TO TEST-STRING ( LOOP-COUNTER ) 
           MOVE "Y"                TO THE-RESULT 
       ELSE 
           MOVE "N"                TO THE-RESULT 
       END-IF 
       GOBACK 
       . 
   SET-UP-STRINGS. 
       PERFORM 
         VARYING                    LOOP-COUNTER 
         FROM                       0 
           BY                       1 
         UNTIL                      LOOP-COUNTER 
           EQUAL TO 11584 
           MOVE 11584               TO DISP-BEFORE-STRING 
           MOVE SPACE               TO START-STRING 
           MOVE LOOP-COUNTER        TO DISP-BEFORE-STRING 
           MOVE INITIAL-STRING      TO THE-SUBSTRING 
           MOVE START-STRING        TO TEST-STRING 
                                        ( LOOP-COUNTER + 1 )
       END-PERFORM 
       MOVE "N"                     TO FIRST-TIME-FLAG 
       . 

A little knowledge can be a dangerous thing.

It can be faster to do one large compare than a lot of small compares; IBM's Enterprise COBOL (up to Version 4.2) can have a maximum WORKING-STORAGE of 128MB (Version 5.0 can have 2GB); LOCAL-STORAGE offers a further 128MB if you need more space.

The task is to confirm that a 11584-byte piece of storage has the value "HELLO WORLD!" somewhere, and the rest is space.

The, fictitious, programmer decides to write a sub-program for this (just in case it is needed elsewhere), and to include their high-performance technique (bonus).

The programmer calculates that 11584 * 11584 is 128MB, so uses WORKING-STORAGE for a huge table, and LOCAL-STORAGE for everything else that is needed.

The programmer codes it up, and smiles knowingly to themselves when the compile is clean. They were right about the 128MB.

Tests the code. It works. Possibly a little slow, but there's a heavy load on machine. Smiles again, thinking how slow it would be if coded without their level of expert knowledge.

The WORKING-STORAGE comes in at 134,189,056 bytes, and there's a good few bytes of other stuff as well. Should be large enough.

The reality is that doing a long compare instead of a short compare, as implemented here, is a very slow way to do it.

Even slower, the LOCAL-STORAGE, which is initialised by run-time routines every time a sub-program is called, causes the entire 128MB to be set up for each CALL.

The programmer was just plain wrong about the size of the table, there is enough room without using LOCAL-STORAGE. Long compares can beat short compares, but only when the actual number of compares is reduced.

I considered swapping the LOCAL-STORAGE and WORKING-STORAGE around, it is just far less likely someone would code it that way round, so I didn't. Putting a VALUE SPACE on the table (if it had been in LOCAL-STORAGE) would have initilised the table twice on each CALL, so even slower.

The Bloat can't be removed, without rewriting the program. Most of the code is bad, though there is one useful technique.

This is not a real-life example, but I can imagine someone doing it, if that someone is clever enough :-)

Compiling is no problem at all. Running it with every possibility quickly proves to be not worth attempting.

Of course, there is a plain old Bug as well. A very common one in "searching" tasks.

Bill Woodger

Posted 2013-12-18T20:35:33.590

Reputation: 1 391

0

PowerBASIC

#BLOAT(104857600)
FUNCTION PBMAIN
  PRINT "Hello World"
  BEEP
END FUNCTION

Juan Sebastian Lozano

Posted 2013-12-18T20:35:33.590

Reputation: 431

It's comparable to the C example in the question. – Vi. – 2013-12-29T22:57:20.707

0

Scala

import scala.{specialized=>s}
import scala.Specializable.{Everything=>E}
class Printer[@s(E) A, @s(E) B, @s(E) C, @s(E) D, @s(E) E, @s(E) F, @s(E) G, @s(E) H]{
    def print(a:A,b:B,c:C)=println(s"$a, $b, $c")
}

object Main extends App{ 
    (new Printer[Int,Int,Int,Int,Int,Int,Int,Int]).print(1,2,3)
}

The specialized annotation creates a new class for each type in order to prevent boxing when the types eventually all get turned into objects. It will create 10^8 ((Everything consists of 10 types)^(8 type parameters on the class)) class files, each 300-500 bytes, if it doesn't crash first.


This could be explained by saying that performance is important, especially if the class actually did more than have a method to print. Using generic specialized methods instead of putting it all in the declaration would also make it harder to notice

user60561

Posted 2013-12-18T20:35:33.590

Reputation: 101

Which scala version do I need to build this? 2.9.2+dfsg-1 does not like s"whatever" and don't know about scala.Specializable. – Vi. – 2013-12-29T23:00:06.873

Scala 2.10 includes the interpolated strings with s"", but you can remove that string without any effect on size. Scala 2.8 has the specialization feature, so if you remove the interpolated string, everything should work fine. – user60561 – 2013-12-30T15:21:32.307

-2

Javascript

function bigenough(){
        var kbytes = $('html').html().length;
        return (kbytes>1024*100);
}
while(!bigenough()){
$('html').append('<p>WASSUP</p>');}

Run this code in Browser Console on this page and when complete, save the page. it should result in a file size greater than 100 MB. Still testing. Will post actual size once done.

update-
the saved page is the result executable. The v8 engine of chrome is the compiler. And the code I posted is the program. i admit that it does take quiet a long to compile. :D

rahulroy9202

Posted 2013-12-18T20:35:33.590

Reputation: 315

1Doesn't work as required. The task is to create a executable file that is way too big, not one that consumes way too much memory when run. Also, overuse of jQuery. – John Dvorak – 2014-02-17T12:04:45.527

@JanDvorak it will create a HTML File with size greater than 100MB. Also the Question doesnot specify any JQuery usage limitations. The program is still executing on my Chrome and the page is consumning 300mb of memory as reported by the Chrome Task Manager. – rahulroy9202 – 2014-02-17T12:19:00.687

It won't. Every append you do happens purely in memory. It will only create an 100MB HTML file if the user triggers a save operation. Which he might not be able to do, nor want to. Also, even if you call this process of appending "compilation" and you manage to store the resulting HTML as a file, I don't think you're allowed to write your own compiler. – John Dvorak – 2014-02-17T12:24:03.967

@JanDvorak I have pointed out in the answer that the page must be saved. here, the page is the result executable. The v8 engine of chrome is the compiler. And the code I posted is the program. – rahulroy9202 – 2014-02-17T12:27:13.433

2V8 is a compiler that produces a tiny "executable" (which never gets it to the hard drive) and executes it, which then proceeds on to generate a huge "source file" (in a language that isn't even compiled, nor a programming language). If you call the result of your script an executable (nope...) then we must call your script the compiler, not V8. It's not called compilation if your script gets executed in the process (macros kinda blur that line, but this isn't a macro) – John Dvorak – 2014-02-17T12:30:17.613