Stitch a picture

45

9

NOTE: This question is currently in the SANDBOX. Please bear this in mind before adding new answers.


I have a large collection of fine art. My toddler is learning to use scissors and glue; lately she has started playing with my art collection. Fortunately she is really quite good with the scissors and cuts things up into perfect squares. She then uses glue to randomly tile the cut-up squares back into a new piece of art. For example, she reinterpreted my Mona Lisa (which wikipedia kindly hosts for me) as follows:

enter image description here

The following python script simulates what my toddler has done:

#!/usr/bin/python

import sys
import random
from PIL import Image

origname = sys.argv[1]

im = Image.open(origname)
width, height = im.size
width = (int(width + 99) / 100) * 100
height = (int(height + 99) / 100) * 100

im = im.crop((0, 0, width, height))

im2 = Image.new("RGB", (width, height), "black")

blocks = []
for x in range(width / 100):
    for y in range(height / 100):
        blocks.append(im.crop((x * 100, y * 100, (x + 1) * 100, (y + 1) * 100)))

random.shuffle(blocks)

for x in range(width / 100):
    for y in range(height / 100):
        im2.paste(blocks.pop().rotate(90 * random.randint(0,3)), (x * 100, y * 100))

im2.save("shuf" + origname)

Please excuse python skills - I'm still learning, but was happy to see how quick and easy it was to write this script. Polite code-reviews will be graciously accepted ;-)

It does the following:

  • loads the image whose file name was given as a command-line parameter
  • pads that image with black pixels such that the width and height are exactly divisible by 100
  • divides the image into 100x100 pixel blocks
  • randomly shuffles the blocks
  • randomly rotates the blocks (by multiples of 90 degrees)
  • reassembles the randomly arranged blocks back into a new image with the same size attributes as the (padded) original
  • saves the new image using the original filename prefixed with shuf

Your task is to write a program that takes the output of this script, analyses the edges of each 100x100 block and reassembles them back to the original picture.

Input:

  • an image filename. This may be passed at the commandline, via STDIN or even hard-coded, whichever is most convenient.

Output:

  • either output the a singular rearranged image to a differnent file, or display the singular rearranged output to the screen.

Input and output specs are intended to be lenient here, as filename-parsing is a non-goal of this question.

Other rules

  • The program must be able to correctly reassemble any random arrangement of the wikipedia Mona Lisa by the python script. Hard-coding of block transformations of the example image above is strictly not allowed.

  • Because all 100x100 pixel blocks are randomly rotated, the output image may also be randomly rotated by a multiple of 90 degrees.

  • It is understood that for some degenerate cases (e.g. chequerboard of 100x100) blocks it is impossible to correctly rearrange the image. In those cases it is acceptable to produce incorrect/undefined output. However I think in general for photo images and almost all "famous" artworks, reassembly should be possible. Works by Mark Rothko are a possible exception.

  • Common image manipulation libraries may be used (e.g. Python's PIL), but any APIs expressly designed for this purpose are strictly banned.

  • Standard “loopholes” which are no longer funny


Now my toddler got a hold of The Scream. This is the result that needs to be fixed:

enter image description here

Digital Trauma

Posted 2014-10-04T21:06:51.267

Reputation: 64 644

Question was closed 2018-09-17T01:00:11.640

2Your toddler is very destructive. :P – Rɪᴋᴇʀ – 2016-01-05T20:02:26.533

Can we assume that the images are RGBA? – HyperNeutrino – 2017-06-20T16:22:04.433

Actually, that doesn't matter. Never mind. – HyperNeutrino – 2017-06-20T16:22:19.017

1This is just edge-matching, right...? Or is there a component I missed? – Magic Octopus Urn – 2017-10-31T20:27:58.990

could we input/output the image through stdin/stdout? (like ./solution1 < input.jpeg > output.jpeg) And what about the image formats? – Johannes Kuhn – 2014-10-04T21:25:01.677

@JohannesKuhn Yes, thats fine. Any common image format is fine. – Digital Trauma – 2014-10-04T21:27:10.543

1So can I just brute force this? – Beta Decay – 2014-10-05T06:08:45.777

Also how is the program supplied the original image? – Beta Decay – 2014-10-05T06:14:43.493

1@BetaDecay as the task is to obtain the original image, I don't think the original image is supplied – edc65 – 2014-10-05T08:54:27.737

@edc65 How on earth do you know that the image is as it should be then? – Beta Decay – 2014-10-05T08:55:31.813

1I think its possible to reconstruct the image up to rotation, if the image is not a degenerate case – Cameron – 2014-10-05T18:51:10.340

12You can improve your Python code by removing all spaces, giving variables single-letter names, using exec with string replacement on the repetitive bits, and replacing (x + 1) * 100 with -~x*100. :-P – xnor – 2014-10-05T19:50:40.073

1@xnor Ha ha funny ;-) – Digital Trauma – 2014-10-05T20:11:06.337

@Cameron Yes, this is a good point - I will edit in a note about rotation of final image – Digital Trauma – 2014-10-05T20:11:52.177

@BetaDecay wrt brute forcing. You may only output a single fixed image. Multiple output images are not acceptable. – Digital Trauma – 2014-10-05T20:57:43.843

1@BetaDecay The point of the challenge is for your program to analyze the edges of blocks to find similarities and attempt to fit similar edges together to reassemble the original picture. Hence the [tag:image-processing] tag. – Digital Trauma – 2014-10-05T20:59:52.947

It occurs to me that http://codegolf.stackexchange.com/questions/36967/put-together-cut-strips-of-paper-to-get-original-message is possibly related. IMO image vs text and 1d vs 2d matching alone are enough to set this one apart. Of course its up to the community to decide if this is a duplicate or not.

– Digital Trauma – 2014-10-05T21:11:54.067

1

You can do easily with brute force. Try every block arrangement (including rotations) and keep the one that minimize the total difference at borders. With monna lisa, 77!477 possibilites (good that 7 and 11 are primes). May I cross-post my (eventual) answer to http://codegolf.stackexchange.com/q/36747/21348?

– edc65 – 2014-10-06T11:44:04.120

1@DigitalTrauma But actually your Python code looks good. I don't have any comments. – xnor – 2014-10-06T21:39:24.493

2

This task is impossible. Sure you can try and minimize MSE on the edges and push it through greedy or the Hungarian algorithm, but its still fundamentally just a 'best guess' for the programmer's definition of 'best'. For any algorithm, you can craft input that defeats it. And images like this one http://www.robertnyman.com/images/0611/lightroom/grid-view-panels-hidden.png will, with 100x100 grids, defeat all attempts.

– Will – 2014-10-07T11:52:43.913

2Furthermore, my naive approach works well enough ... except the MSE of the black padding pixel edges is 0 with each other! It seems wrong to special-case black edges, and black edges can legitimately occur in-image. Alas I think the contest is, fundamentally, flawed. – Will – 2014-10-07T12:18:08.753

@Will You may assume that the unpadded original does not have black edges. – Digital Trauma – 2014-10-07T16:13:57.277

1Well I can't get MSE to work on these edges. Because the edges are not exact matches, and because I'm not trying to model the JPG DCT, I keep getting mismatches that are actually lower edge MSE than the real image. Oh well. – Will – 2014-10-08T08:53:02.300

@Will Perhaps the MSE is not the better way ... – Dr. belisarius – 2014-10-08T11:40:35.413

10Maybe you should turn this into a code-challenge and make the score the number of correctly aligned edges (code size being the tie breaker). And it would be huge help if you used a lossless image format. – Martin Ender – 2014-10-15T10:37:40.387

3This really sounds intresting/challenging but I think it might be too challenging. I tried for the mona lisa various distance metrics and calculated MSE for a lot of rotated/swapped tiles of the original mona lisa, and was always able to find configurations that had a lower global MSE than the original image. I am going to try some totally different approach now, but I think the idea is good to transform this into some "best out of 10 images" alignment contest like others suggested. – PlasmaHH – 2014-10-22T10:26:59.553

6

This exact problem is an active area of research, see eg http://www.cs.umanitoba.ca/~ywang/papers/crv13_jigsaw.pdf and https://github.com/typeinference/jigsaw-puzzle which has an implementation of that. State of the art is just 96% correct; the paper also explains how it calculates that score (hint hint)

– bazzargh – 2014-10-24T02:48:51.710

No answers