A skeleton and multiple entries
These are a few of my creations. They all use 'goodvsevil.py', described further below. They are otherwise all quite dumb.
getrichquick.py
This one always copies the decision that made the most points last round (random in case of a tie). It also contains a hint on how you might use 'goodvsevildebug.py' while developing.
As with any get-rich-quick scheme, this method doesn't generally work.
#!/usr/bin/python3
import sys
from goodvsevil import *
# from goodvsevildebug import *
setSaveFile('./getrichquick.pickled')
def decider(stats):
points = { False:0, True:0 }
for i in range(stats.players):
points[stats.playerLines[i][-1]] += stats.scores[i][-1]
t = points[True]
f = points[False]
if t == f:
return PetyrBaelish()
else:
return f < t
run(sys.argv, decider)
# printStats(randomRounds(7, 70, decider))
# printStats(hostRounds(70, decider, [angel, demon] + 4*[PetyrBaelish,]))
findstreaks.py
This one looks for the player(s) with the longest lasting streak of going with or against majority, and votes for minority based on that player's last vote (average if several players have the same length of streak, random if 50:50 among them).
Its fallacy is the assumption that players are always a) going to continue their streak b) by taking the same side again this turn.
#!/usr/bin/python3
import sys
from goodvsevil import *
identity = lambda x: x
negation = lambda x: not x
def decider(stats):
count = 0
minvotes = []
for i in range(stats.players):
streak = stats.winningStreaks[i][-1]
filtr = negation if streak.good else identity # negate if majority streak
if count < streak.length:
count = streak.length
minvotes = [filtr(stats.playerLines[i][-1])]
elif count == streak.length:
minvotes.append(filtr(stats.playerLines[i][-1]))
avg = 0.5
if minvotes:
avg = average(minvotes)
if avg > 0.5:
return True
elif avg < 0.5:
return False
else:
return PetyrBaelish()
setSaveFile('./findstreaks.pickled')
run(sys.argv, decider)
trendalizer.py
This one likes to be 65% certain to go with the trend. It averages windows of previous 10, 25, 50, 100 and 200 rounds, then averages these results to determine THE trend.
It prefers the trend from majority decisions over that from all votes. If neither trend is sufficiently clear, it chooses sides randomly.
#!/usr/bin/python3
import sys
from goodvsevil import *
setSaveFile('./trendalizer.pickled')
def decider(stats):
maj = {}
tot = {}
for x in (10, 25, 50, 100, 200):
maj[x] = average(stats.majorityLine, x)
tot[x] = []
for i in range(stats.players):
tot[x].append(average(stats.playerLines[i], x))
tot[x] = average(tot[x])
maj = average(maj.values())
tot = average(tot.values())
if 0.15 < abs(maj - 0.5):
return maj > 0.5
if 0.15 < abs(tot - 0.5):
return tot > 0.5
return PetyrBaelish()
run(sys.argv, decider)
coward.py
This one is normally random, but turns to the forces of good when it feels the presence of evil. Which has nothing to do with what's going on in the world, but is based solely upon the position in which coward finds itself.
According to coward's belief, prime numbers are evil, as are the numbers 6 and, therefore, 9. And 0, obviously. He doesn't believe in new age stuff like even prime numbers or prime numbers having several digits. Any numbers containing an evil number are evil.
#!/usr/bin/python3
import sys
from goodvsevil import *
setSaveFile('./coward.pickled')
evil = [0, 3, 5, 6, 7, 9]
def decider(stats):
if (None == stats.custom) and (stats.myIndex >= 0):
if stats.myIndex in evil:
stats.custom = True
else:
indexstr = str(stats.myIndex)
for bad in evil:
if str(bad) in indexstr:
stats.custom = True
return angel()
stats.custom = False
if stats.custom:
return angel()
return PetyrBaelish()
run(sys.argv, decider)
Skeleton for Python 3
Here's a skeleton for everybody who wishes to build upon. By persisting the Stats, it only ever needs to parse the last round.
It consists of two files:
- goodvsevil.py: provides infrastructure, saving/loading stats to/from file and adding the latest results each round. Six methods are intended for external use:
setSaveFile(path)
:
sets the path where the stats are saved. Must be called before run().
run(argv, decider)
:
parses the latest result, adds it to the stats, calls the decider function, correctly prints the decision to stdout, and saves the stats.
argv
will usually be sys.argv
decider
must be a function that takes a Stats object as argument and returns True for 'good', False for 'evil'.
average(list, count=0)
:
returns the average of the last count
elements in list
. If count
is falsy, the entire list
is used.
angel
, demon
, PetyrBaelish
:
are decider functions with the behavior their name indicates.
- goodvsevildebug.py: may be useful while developing. It can simulate rounds and print stats. It's not to be used in submissions, because it prints stuff to stdout.
fileStats(filename)
takes the path to your save file and prints the stats it contains.
Edit:
- non-breaking change to run() in goodvsevil.py:
run has a new parameter suppress
, to support silent operation in goodvsevildebug.py
- new function getSaveFile() in goodvsevil.py:
returns the path to the savefile. Needed in goodvsevildebug.py.
- non-breaking changes to fileStats() and printStats() in goodvsevildebug.py:
they both have a new parameter onlyMe
to enable short listing. fileStats() now also tries to append '.pickled' to the filename if it doesn't otherwise exist.
- non-breaking change to hostRounds() in goodvsevildebug.py:
there are two new parameters, opponentnames
and myname
. They are used when printing. opponentnames
also determines the filenames for the opponents saved stats. opponentnames
are matched to opponentdeciders
by position. deciders without a name get named after the position where they are placed.
- hostRounds() now announces all players positions (converting them to 1-based), shows progress and timing information every 10%, and displays the final ranking (along with average and max times) when done.
- hostRounds() doesn't simply call the deciders anymore, but passes them to run(). This means that all deciders have a separate automatically updated save file each, and no work or data is shared between them.
Out of curiosity, I ran 1000 rounds with 29 players (1 Homer-clone with 200ms delay, 1 angel, 1 demon, 1 coward, 1 getrichquick, 1 findstreaks, 1 trendalizer, and 22 PetyrBaelishes, each on top of the goodvsevil.py infrastructure).
Average time per round was 72ms for most players, max time 220ms. Only Homer was slower, 280ms on average and 420ms max. Average time in the last 10% (across all 29) was 151ms per player/round.
That includes loading stats from file, parsing last round, updating stats, making a decision, saving stats back to file, and negligible hostRounds() overhead. It does not include the time to start python and load the script. It also doesn't account for the fact that run() prints the decision to stdout before it saves the stats to help meet timing requirements.
Here is the code I used for the experiment. Feel free to use it as a starting point if you'd like to try out my 'framework'. NOTE: if you don't like to wait half an hour or more, lower the number of rounds and/or opponents.
#!/usr/bin/python3
import sys
from goodvsevil import *
def decider(stats):
'implement my decision logic here'
import threading #
threading._sleep(0.2) # please, don't do this. It's just here to prove that time measurement works.
return True
setSaveFile('./testing.pickled')
run(sys.argv, decider)
#### everything below is to be removed before submitting. ####
rounds = 1000
from goodvsevildebug import *
class DevNull:
def write(self, *args, **kwargs):
pass
savefile = getSaveFile() # store the save file name
stdout = sys.stdout
sys.stdout = DevNull() # silence during import
import coward, streakfinder, getrichquick, trendalizer # these print stuff and change the save file
sys.stdout = stdout # restore stdout
setSaveFile(savefile) # restore save file name
opponents = [angel, demon, coward.decider, streakfinder.decider, getrichquick.decider, trendalizer.decider] + 22* [PetyrBaelish,]
oppnames = ['angel', 'demon', 'coward','streakfinder','getrichquick', 'trendalizer', 'PetyrBaelish1', 'PetyrBaelish2']
stats = hostRounds(rounds, decider, opponents, oppnames, '* Homer Clone *')
### NOTE that the stats object/files do not contain the decisions from the very last round.
### printStats() and fileStats() will therefore report lower scores than hostRounds() did.
#
#printStats(stats) # print full run info: majority decisions, player decisions and scores
#
#oppnames.remove('angel') # we know their decisions - no need to display
#oppnames.remove('demon')
#for x in oppnames:
# print('\n%s:' % (x,))
# fileStats(x, onlyMe=True) # print short info about each opponent in oppnames: position, decisions and score
goodvsevil.py:
#!/usr/bin/python3
import pickle, random
def run(argv, decider, suppress=False):
'run an invocation. Pass argv = sys.argv, decider = function(Stats) returning boolean.'
stats = updateStats(argv)
decision = decider(stats)
if not suppress:
print('good' if decision else 'evil')
if stats.myIndex < 0:
stats._myDecisions.append(decision)
with open(savefilename, "wb") as file:
pickle.dump(stats, file=file)
return (decision, stats)
def angel(stats=None):
return True
def demon(stats=None):
return False
def PetyrBaelish(stats=None):
return random.randint(0,1)
class Streak:
def __init__(self, good=False, length=0):
self.good = good # meaning depends on context
self.length = length # streak length in number of rounds
class Stats:
def __init__(self):
self.players = 0 # number of players
self.majorityLine = [] # list of majority decisions of { True:'good', False:'evil' }
self.playerLines = [] # list of decisions for each player of { True:'good', False:'evil' }
self.scores = [] # list of scores for each player (3 for majority, streak.length-1 for minority)
self.totalScores = [] # total score for each player
self.decisionStreaks = [] # playerLines as lists of streaks
self.winningStreaks = [] # list of streaks for each player hitting the same group of { True:majority, False:minority }
self.majorityLineStreaks = [Streak()] # majorityLine as a list of streaks
self._myDecisions = [] # used in the first few rounds until I know who I am
self.myIndex = -1 # which of the playerLines is Me?
self.totalGood = 0 # total number of player decisions for 'good'
self.totalEvil = 0 # total number of player decisions for 'evil'
self.roundsGood = 0 # total number of rounds won for 'good'
self.roundsEvil = 0 # total number of rounds won for 'evil'
self.custom = None # add some custom state object if you wish
savefilename = ''
def setSaveFile(path):
global savefilename
savefilename = path
def getSaveFile():
return savefilename
def majorityVote(sround):
return sround.count('1') > sround.count('0')
def isGood(char):
return char == '1'
def average(listnums, lastcount=0):
if lastcount:
listnums = listnums[-lastcount:]
if listnums:
return sum(listnums) / len(listnums)
return 0
def updateStats(argv):
if not savefilename:
raise Exception('No save file set. Please use setSaveFile(path) right after "import goodvsevil".')
decision = 0
lastround = ''
stats = None
firstround = (len(argv) < 2)
if firstround:
stats = Stats()
else:
with open(savefilename, "rb") as file:
stats = pickle.load(file)
maxi = len(stats.playerLines) - 1
lastround = argv[1].split(',')[-1]
# record majority:
majvote = majorityVote(lastround)
if majvote:
stats.roundsGood += 1
else:
stats.roundsEvil += 1
if stats.majorityLine and (stats.majorityLine[-1] == majvote):
stats.majorityLineStreaks[-1].length += 1
else:
if stats.majorityLineStreaks[-1].length:
stats.majorityLineStreaks.append(Streak())
stats.majorityLineStreaks[-1].length = 1
stats.majorityLineStreaks[-1].good = majvote
stats.majorityLine.append(majvote)
# record player histories:
for i in range(len(lastround)):
good = isGood(lastround[i])
if i > maxi:
stats.playerLines.append([])
stats.scores.append([])
stats.totalScores.append(0)
stats.decisionStreaks.append([Streak()])
stats.winningStreaks.append([Streak()])
guessedright = (majvote == good)
if stats.playerLines[i]:
if good != stats.playerLines[i][-1]:
stats.decisionStreaks[i].append(Streak())
if guessedright != stats.winningStreaks[i][-1].good:
stats.winningStreaks[i].append(Streak())
stats.playerLines[i].append(good)
stats.decisionStreaks[i][-1].length += 1
stats.winningStreaks[i][-1].length += 1
stats.decisionStreaks[i][-1].good = good
stats.winningStreaks[i][-1].good = guessedright
stats.scores[i].append(3 if guessedright else (stats.winningStreaks[i][-1].length - 1))
stats.totalScores[i] += stats.scores[i][-1]
if good:
stats.totalGood += 1
else:
stats.totalEvil += 1
if stats.myIndex < 0:
# try to find myself:
candidates = []
for i in range(len(stats.playerLines)):
fits = True
for k in range(len(stats._myDecisions)):
if stats._myDecisions[k] != stats.playerLines[i][k]:
fits = False
break
if fits:
candidates.append(i)
if len(candidates) == 1:
stats.myIndex = candidates[0]
if not stats.players:
stats.players = len(stats.playerLines)
return stats
goodvsevildebug.py:
#!/usr/bin/python3
import os, time
from goodvsevil import *
def randomRounds(players, rounds, decider=PetyrBaelish):
'host a number of rounds against a number of PetyrBaelishes'
return hostRounds(rounds, decider, (players-1) * [PetyrBaelish,])
def hostRounds(rounds, decider=PetyrBaelish, opponentdeciders=[], opponentnames=[], myname='* me *'):
'host a number of rounds against a list of opponents, part of which may have names'
opponentdeciders = opponentdeciders[:] # copy lists because we'll alter them
opponentnames = opponentnames[:]
increment = rounds / 10 # how many rounds are 10%?
filename = getSaveFile()
s = ''
temps = ''
stats = None
deciders = []
names = []
players = len(opponentdeciders)
if (players % 2):
opponentdeciders.append(PetyrBaelish)
players += 1
mypos = random.randint(0,players)
print('my position: %d of %d' % (mypos+1, players+1))
print()
players += 1
found = players * [False,]
decisions = players * [False,]
scores = players * [0,]
streaks = players * [None,]
times = players * [None,]
# randomly arrange players:
print('Positioning:\n------------')
for i in range(players):
if i == mypos:
deciders.append(decider)
names.append(myname)
else:
x = random.randint(0, len(opponentdeciders)-1)
randchoice = opponentdeciders[x]
opponentdeciders.remove(randchoice)
deciders.append(randchoice)
if opponentnames and (len(opponentnames) > x):
name = opponentnames[x]
names.append(name)
opponentnames.remove(name)
else:
names.append(str(i+1))
print('%02d: %s' % (i+1, names[i]))
print()
print('Run:\n----')
t0 = time.time()
for r in range(rounds):
temps = ''
tstart = time.time()
for i in range(players):
# set a different save file for each player:
if i == mypos:
setSaveFile(filename)
else:
setSaveFile('./%s.pickled' % (names[i],))
# call the player, with or without argv[1]:
if r == 0:
(decision, st) = run([None], deciders[i], suppress=True)
times[i] = []
else:
(decision, st) = run([None, s], deciders[i], suppress=True)
# remember everything we need to score the final round:
decisions[i] = decision
if r > 0:
scores[i] = st.totalScores[i]
streaks[i] = st.winningStreaks[i][-1]
# prepare next round's argv[1]:
temps += '1' if decision else '0'
# check if player has determined its position this round:
if (not found[i]) and (st.myIndex >= 0):
found[i] = True
if i == mypos:
print('*** Found myself in position %d after round %d.' % (st.myIndex+1, r))
else:
print(" '%s' found itself in position %d after round %d." % (names[i], st.myIndex+1, r))
if i == mypos:
stats = st
# measure time:
tend = time.time()
times[i].append(tend - tstart)
tstart = tend
# finish preparation of next argv[1]:
if r != 0:
s += ','
s += temps
# Inform about progress in steps of ~10%:
if 1 > ((r+1) % increment): # 1> instead of 0== because increment is float
print('%d%% - t0 + %f' % (10*(r+1)/increment, time.time()-t0))
print()
# score the last round:
majority = (players/2) < sum(decisions)
maxtimes = players * [0,]
for i in range(players):
if decisions[i] == majority:
scores[i] += 3
elif not streaks[i].good: # minority streak
scores[i] += streaks[i].length
maxtimes[i] = max(times[i])
times[i] = average(times[i])
scores = list(zip(scores, names, times, maxtimes))
scores.sort(key = lambda x: x[1]) # sort by name, so that equal scores will appear in lexical ordering
scores.sort(key = lambda x: x[0], reverse=True) # sort by score
print('Ranking:\n--------')
for x in scores:
print('%d - %s (avg time %f, max %f)' % x)
print()
print('hosted %d rounds with %d players in %f seconds.' % (rounds, players, time.time()-t0))
print()
return stats
def fileStats(filename, onlyMe = False):
'load a Stats object from file and print relevant info.'
if not os.path.exists(filename):
if os.path.exists(filename + '.pickled'):
filename += '.pickled'
with open(filename, 'rb') as file:
printStats(pickle.load(file), onlyMe)
def printStats(stats, onlyMe=False):
"print relevant info from the passed Stats object. if onlyMe, don't print majority or opponents."
playerpaths = ''
playerguessing = ''
players = len(stats.playerLines)
majoritypath = ''
if onlyMe:
score = 0
if stats.myIndex >= 0: # I found myself at some point, use playerLines[myIndex]
print('Position: %d / %d' % (stats.myIndex+1, stats.players))
for i in range(len(stats.playerLines[stats.myIndex])):
dec = stats.playerLines[stats.myIndex][i]
majoritypath += '1' if dec else '0'
playerguessing += '1' if dec == stats.majorityLine[i] else '0'
score = stats.totalScores[stats.myIndex]
else: # I never found myself, use _myDecisions
print('Position: unknown')
good = False
streak = 0
for i in range(len(stats._myDecisions)-1):
dec = stats._myDecisions[i]
majoritypath += '1' if dec else '0'
if good == (dec == stats.majorityLine[i]):
if good:
score += 3
else:
score += streak
streak += 1
else: # change between minority/majority
good = (dec == stats.majorityLine[i])
streak = 1
if good:
score += 3
playerguessing += '1' if good else '0'
print('- %s\n %s (%d)' % (majoritypath, playerguessing, score))
return
for i in range(players):
decs = '- '
ress = '- '
pscore = 0
for decision in stats.playerLines[i]:
decs += '1' if decision else '0'
for streak in stats.winningStreaks[i]:
ress += streak.length * ('1' if streak.good else '0')
ress += ' (%d)' % stats.totalScores[i]
if i == stats.myIndex:
ress = '* %s *' % ress[2:]
decs = '* %s *' % decs[2:]
playerpaths += decs + '\n'
playerguessing += ress + '\n'
for dec in stats.majorityLine:
majoritypath += '1' if dec else '0'
if stats.myIndex < 0:
majoritypath += '\n\nMy decision line:\n-----------------\n- '
for dec in stats._myDecisions:
majoritypath += '1' if dec else '0'
print('''
Position:
---------
%d / %d
Majority decision line:
-----------------------
- %s
Decision lines:
---------------
%s
Alignment with majority (Score):
--------------------------------
%s''' % (stats.myIndex+1, players, majoritypath, playerpaths, playerguessing[:-1]))
Is there a submission deadline? – Jonathan Pullano – 2014-07-09T15:18:59.853
2Comments purged, as they were obsolete and at request of OP. Please notify me of any comments that need to be undeleted. – Doorknob – 2014-07-16T11:28:29.823
@Doorknob The "When's the official testing?" question should stick around / be incorporated into the main post. – isaacg – 2014-07-17T09:57:11.803
@isaacg I've undeleted those two comments until Rusher can edit them into his post. – Doorknob – 2014-07-17T10:40:10.970
7Woah, OP changes his username. Ok, so when will the result be displayed? – justhalf – 2014-07-18T06:52:58.687
2Bounty refunded and reinstated as the OP couldn't compile all the submissions in time. Minimum time for the new bounty to be awarded is 24 hours, so that's approximately when the new one will be awarded. – Doorknob – 2014-07-19T07:19:06.893
6@Rainbolt This must be one freakin' hell of a job, running this challenge! The reason for this amount of attention is the simplicity of the protocol and the rules, making it accessible while also allowing simple, working entries. TL;DR: Your challenge is too good! :D – tomsmeding – 2014-07-19T18:54:20.987
@Rainbolt I would suggest applying a cut date for the current round of testing to limit it to the ones that are already there so you don't have to do it all over again if something new pops up. – Angelo Fuchs – 2014-07-20T08:15:56.290
@Rainbolt: could you post upper and lower bounds on scores as well as averages? – dgel – 2014-07-21T10:41:51.810
3@dgel I'll post the raw data, upper, lower, averages, and maybe a line chart so we can see who did better as the competition dragged on. – Rainbolt – 2014-07-21T12:03:06.023
6One of the pods ended up with 10 entries that voted the same way every single time. Consequently, two users ended up with perfect or "one round short of perfect" scores of around 450,000. The same entries scored around 1900 in other trials. The average score is close to 2000. Because of the extreme imbalance in results, I decided that a more meaningful number would be a median. I edited the challenge so that after 5 trials, the winner will be the submission with the highest median. If anyone thinks that moving from mean to median is unfair or otherwise a poor choice, please comment. – Rainbolt – 2014-07-21T18:57:16.903
@Rainbolt Another option would be to issue points based where each entry placed in each round (i.e. how many other entries that entry outscored), instead of making absolute scores important. I favor that idea, although I don't suppose it matters very much to me, really. – Brilliand – 2014-07-21T19:34:13.490
1Either median or summing the ranking in each of the 3 trials is fine by my – dgel – 2014-07-21T20:08:22.333
Probably a bit late, but wouldn't it be better to do around 100 rounds for around 100 trials instead of 1000/3? Besides that, I also think the average rank would be a better measure of performance than the average score. – moooeeeep – 2014-07-22T07:06:53.893
The problem with doing just 100 rounds is that some of the entries were designed around the fact that there will be 1000 rounds. In fact, a number of the statistics-based entries vote randomly for the first n rounds (and I think 100 was the popular choice) for enough data to be available for statistical analysis. – jaybz – 2014-07-22T08:28:24.180
1I was going to add this as an edit, but it doesn't fit. Anyway turns out I might have been wrong. I did a quick check and I found only two entries that was specifically designed for more than 100 rounds. That said, 100 is a much smaller sampler size than 1000 and would negatively affect the statistics-based entries severely. – jaybz – 2014-07-22T08:43:00.033
1Thanks for posting the scoreboard! It looks like the first few rounds may be getting cut off (or skipped, or something)--Biographer is supposed to choose Good for the first ten rounds, but turns to evil around round 6, and Linus isn't matching his hardcoded behavior either. – histocrat – 2014-07-22T16:37:45.023
@histocrat I will investigate this later. If I find that the posted results don't match the actual results, I'll take the blame and award another bounty (because what am I doing with all of these imaginary points anyway?). This could just be an error on my part with matching the raw data to the scores. – Rainbolt – 2014-07-22T17:17:12.493
How come my Scientists aren't in there? – tomsmeding – 2014-07-22T18:07:19.603
@tomsmeding I expect that you aren't the only one with this question. Please see the note on the side of the Scoreboard. I can't say for certain why you were excluded. When I have time to review my detailed notes at home, I'll transcribe them to the Scoreboard, and then you'll have your answer. I will accept appeals if you feel like I messed up, because I mess up a lot. Hopefully everyone here understands how large of a project this was, that it's all just a game anyway, and that I'll keep fixing it until I get it right. – Rainbolt – 2014-07-22T18:22:34.460
Ok so what ended up happening was when you separated us into groups, the scores were highly dependant on who we are paired with. As I was fortunate enough to be sided with goody two shoes and opportunists (literally), every round from mine went to good so I lost every round (if I won I would consider switching). I, therefore, got the one perfect score. You noted that this is not really fair (good observation) as mine could not hold weight on a more variable field. You chose to score by median, therefore. I'm not sure that is the best measure, but mine did not diserve to win so thats fine. – kaine – 2014-07-22T19:14:31.193
I thought Angel and Devil would be in each group? Obviously I misunderstood this. – Paŭlo Ebermann – 2014-07-22T19:57:57.083
@anotherguest I chose median because it is the most resistant to being swayed by outliers. A simple average varied from 1800 to 100000. A geometric mean using only the three middle scores ranged from 1700 to 5000. The median ranged from 1700 to 2900. The median has the tightest grouping, which indicates to me that it will change the least as I run more and more trials. Wikipedia will back me up here: [The median] is the most resistant statistic, having a breakdown point of 50%: so long as no more than half the data is contaminated, the median will not give an arbitrarily large result. – Rainbolt – 2014-07-22T20:05:09.080
@PaŭloEbermann Angel and Demon participated in every trial. They were shuffled and separated into pods just like the rest of the submissions. – Rainbolt – 2014-07-22T20:07:48.940
@Rainbolt Rainbot, just to be clear I'm in agreement for the most part for eliminating outliers. I do believe median works alot better when there are alot of samples involved as it helps to distinguish what actually is an outlier. For instance, Whatever did great on two separate ones...are either of them outliers? Nevertheless, you are the boss and the call was good. – kaine – 2014-07-22T21:03:28.000
Just to be clear, the issue with median can be seen from this result. Mercenary did quite well in 3 rounds, rounds 1,2 and 3, and quite poorly in rounds 4 and 5, leading to it having a very high median. I think a statistic that incorporates all of the results, but doesn't scale with any of them, might be better. – isaacg – 2014-07-22T22:57:37.587
@histocrat It turns out that Biographer was calling another entry (python BigMac.py... don't ask why). If Biographer is in the top 3 the next time I post the scoreboard, I'll make it up to you. Linus, as far as I can tell, is behaving just as expected. To everyone else, the scores should be correct. I just messed up in this one particular case. – Rainbolt – 2014-07-23T02:03:18.587
1@tomsmeding and all other participants: I added a disqualifications tab to the scoreboard and summarized my notes. It's likely that I made mistakes at some point. For that reason, I added an extremely simple to follow appeal process. Just visit the Scoreboard and read the rules, then leave a comment here if you have something to say. – Rainbolt – 2014-07-23T03:49:36.660
@Rainbolt I fully believe my stuff timed out, but I didn't expect an exception. Do you have any more information on Scientist's exception? – tomsmeding – 2014-07-23T07:53:14.657
@Rainbolt, my entries (RidgeProfessor and NaiveBayesian) aren't in the scoreboard, and not listed amongst the disqualifications either. Could you tell me what went wrong? – dgel – 2014-07-23T08:11:53.853
@dgel I added a note to one of your Eigen entries. In short, I forgot to add them as "Couldn't compile". – Rainbolt – 2014-07-23T13:07:41.760
1@Rainbolt, do we get a next round? – Emil – 2014-08-11T15:28:05.897
2@Emil Yes. It has been nearly a month so I had better get to it. I have been wanting to run some other challenges, but promised myself I would wait until after I got a new scoreboard posted for this one. I "undisqualified" everyone to give them a second chance. Unfortunately, this means that some people will start failing 7 hours into the rounds, making me start all the way over, so it will be a few days before I can post the new scoreboard. – Rainbolt – 2014-08-12T16:42:20.813
Great. Thanks for the hard work. I'm looking fortward to seeing the results. – Emil – 2014-08-13T06:28:23.200
Is the new scoreboard going to be up any time soon? – overactor – 2014-08-28T11:16:35.693
2@overactor I gave up on running trials. Every output file looks like a bunch of Chinese characters and other nonprintable characters. Previously, this only occurred once every two or three trials. Now, it's messing up every output file. It takes about 7 hours to run a trial, so it is really tedious to watch over (I have to keep checking my graphs to make sure that nothing interferes with my CPU or memory usage). If you want, I can post the source code this afternoon. It has every single submission included. Maybe someone else can get it to work. – Rainbolt – 2014-08-28T13:30:52.353
2I would appreciate it is you would post the source. I likely won't be able to help get a new scoreboard up, but I want to learn how you set it all up and really hope someone else does. This was an interesting puzzle and the results were underwelming. – kaine – 2014-09-03T20:03:10.487