CJam, 36 35 34 32 30 bytes
riP*30/_60/_C/]2m*::-:mc:mC$3=
The output is in radians. I've verified the solutions for all 86400 possible inputs.
Try it online in the CJam interpreter.
Idea
Since 2π radians is a full lap, each minute/second interval on the clock is 2π/60 = π/30 radians wide.
Thus, dividing the number of seconds by π/30 yields the position of the second hand.
The minute hand moves at one sixtieth of the pace of the second hand, so dividing the result from above by 60 yields the position of the minute hand.
Likewise, dividing the last result by 12 yields the position of the hour hand.
Note that our three quotient from above are not necessarily in the range [0,2π).
By calculating all nine possible differences of the hands' angles, we obtain three 0's (angular distance between a hand and itself) and the six distances between the different hands.
If the closest hands are on a half that does not include 12, one of the differences from above will be the desired output (mod 2π).
However, at 01:55:30 (for example), the hour hand is at an angle of 1.008 rad (57.75 deg) and the minute hand at an angle of 5.812 rad (333.00 deg) from 12, giving a difference of 4.804 rad (275.25 deg). By subtracting this result from a full lap, we obtain the angle measured "in the other direction", which equals 1.479 rad (84.75 rad).
Now, rather than mapping each angle θ in [0,2π) and conditionally subtracting the result from π, we can simply calculate arccos(cos(θ)), since cos is both periodic and even, and arccos always yields a value in [0,π).
Skipping over the three smallest results (all zero), the fourth smallest will be the desired output.
Code
ri e# Read an integer from STDIN.
P*30/ e# Multiply by π and divide by 30.
_60/ e# Divide a copy by 60.
_C/ e# Divide a copy by 12.
]2m* e# Push the array of all pairs of quotients.
::- e# Replace each pair by its difference.
:mc e# Apply cosine to each difference.
:mC e# Apply arccosine to each cosine.
$3= e# Sort and select the fourth smallest element.
Alternate version (34 bytes)
rd6*_60/_C/]360f%2m*::m360X$f-+$6=
The output is in degrees and no trigonometric functions are used.
Try it online in the CJam interpreter.
Related challenge (minute and hour hands only, in degrees) – Sp3000 – 2015-07-29T10:43:28.343
1You should probably specificy that there is a second hand on the clock. – isaacg – 2015-07-29T11:35:24.393
Can you add some test cases? – Beta Decay – 2015-07-29T12:54:42.497
@BetaDecay, I've shown some in my answer. They focus on hands pointing near 1, 6 and 11, so the 1 and 11 are closest in each case. – Hand-E-Food – 2015-07-29T13:03:18.463
1On some clocks, the minute hand jumps to the next minute when the second hand reaches the top. On others, it moves continuously. I figure this is a clock where it moves continuously? Also, while it's clear once you read carefully, I initially found "second hand" ambiguous, because most clocks have at least two hands anyway, so adding the "second hand" really adds a third hand. – Reto Koradi – 2015-07-29T13:50:42.700
@RetoKoradi But then again, no one refers to the second hand as the third hand and in turn would confuse more people if changed – Beta Decay – 2015-07-29T14:00:05.757
I have updated the question to address these points. – toto – 2015-07-29T14:02:41.720
1@BetaDecay Certainly. I might have said something like: "The clocks has three hands: hours, minutes, and seconds." – Reto Koradi – 2015-07-29T14:03:09.767