What's eating my entropy? Or what does entropy_avail really show?

Question

Below are graphs with the value of /proc/sys/kernel/random/entropy_avail on a Raspberry Pi. This answer ^{which might not be correct} describes it as:

/proc/sys/kernel/random/entropy_avail simply gives you the number of bits that can currently be read from /dev/random. Attempts to read more than that will block until more entropy becomes available.

The pattern always comes to the same "stable" saw pattern with ~130 bit decrease every single minute. If entropy grows too much, something is "eating" it to return to the 700-800 range. If I reboot the device, the entropy is still being eaten each minute, but in a smaller chunks thus enabling it to grow again to 700-800 range.

How shall I interpret the graphs? What is happening?

My feeling is that if there was just a process using a random number generator, the entropy_avail once thrown out of balance (by using the device hardware) should either grow infinitely or decrease to the level of 200, when /dev/random would stop supplying the values.

Also if any of the monitoring methods (see checks below) influenced the entropy, it should rather decrease entropy every second, rather than let it grow and drop suddenly at one minute intervals.

^{(if I leave the machine idle, the stable "saw" pattern continues for days, I took the screenshots in a shorter period of time)}

---

The graphs

• The machine is idle for a long time:

  

• At around 19:14:45 another machine accessed apt-cacher on the Pi - entropy grew (I guess from network usage). After that at 19:16:30 the drop to "usual levels" was larger than usual (it is also repeatable - if entropy_avail grows too large, it drops faster):

  

• I reboot the machine, the entropy grows until it reaches the "usual" level:

  

  

• Once again it reaches an idle state:

  

• After another reboot the point in time of the entropy decrease changes, but it still occurs every single minute:

  

---

Checks

• I stopped netdata (monitoring program) and checked with watch -n1 cat /proc/sys/kernel/random/entropy_avail. The value of entropy_avail grows to ~800 and drops to ~680 at regular one minute intervals.

• Per advice "trace all processes for access to /dev/random and /dev/urandom" I checked with inotifywait (idea from an answer to a similar question) on Debian VM and there is no access to either /dev/random or /dev/urandom at the moment entropy_avail drops (of course checking manually logs the event).

• I used the entropy-watcher to check the entropy as advised against using watch. The results are still consistent with a steady increase and a sharp drop every single minute:

  833 (-62)
  836 (+3)
  838 (+2)
  840 (+2)
  842 (+2)
  844 (+2)
  846 (+2)
  848 (+2)
  850 (+2)
  852 (+2)
  854 (+2)
  856 (+2)
  858 (+2)
  860 (+2)
  862 (+2)
  864 (+2)
  866 (+2)
  868 (+2)
  871 (+3)
  873 (+2)
  811 (-62)
  

---

Two questions on Unix StackExchange describing the same phenomenon (found later):

• What keeps draining entropy?
• What program(s) is(are) lowering my entropy pool level?

Answer 1

First, the claim that "/proc/sys/kernel/random/entropy_avail simply gives you the number of bits that can currently be read from /dev/random" is false.

The entropy_avail field reads the input_pool.entropy_count, the "output" pool refers to the pool used for urandom (non-blocking pool) and random (blocking pool).

---

As mentioned in this answer, spawning new processes consumes entropy for things like ASLR. The watch program spawns a new process for every invocation, perhaps the monitoring tool does the same (possibly via one of the other monitoring sources that have to invoke an external program to obtain the status?).

For monitoring the entropy pool without draining it you can try the entropy-watcher program (see linked answer).

Watching the entropy-watcher numbers closely it seems that you lose about 64 bits of entropy at intervals. Based on the analysis in the other answer, this seems to be the result of moving entropy to an "output pool" to avoid wasting it. This is observed on Linux v4.6, future implementations may be different.

Based on the source code (drivers/char/random.c in v4.6), I can see that reading the output pools (/dev/{u,}random or get_random_bytes()) invokes extract_entropy{,_user} which calls xfer_secondary_pool and account. The blocking pool has the property limit set (r->limit == 1) which affects both functions:

• For account() it will return no data from the blocking pool if its entropy is too low. For the non-blocking output pool, the remaining entropy will be consumed but data is still returned.
• xfer_secondary_pool() ensures that enough entropy is available in the output pool. If the blocking output pool has insufficient entropy, it will take some from the input pool (when possible).
• xfer_secondary_pool() for the non-blocking output pool behaves specially according to the /proc/sys/kernel/random/urandom_min_reseed_secs parameter. If this value is non-zero, entropy is only taken from the input pool if at least urandom_min_reseed_secs seconds have elapsed since the last transfer. By default this value is set to 60 seconds.

The last point finally explains why you see an entropy drain in the input pool every 60 seconds. If some consumer requests random bytes from the non-blocking output pool (TCP sequence numbers, ASLR, /dev/urandom, getrandom(), ...), then 128 bits will be consumed from the input pool to reseed the non-blocking output pool.