k-independent hashing

In computer science, a family of hash functions is said to be k-independent or k-universal[1] if selecting a function at random from the family guarantees that the hash codes of any designated k keys are independent random variables (see precise mathematical definitions below). Such families allow good average case performance in randomized algorithms or data structures, even if the input data is chosen by an adversary. The trade-offs between the degree of independence and the efficiency of evaluating the hash function are well studied, and many k-independent families have been proposed.

Background

See also: Hash function

The goal of hashing is usually to map keys from some large domain (universe) ${\displaystyle U}$ into a smaller range, such as ${\displaystyle m}$ bins (labelled ${\displaystyle [m]=\{0,\dots ,m-1\}}$). In the analysis of randomized algorithms and data structures, it is often desirable for the hash codes of various keys to "behave randomly". For instance, if the hash code of each key were an independent random choice in ${\displaystyle [m]}$, the number of keys per bin could be analyzed using the Chernoff bound. A deterministic hash function cannot offer any such guarantee in an adversarial setting, as the adversary may choose the keys to be the precisely the preimage of a bin. Furthermore, a deterministic hash function does not allow for rehashing: sometimes the input data turns out to be bad for the hash function (e.g. there are too many collisions), so one would like to change the hash function.

The solution to these problems is to pick a function randomly from a large family of hash functions. The randomness in choosing the hash function can be used to guarantee some desired random behavior of the hash codes of any keys of interest. The first definition along these lines was universal hashing, which guarantees a low collision probability for any two designated keys. The concept of ${\displaystyle k}$-independent hashing, introduced by Wegman and Carter in 1981,[2] strengthens the guarantees of random behavior to families of ${\displaystyle k}$ designated keys, and adds a guarantee on the uniform distribution of hash codes.

Definitions

The strictest definition, introduced by Wegman and Carter[2] under the name "strongly universal${\displaystyle _{k}}$ hash family", is the following. A family of hash functions ${\displaystyle H=\{h:U\to [m]\}}$ is ${\displaystyle k}$-independent if for any ${\displaystyle k}$ distinct keys ${\displaystyle (x_{1},\dots ,x_{k})\in U^{k}}$ and any ${\displaystyle k}$ hash codes (not necessarily distinct) ${\displaystyle (y_{1},\dots ,y_{k})\in [m]^{k}}$, we have:

${\displaystyle \Pr _{h\in H}\left[h(x_{1})=y_{1}\land \cdots \land h(x_{k})=y_{k}\right]=m^{-k}}$

This definition is equivalent to the following two conditions:

1. for any fixed ${\displaystyle x\in U}$, as ${\displaystyle h}$ is drawn randomly from ${\displaystyle H}$, ${\displaystyle h(x)}$ is uniformly distributed in ${\displaystyle [m]}$.
2. for any fixed, distinct keys ${\displaystyle x_{1},\dots ,x_{k}\in U}$, as ${\displaystyle h}$ is drawn randomly from ${\displaystyle H}$, ${\displaystyle h(x_{1}),\dots ,h(x_{k})}$ are independent random variables.

Often it is inconvenient to achieve the perfect joint probability of ${\displaystyle m^{-k}}$ due to rounding issues. Following,[3] one may define a ${\displaystyle (\mu ,k)}$-independent family to satisfy:

${\displaystyle \forall }$ distinct ${\displaystyle (x_{1},\dots ,x_{k})\in U^{k}}$ and ${\displaystyle \forall (y_{1},\dots ,y_{k})\in [m]^{k}}$, ${\displaystyle ~~\Pr _{h\in H}\left[h(x_{1})=y_{1}\land \cdots \land h(x_{k})=y_{k}\right]\leq \mu /m^{k}}$

Observe that, even if ${\displaystyle \mu }$ is close to 1, ${\displaystyle h(x_{i})}$ are no longer independent random variables, which is often a problem in the analysis of randomized algorithms. Therefore, a more common alternative to dealing with rounding issues is to prove that the hash family is close in statistical distance to a ${\displaystyle k}$-independent family, which allows black-box use of the independence properties.

Techniques

Polynomials with random coefficients

The original technique for constructing k-independent hash functions, given by Carter and Wegman, was to select a large prime number p, choose k random numbers modulo p, and use these numbers as the coefficients of a polynomial of degree k − 1 whose values modulo p are used as the value of the hash function. All polynomials of the given degree modulo p are equally likely, and any polynomial is uniquely determined by any k-tuple of argument-value pairs with distinct arguments, from which it follows that any k-tuple of distinct arguments is equally likely to be mapped to any k-tuple of hash values.[2]

Tabulation hashing

Main article: Tabulation hashing

Tabulation hashing is a technique for mapping keys to hash values by partitioning each key into bytes, using each byte as the index into a table of random numbers (with a different table for each byte position), and combining the results of these table lookups by a bitwise exclusive or operation. Thus, it requires more randomness in its initialization than the polynomial method, but avoids possibly-slow multiplication operations. It is 3-independent but not 4-independent.[4] Variations of tabulation hashing can achieve higher degrees of independence by performing table lookups based on overlapping combinations of bits from the input key, or by applying simple tabulation hashing iteratively.[5][6]

Independence needed by different hashing methods

The notion of k-independence can be used to differentiate between different hashing methods, according to the level of independence required to guarantee constant expected time per operation.

For instance, hash chaining takes constant expected time even with a 2-independent hash function, because the expected time to perform a search for a given key is bounded by the expected number of collisions that key is involved in. By linearity of expectation, this expected number equals the sum, over all other keys in the hash table, of the probability that the given key and the other key collide. Because the terms of this sum only involve probabilistic events involving two keys, 2-independence is sufficient to ensure that this sum has the same value that it would for a truly random hash function.[2]

Double hashing is another method of hashing that requires a low degree of independence. It is a form of open addressing that uses two hash functions: one to determine the start of a probe sequence, and the other to determine the step size between positions in the probe sequence. As long as both of these are 2-independent, this method gives constant expected time per operation.[7]

On the other hand, linear probing, a simpler form of open addressing where the step size is always one, requires 5-independence. It can be guaranteed to work in constant expected time per operation with a 5-independent hash function,[8] and there exist 4-independent hash functions for which it takes logarithmic time per operation.[9]

References

1. Cormen, Thomas H.; Leiserson, Charles E.; Rivest, Ronald L.; Stein, Clifford (2009) [1990]. Introduction to Algorithms (3rd ed.). MIT Press and McGraw-Hill. ISBN 0-262-03384-4.
2. Wegman, Mark N.; Carter, J. Lawrence (1981). "New hash functions and their use in authentication and set equality" (PDF). Journal of Computer and System Sciences. 22 (3): 265–279. doi:10.1016/0022-0000(81)90033-7. Conference version in FOCS'79. Retrieved 9 February 2011.
3. Siegel, Alan (2004). "On universal classes of extremely random constant-time hash functions and their time-space tradeoff" (PDF). SIAM Journal on Computing. 33 (3): 505–543. doi:10.1137/S0097539701386216. Conference version in FOCS'89.
4. Pătraşcu, Mihai; Thorup, Mikkel (2012), "The power of simple tabulation hashing", Journal of the ACM, 59 (3): Art. 14, arXiv:1011.5200, doi:10.1145/2220357.2220361, MR 2946218.
5. Siegel, Alan (2004), "On universal classes of extremely random constant-time hash functions", SIAM Journal on Computing, 33 (3): 505–543, doi:10.1137/S0097539701386216, MR 2066640.
6. Thorup, M. (2013), "Simple tabulation, fast expanders, double tabulation, and high independence", Proceedings of the 54th Annual IEEE Symposium on Foundations of Computer Science (FOCS 2013), pp. 90–99, arXiv:1311.3121, doi:10.1109/FOCS.2013.18, ISBN 978-0-7695-5135-7, MR 3246210.
7. Bradford, Phillip G.; Katehakis, Michael N. (2007), "A probabilistic study on combinatorial expanders and hashing" (PDF), SIAM Journal on Computing, 37 (1): 83–111, doi:10.1137/S009753970444630X, MR 2306284, archived from the original (PDF) on 2016-01-25, retrieved 2016-01-19.
8. Pagh, Anna; Pagh, Rasmus; Ružić, Milan (2009), "Linear probing with constant independence", SIAM Journal on Computing, 39 (3): 1107–1120, arXiv:cs/0612055, doi:10.1137/070702278, MR 2538852
9. Pătraşcu, Mihai; Thorup, Mikkel (2010), "On the k-independence required by linear probing and minwise independence" (PDF), Automata, Languages and Programming, 37th International Colloquium, ICALP 2010, Bordeaux, France, July 6-10, 2010, Proceedings, Part I, Lecture Notes in Computer Science, 6198, Springer, pp. 715–726, arXiv:1302.5127, doi:10.1007/978-3-642-14165-2_60, ISBN 978-3-642-14164-5

Further reading

• Motwani, Rajeev; Raghavan, Prabhakar (1995). Randomized Algorithms. Cambridge University Press. p. 221. ISBN 978-0-521-47465-8.