Nyström method

In numerical analysis, the Nyström method[1] or quadrature method seeks the numerical solution of an integral equation by replacing the integral with a representative weighted sum. The continuous problem is broken into ${\displaystyle n}$ discrete intervals; quadrature or numerical integration determines the weights and locations of representative points for the integral.

The problem becomes a system of linear equations with ${\displaystyle n}$ equations and ${\displaystyle n}$ unknowns, and the underlying function is implicitly represented by an interpolation using the chosen quadrature rule. This discrete problem may be ill-conditioned, depending on the original problem and the chosen quadrature rule.

Since the linear equations require ${\displaystyle O(n^{3})}$ operations to solve, high-order quadrature rules perform better because low-order quadrature rules require large ${\displaystyle n}$ for a given accuracy. Gaussian quadrature is normally a good choice for smooth, non-singular problems.

Discretization of the integral

Standard quadrature methods seek to represent an integral as a weighed sum in the following manner:

${\displaystyle \int _{a}^{b}h(x)\;\mathrm {d} x\approx \sum _{k=1}^{n}w_{k}h(x_{k})}$

where ${\displaystyle w_{k}}$ are the weights of the quadrature rule, and points ${\displaystyle x_{k}}$ are the abscissas.

Example

Applying this to the inhomogeneous Fredholm equation of the second kind

${\displaystyle f(x)=\lambda u(x)-\int _{a}^{b}K(x,x')f(x')\;\mathrm {d} x'}$,

results in

${\displaystyle f(x)\approx \lambda u(x)-\sum _{k=1}^{n}w_{k}K(x,x_{k})f(x_{k})}$.

Sampling for Nyström method

The most important step of the Nyström method is sampling, because different sampled landmark points give different approximations of the original matrix.

Uniform sampling without replacement is the most used approach for this purpose, where every point has the same probability of being included in the sample.

Minimum Sum of Squared Similarities Sampling (MSSS)

MSSS is a non-uniform sampling approach that considers both the variance and the similarity of the data distribution in its sampling data, which approximately maximizes the determinant of the reduced similarity matrix that represents the mutual similarities between sampled data points. It is shown in[2] that this approach increase the speed of the clustering on large datasets.

References

1. Nyström, Evert Johannes (1930). "Über die praktische Auflösung von Integralgleichungen mit Anwendungen auf Randwertaufgaben". Acta Mathematica. 54 (1): 185–204. doi:10.1007/BF02547521.
2. Bouneffouf, Djallel; Birol, Inanc (2015), "Sampling with Minimum Sum of Squared Similarities for Nystrom-Based Large Scale Spectral Clustering", Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, {IJCAI} 2015, Buenos Aires, Argentina, July 25–31, 2015, 8834, AAAI Press, pp. 2313–2319, ISBN 978-1-57735-738-4

• Leonard M. Delves & Joan E. Walsh (eds): Numerical Solution of Integral Equations, Clarendon, Oxford, 1974.
• Hans-Jürgen Reinhardt: Analysis of Approximation Methods for Differential and Integral Equations, Springer, New York, 1985.