Image derivatives

Image derivatives can be computed by using small convolution filters of size 2 x 2 or 3 x 3, such as the Laplacian, Sobel, Roberts and Prewitt operators.[1] However, a larger mask will generally give a better approximation of the derivative and examples of such filters are Gaussian derivatives [2] and Gabor filters.[3] Sometimes high frequency noise needs to be removed and this can be incorporated in the filter so that the Gaussian kernel will act as a band pass filter.[4] The use of Gabor filters [5] in image processing has been motivated by some of its similarities to the perception in the human visual system.[6]

The pixel value is computed as a convolution

${\displaystyle p'_{u}=\mathbf {d} \ast G}$

where ${\displaystyle \mathbf {d} }$ is the derivative kernel and ${\displaystyle G}$ is the pixel values in a region of the image and ${\displaystyle \ast }$ is the operator that performs the convolution.

Sobel derivatives

The derivative kernels, known as the Sobel operator are defined as follows, for the ${\displaystyle u}$ and ${\displaystyle v}$ directions respectively:

${\displaystyle p'_{u}={\begin{bmatrix}+1&+2&+1\\0&0&0\\-1&-2&-1\end{bmatrix}}*\mathbf {G} \quad {\mbox{and}}\quad p'_{v}={\begin{bmatrix}+1&0&-1\\+2&0&-2\\+1&0&-1\end{bmatrix}}*\mathbf {G} }$

where ${\displaystyle *}$ here denotes the 2-dimensional convolution operation.

This operator is separable and can be decomposed as the products of an interpolation and a differentiation kernel, so that, ${\displaystyle p'_{v}}$, for an example can be written as

${\displaystyle {\begin{bmatrix}+1&0&-1\\+2&0&-2\\+1&0&-1\end{bmatrix}}={\begin{bmatrix}1\\2\\1\end{bmatrix}}{\begin{bmatrix}+1&0&-1\end{bmatrix}}}$

Farid and Simoncelli derivatives

Farid and Simoncelli.[7][8] propose to use a pair of kernels, one for interpolation and another for differentiation (compare to Sobel above). These kernels, of fixed sizes 5 x 5 and 7 x 7, are optimized so that the Fourier transform approximates their correct derivative relationship.

In Matlab code the so called 5-tap filter is

k  = [0.030320  0.249724  0.439911  0.249724  0.030320];
d  = [0.104550  0.292315  0.000000 -0.292315 -0.104550];
d2 = [0.232905  0.002668 -0.471147  0.002668  0.232905];

And the 7-tap filter is

k  = [ 0.004711  0.069321  0.245410  0.361117  0.245410  0.069321  0.004711];
d  = [ 0.018708  0.125376  0.193091  0.000000 -0.193091 -0.125376 -0.018708];
d2 = [ 0.055336  0.137778 -0.056554 -0.273118 -0.056554  0.137778  0.055336];

As an example the first order derivatives can be computed in the following using Matlab in order to perform the convolution

 
Iu = conv2(d, k, im, 'same');  % derivative vertically (wrt Y)
Iv = conv2(k, d, im, 'same');  % derivative horizontally (wrt X)

It is noted that Farid and Simoncelli have derived first derivative coefficients which are more accurate compared to the ones provided above. However, the latter are consistent with the second derivative interpolator and, therefore, are better to use if both the first and second derivatives are sought. In the opposite case, when only the first derivative is desired, the optimal first derivative coefficients should be employed; more details can be found in their paper.

Hast derivatives

Derivative filters based on arbitrary cubic splines was presented by Hast.[9] He showed how both first and second order derivatives can be computed more correctly using cubic or trigonometric splines. Efficient derivative filters need to be of odd length so that the derivative is computed for the central pixel. However, any cubic filter is fitted over 4 sample points, giving a centre that falls between pixels. This is solved by a double filtering approach giving filters of size 7 x 7. The idea is to first filter by interpolation so that the interpolated value between pixels are obtained, whereafter the procedure is repeated using a derivative filters, where the centre value now falls on pixel centres. This can easily be proved by the associative law for convolution

${\displaystyle p'_{u}=\mathbf {d} \ast (\mathbf {k} \ast G)=(\mathbf {d} \ast \mathbf {k} )\ast G}$

Therefore the convolution kernel for computing the derivative ${\displaystyle \mathbf {k_{d}} }$ using an interpolating kernel ${\displaystyle \mathbf {k} }$ and a derivative kernel ${\displaystyle \mathbf {d} }$ becomes

${\displaystyle \mathbf {k_{d}} =\mathbf {d} \ast \mathbf {k} }$

Also keep in mind that convolution is commutative, so that the order of the two kernels does not matter and it is also possible to insert a second order derivative as well as a first order derivative kernel. These kernels are derived from the fact that any spline surface can be fitted over a square pixel region, compare to Bezier surfaces. Hast proves that such a surface can be performed as a separable convolution

${\displaystyle p(u,v)=\mathbf {u} ^{T}MGM^{T}\mathbf {v} =M^{T}\mathbf {u} \otimes \mathbf {v} ^{T}M\ast G}$

where ${\displaystyle M}$ is the spline basis matrix, ${\displaystyle \mathbf {u} }$ and ${\displaystyle \mathbf {v} }$ are vectors containing the variables ${\displaystyle u}$ and ${\displaystyle v}$, such as

${\displaystyle \mathbf {u} =[u^{3},u^{2},u,1]^{T}}$

${\displaystyle \mathbf {v} =[v^{3},v^{2},v,1]^{T}}$

The convolution kernels can now be set to

${\displaystyle \mathbf {k} =\mathbf {u} ^{T}M=(M^{T}\mathbf {v} )^{T}}$

${\displaystyle \mathbf {d} ={\frac {\partial \mathbf {u} ^{T}}{\partial u}}M=\left(M^{T}{\frac {\partial \mathbf {v} }{\partial v}}\right)^{T}}$

${\displaystyle \mathbf {d^{2}} ={\frac {\partial ^{2}\mathbf {u} ^{T}}{\partial u^{2}}}M=\left(M^{T}{\frac {\partial ^{2}\mathbf {v} }{\partial v^{2}}}\right)^{T}}$

The first order derivatives at the central pixel are hence computed as

${\displaystyle D_{u}={\frac {\partial \mathbf {u} ^{T}}{\partial u}}M\ast \mathbf {u} ^{T}MGM^{T}\mathbf {v} \ast M^{T}\mathbf {v} =\mathbf {d} \ast \mathbf {k} \otimes (\mathbf {k} \ast \mathbf {k} )^{T}\ast G}$

and

${\displaystyle D_{v}=\mathbf {u} ^{T}M\ast \mathbf {u} ^{T}MGM^{T}\mathbf {v} \ast M^{T}{\frac {\partial \mathbf {v} }{\partial v}}=\mathbf {k} \ast \mathbf {k} \otimes (\mathbf {d} \ast \mathbf {k} )^{T}\ast G}$

Likewise, with the second order derivative kernels are

${\displaystyle D_{u}^{2}={\frac {\partial ^{2}\mathbf {u} ^{T}}{\partial u^{2}}}M\ast \mathbf {u} ^{T}MGM^{T}\mathbf {v} \ast M^{T}\mathbf {v} =\mathbf {d^{2}} \ast \mathbf {k} \otimes (\mathbf {k} \ast \mathbf {k} )^{T}\ast G}$

and

${\displaystyle D_{v}^{2}=\mathbf {u} ^{T}M\ast \mathbf {u} ^{T}MGM^{T}\mathbf {v} \ast M^{T}{\frac {\partial ^{2}\mathbf {v} }{\partial v^{2}}}=\mathbf {k} \ast \mathbf {k} \otimes (\mathbf {d^{2}} \ast \mathbf {k} )^{T}\ast G}$

The cubic spline filter is evaluated in its centre ${\displaystyle u=v=0.5}$ and therefore

${\displaystyle \mathbf {u} =\mathbf {v} =[(0.5)^{3},(0.5)^{2},0.5,1]^{T}=[0.125,0.25,0.5,1]^{T}}$

Likewise the first order derivatives becomes

${\displaystyle {\frac {\partial \mathbf {u} (0.5)}{\partial u}}={\frac {\partial \mathbf {v} (0.5)}{\partial v}}=[3\cdot (0.5)^{2},2\cdot (0.5),1,0]^{T}=[0.75,1,1,0]^{T}}$

And in a similar manner the second order derivatives are

${\displaystyle {\frac {d^{2}\mathbf {u} (0.5)}{u^{2}}}={\frac {d^{2}\mathbf {v} (0.5)}{dv^{2}}}=[6\cdot (0.5),2,0,0]^{T}=[3,2,0,0]^{T}}$

Any cubic filter can be applied and used for computing the image derivates using the above equations, such as Bézier, Hermite or B-splines.

The example in below in Matlab use the Catmull-Rom spline to compute the derivatives

 
M = [1,-3,3,-1; -1,4,-5,2; 0,1,0,-1; 0,0,2,0] * 0.5;
u = [0.125;0.25;0.5;1];
up = [0.75;1;1;0];
d = up'*M;
k = u'*M;
Iu = conv2(conv(d,k), conv(k,k), im,'same');  % vertical derivative (wrt Y)
Iv = conv2(conv(k,k), conv(d,k), im,'same');  % horizontal derivative (wrt X)

Other approaches

Steerable filters can be used for computing derivatives [10] Moreover, Savitzky and Golay [11] propose a least-squares polynomial smoothing approach, which could be used for computing derivatives and Luo et al [12] discuss this approach in further detail. Scharr [13][14][15] shows how to create derivative filters by minimizing the error in the Fourier domain and Jähne et al [16] discuss in more detail the principles of filter design, including derivative filters.

References

1. Pratt, W.K., 2007. Digital image processing (4th ed.). John Wiley & Sons, Inc. pp. 465–522
2. H. Bouma, A. Vilanova, J.O. Bescós, B.M.T.H. Romeny, F.A. Gerritsen, Fast and accurate gaussian derivatives based on b-splines, in: Proceedings of the 1st International Conference on Scale Space and Variational Methods in Computer Vision, Springer-Verlag, Berlin, Heidelberg, 2007, pp. 406–417.
3. P. Moreno, A. Bernardino, J. Santos-Victor, Improving the sift descriptor with smooth derivative filters, Pattern Recognition Letters 30 (2009) 18–26.
4. J.J. Koenderink, A.J. van Doorn, Generic neighborhood operators, IEEE Trans. Pattern Anal. Mach. Intell. 14 (1992) 597–605.
5. D. Gabor, Theory of communication, J. Inst. Electr. Eng. 93 (1946) 429–457.
6. J.G. Daugman, Complete discrete 2-D Gabor transforms by neural networks for image analysis and compression, IEEE Trans. Acoust. Speech Signal Process. 36 (1988) 1169–1179.
7. H. Farid and E. P. Simoncelli, Differentiation of discrete multi-dimensional signals, IEEE Trans Image Processing, vol.13(4), pp. 496--508, Apr 2004.
8. H. Farid and E. P. Simoncelli, Optimally Rotation-Equivariant Directional Derivative Kernels, Int'l Conf Computer Analysis of Images and Patterns, pp. 207--214, Sep 1997.
9. A. Hast., "Simple filter design for first and second order derivatives by a double filtering approach", Pattern Recognition Letters, Vol. 42, no.1 June, pp. 65--71. 2014.
10. W.T. Freeman, E.H. Adelson, The design and use of steerable filters, IEEE Trans. Pattern Anal. Mach. Intell. 13 (1991) 891–906.
11. A. Savitzky, M.J.E. Golay, Smoothing and differentiation of data by simplified least squares procedures, Anal. Chem. 36 (1964) 1627–1639.
12. J. Luo, K. Ying, P. He, J. Bai, Properties of savitzky–golay digital differentiators, Digit. Signal Process. 15 (2005) 122–136.
13. H. Scharr, Optimal second order derivative filter families for transparent motion estimation, in: M. Domanski, R. Stasinski, M. Bartkowiak (Eds.), EUSIPCO 2007.
14. Scharr, Hanno, 2000, Dissertation (in German), Optimal Operators in Digital Image Processing .
15. B. Jähne, H. Scharr, and S. Körkel. Principles of filter design. In Handbook of Computer Vision and Applications. Academic Press, 1999.
16. B. Jähne, P. Geissler, H. Haussecker (Eds.), Handbook of Computer Vision and Applications with Cdrom, 1st ed., Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1999, pp. 125–151 (Chapter 6).

External links

• derivative5.m Farid and Simoncelli: 5-Tap 1st and 2nd discrete derivatives.
• derivative7.m Farid and Simoncelli: 7-Tap 1st and 2nd discrete derivatives
• kernel.m Hast: 1st and 2nd discrete derivatives for Cubic splines, Catmull-Rom splines, Bezier splines, B-Splines and Trigonometric splines.