Nullspace property

In compressed sensing, the nullspace property gives necessary and sufficient conditions on the reconstruction of sparse signals using the techniques of ${\displaystyle \ell _{1}}$-relaxation. The term "nullspace property" originates from Cohen, Dahmen, and DeVore.[1] The nullspace property is often difficult to check in practice, and the restricted isometry property is a more modern condition in the field of compressed sensing.

The technique of ${\displaystyle \ell _{1}}$-relaxation

The non-convex ${\displaystyle \ell _{0}}$-minimization problem,

${\displaystyle \min \limits _{x}\|x\|_{0}}$ subject to ${\displaystyle Ax=b}$,

is a standard problem in compressed sensing. However, ${\displaystyle \ell _{0}}$-minimization is known to be NP-hard in general.[2] As such, the technique of ${\displaystyle \ell _{1}}$-relaxation is sometimes employed to circumvent the difficulties of signal reconstruction using the ${\displaystyle \ell _{0}}$-norm. In ${\displaystyle \ell _{1}}$-relaxation, the ${\displaystyle \ell _{1}}$ problem,

${\displaystyle \min \limits _{x}\|x\|_{1}}$ subject to ${\displaystyle Ax=b}$,

is solved in place of the ${\displaystyle \ell _{0}}$ problem. Note that this relaxation is convex and hence amenable to the standard techniques of linear programming - a computationally desirable feature. Naturally we wish to know when ${\displaystyle \ell _{1}}$-relaxation will give the same answer as the ${\displaystyle \ell _{0}}$ problem. The nullspace property is one way to guarantee agreement.

Definition

An ${\displaystyle m\times n}$ complex matrix ${\displaystyle A}$ has the nullspace property of order ${\displaystyle s}$, if for all index sets ${\displaystyle S}$ with ${\displaystyle s=|S|\leq n}$ we have that: ${\displaystyle \|\eta _{S}\|_{1}<\|\eta _{S^{C}}\|_{1}}$ for all ${\displaystyle \eta \in \ker {A}\setminus \left\{0\right\}}$.

Recovery Condition

The following theorem gives necessary and sufficient condition on the recoverability of a given ${\displaystyle s}$-sparse vector in ${\displaystyle \mathbb {C} ^{n}}$ . The proof of the theorem is a standard one, and the proof supplied here is summarized from Holger Rauhut.[3]

${\displaystyle {\textbf {Theorem:}}}$ Let ${\displaystyle A}$ be a ${\displaystyle m\times n}$ complex matrix. Then every ${\displaystyle s}$-sparse signal ${\displaystyle x\in \mathbb {C} ^{n}}$ is the unique solution to the ${\displaystyle \ell _{1}}$-relaxation problem with ${\displaystyle b=Ax}$ if and only if ${\displaystyle A}$ satisfies the nullspace property with order ${\displaystyle s}$.

${\displaystyle {\textit {Proof:}}}$ For the forwards direction notice that ${\displaystyle \eta _{S}}$ and ${\displaystyle -\eta _{S^{C}}}$ are distinct vectors with ${\displaystyle A(-\eta _{S^{C}})=A(\eta _{S})}$ by the linearity of ${\displaystyle A}$, and hence by uniqueness we must have ${\displaystyle \|\eta _{S}\|_{1}<\|\eta _{S^{C}}\|_{1}}$ as desired. For the backwards direction, let ${\displaystyle x}$ be ${\displaystyle s}$-sparse and ${\displaystyle z}$ another (not necessary ${\displaystyle s}$-sparse) vector such that ${\displaystyle z\neq x}$ and ${\displaystyle Az=Ax}$. Define the (non-zero) vector ${\displaystyle \eta =x-z}$ and notice that it lies in the nullspace of ${\displaystyle A}$. Call ${\displaystyle S}$ the support of ${\displaystyle x}$, and then the result follows from an elementary application of the triangle inequality: ${\displaystyle \|x\|_{1}\leq \|x-z_{S}\|_{1}+\|z_{S}\|_{1}=\|\eta _{S}\|_{1}+\|z_{S}\|_{1}<\|\eta _{S^{C}}\|_{1}+\|z_{S}\|_{1}=\|-z_{S^{C}}\|_{1}+\|z_{S}\|_{1}=\|z\|_{1}}$, establishing the minimality of ${\displaystyle x}$. ${\displaystyle \square }$

References

1. Cohen, Albert; Dahmen, Wolfgang; DeVore, Ronald (2009-01-01). "Compressed sensing and best 𝑘-term approximation". Journal of the American Mathematical Society. 22 (1): 211–231. doi:10.1090/S0894-0347-08-00610-3. ISSN 0894-0347.
2. Natarajan, B. K. (1995-04-01). "Sparse Approximate Solutions to Linear Systems". SIAM J. Comput. 24 (2): 227–234. doi:10.1137/S0097539792240406. ISSN 0097-5397.
3. Rauhut, Holger (2011). Compressive Sensing and Structured Random Matrices. CiteSeerX 10.1.1.185.3754.

External links