Continuation of JASA paper

Darren Warren — Mon, 11 Apr 2011 14:52:54 +0000

After finishing and submitting JASA paper (May 7), write an extension that utilizes the asymptotic approximation of Hankle and Toeplitz matrices with circulant matrices. This would allow for an asymptotic equivalence type argument.

Also, do kernel match + detection problem (June 15).

Finish NIPS paper with Dan, (June 1).

Questions

Darren Warren — Tue, 08 Feb 2011 23:06:57 +0000

Algorithmic stability in IPs
Chris: funding
Lucky Imaging: projection estimator

Looming: COLT, Texas A&M, and JSM

Darren Warren — Fri, 21 Jan 2011 21:58:29 +0000

Find minimax risk for sequential problem (asymptotic in n). Adapt Belitzer and Levit.
Bound ratio of risks of Beran’s estimators and Berteros (and lucky imaging).
Read Chapter 10 of Cesa-Bianchi: Online Inverse Problem?
Convex optimization for monotone estimator
Read minimax detection in inverse problems paper.

DS and inverse problems

Darren Warren — Sun, 16 Jan 2011 17:04:39 +0000

Suppose and let be a basis (frame?) for space to which belongs.
Act this observation functional on , then we get a sequence space representation as where forms a non-independent, but nearly independent set if is sufficiently well behaved (think homogeneous, dilation invariant; or polynomial decay convolution).

Ask James about his DS for correlated noise
We can apply this to WVD by the above formulation.
Fundamental Question: Can we use the adaptive sampling framework to estimate $\latex \theta$ better?

Power: Lower Bound on Polynomial Convergence

Darren Warren — Mon, 11 Oct 2010 18:40:13 +0000

Introduction

Suppose we are attempting to match two images, each with differing amounts of seeing and noise. One we call the reference image and it is obtained with no noise. The second is called the science image and it generally has more severe seeing and has noise. We wish to find some transformation that maps to such that the difference is noise-like if there are no additional sources.

In particular, we posit a sequence of operators and 0}' title='{\sigma > 0}' class='latex' /> such that there exists a where . Our goal is to choose based only on data.

1. Result

We propose a multiresolution noise-like statistic based on an idea in Davies and Kovak 1991, defined as:

where is a multiresolution analysis of the pixelized grid and . Note that we do not include the multiplicative factor whenever it is not explicitly needed.

Observe that we can write this as

by adding and subtracting . Note that the summation of is supressed in the first term for notational clarity. Also, when is fixed, we supress that argument.

Now, one quality this statistic could have would be to asymptotically distinguish between competing hypotheses. In this case, low-noise asymptotics makes more sense than large sample, so we choose this regime.

Our goal is to look at the power of this statistic to determine amongst hypothesis asymptotically. It is known (Das Gupta 2008) that asymptotics for fixed alternative hypothesis leads to trivial results, such as power always tending toward 1.

Hence, we wish to look at an analogy to the Pittman slope. This can be phrased as follows. Let 0}' title='{\tau > 0}' class='latex' /> be given. Then we want to look at

\tau \right) \ \ \ \ \ (2)' title='\displaystyle \lim_{\sigma \rightarrow 0} \mathbb{P} \left( \frac{NL(\lambda_0 + \Delta C_{\sigma})}{NL(\lambda_0)} > \tau \right) \ \ \ \ \ (2)' class='latex' />

where is a function going to zero with and is a constant. We look at the ratio of the test under the alternate and null hypothesis as a way of rescaling. Alternatively, we can make a function of . We see in what follows the ratio in effect chooses that function.

Lemma 1 We can rewrite (2) as
\tau \right) & = & \lim_{\sigma \rightarrow 0} \mathbb{P} \left( \frac{ \sup_{I \in \mathcal{I}} \left| \sum_{i \in I} (K_{\lambda_0 + \Delta C(\sigma)} - K_{\lambda_0})R + \sigma Z_i \right| } { \sup_{I \in \mathcal{I}} \left| \sum_{i \in I} Z_i \right| } > \sigma \tau \right) \end{array} ' title='\displaystyle \begin{array}{rcl} \lim_{\sigma \rightarrow 0} \mathbb{P} \left( \frac{NL(\lambda_0 + \Delta C_{\sigma})}{NL(\lambda_0)} > \tau \right) & = & \lim_{\sigma \rightarrow 0} \mathbb{P} \left( \frac{ \sup_{I \in \mathcal{I}} \left| \sum_{i \in I} (K_{\lambda_0 + \Delta C(\sigma)} - K_{\lambda_0})R + \sigma Z_i \right| } { \sup_{I \in \mathcal{I}} \left| \sum_{i \in I} Z_i \right| } > \sigma \tau \right) \end{array} ' class='latex' />

Proof: Use (1) and multiply by .

This is a difficult seeming probability to calculate, even asymptotically. Hence we use the limit as a heuristic that the absence of in the denominator within the probability allows us to consider the following instead

\sigma \tau \right) ' title='\displaystyle \lim_{\sigma \rightarrow 0} \mathbb{P} \left( \sup_{I \in \mathcal{I}} \left| \sum_{i \in I} (K_{\lambda_0 + \Delta C(\sigma)} - K_{\lambda_0})R + \sigma Z_i \right| > \sigma \tau \right) ' class='latex' />

Before continuing, we need a result for exchanging and :

Lemma 2 Let be a sequence of random variables over some index such that for some . Then for any 0}' title='{\tau > 0}' class='latex' />
\tau) \geq \sup_t \mathbb{P}( X_t > \tau) ' title='\displaystyle \mathbb{P} ( \sup_t X_t > \tau) \geq \sup_t \mathbb{P}( X_t > \tau) ' class='latex' />

Proof: Write \tau) = \mathbb{E} \mathbf{1}(\sup_t X_t > \tau)}' title='{\mathbb{P} ( \sup_t X_t > \tau) = \mathbb{E} \mathbf{1}(\sup_t X_t > \tau)}' class='latex' />. Now, since \tau) = \sup_t \mathbf{1}(X_t > \tau)}' title='{\mathbf{1}(\sup_t X_t > \tau) = \sup_t \mathbf{1}(X_t > \tau)}' class='latex' />, we see that

\tau) = \mathbb{E} \sup_t \mathbf{1}( X_t > \tau) \geq \sup_t \mathbb{E} \mathbf{1}( X_t > \tau) ' title='\displaystyle \mathbb{P} ( \sup_t X_t > \tau) = \mathbb{E} \sup_t \mathbf{1}( X_t > \tau) \geq \sup_t \mathbb{E} \mathbf{1}( X_t > \tau) ' class='latex' />

where for the last inequality we use that for the necessary kinds of sequences of functions and measures.

Using Lemma 2, we can write

\sigma \tau \right) \geq \sup_{I \in \mathcal{I}} \mathbb{P} \left( \left| \sum_{i \in I} (K_{\lambda_0 + \Delta C(\sigma)} - K_{\lambda_0})R + \sigma Z_i \right| > \sigma \tau \right). \ \ \ \ \ (3)' title='\displaystyle \mathbb{P} \left( \sup_{I \in \mathcal{I}} \left| \sum_{i \in I} (K_{\lambda_0 + \Delta C(\sigma)} - K_{\lambda_0})R + \sigma Z_i \right| > \sigma \tau \right) \geq \sup_{I \in \mathcal{I}} \mathbb{P} \left( \left| \sum_{i \in I} (K_{\lambda_0 + \Delta C(\sigma)} - K_{\lambda_0})R + \sigma Z_i \right| > \sigma \tau \right). \ \ \ \ \ (3)' class='latex' />

Now, we would like to examine the such that the RHS of (3) . First, we can compute the RHS of (3) as follows. Define . Then

\sigma \tau \right) = 1 + \Phi\left( -\sqrt{|I|}\left( \tau + \frac{\mu_{I,\sigma}}{\sigma} \right) \right) - \Phi\left( \sqrt{|I|}\left( \tau - \frac{\mu_{I,\sigma}}{\sigma} \right) \right) \ \ \ \ \ (4)' title='\displaystyle \mathbb{P} \left( \left| \sum_{i \in I} (K_{\lambda_0 + \Delta C(\sigma)} - K_{\lambda_0})R + \sigma Z_i \right| > \sigma \tau \right) = 1 + \Phi\left( -\sqrt{|I|}\left( \tau + \frac{\mu_{I,\sigma}}{\sigma} \right) \right) - \Phi\left( \sqrt{|I|}\left( \tau - \frac{\mu_{I,\sigma}}{\sigma} \right) \right) \ \ \ \ \ (4)' class='latex' />

by noticing that the sum inside the absolute value is a random variable.

Using that for any doubly indexed sequence we see that under (3) and (4)

\sigma \tau \right) & \geq & \sup_{I \in \mathcal{I}} \lim_{\sigma \rightarrow 0} \bigg[ 1 + \Phi\left( -\sqrt{|I|}\left( \tau + \frac{\mu_{I,\sigma}}{\sigma} \right) \right)- \\ && \Phi\left( \sqrt{|I|}\left( \tau - \frac{\mu_{I,\sigma}}{\sigma} \right) \right)\bigg]. \end{array} ' title='\displaystyle \begin{array}{rcl} \lim_{\sigma \rightarrow 0} \mathbb{P} \left( \sup_{I \in \mathcal{I}} \left| \sum_{i \in I} (K_{\lambda_0 + \Delta C(\sigma)} - K_{\lambda_0})R + \sigma Z_i \right| > \sigma \tau \right) & \geq & \sup_{I \in \mathcal{I}} \lim_{\sigma \rightarrow 0} \bigg[ 1 + \Phi\left( -\sqrt{|I|}\left( \tau + \frac{\mu_{I,\sigma}}{\sigma} \right) \right)- \\ && \Phi\left( \sqrt{|I|}\left( \tau - \frac{\mu_{I,\sigma}}{\sigma} \right) \right)\bigg]. \end{array} ' class='latex' />

Now, we see that this probability goes to 1 when . We did this calculation for the case where has for a kernel a non-normalized Gaussian kernel with variance for all .

The result was that

and

However, the second of the two results is not informative. If we additionally assume that for 0}' title='{\alpha > 0}' class='latex' /> we get

and

1. ' title='\displaystyle \frac{\mu_{I,\sigma}}{\sigma} \rightarrow 0 \quad \textrm{if} \quad \alpha > 1. ' class='latex' />

We’re not sure what happens if .

Protected: Risk Estimation

Darren Warren — Wed, 01 Sep 2010 18:21:39 +0000

Statistics as an Inverse Problem

Darren Warren — Sun, 29 Aug 2010 23:37:05 +0000

As everyone in who has shared an office with me knows, I spend a lot of time thinking about inverse problems. In particular, statistical inverse problems. I think this is important because fundamentally statistics is an inverse problem. Observe the following very general statistical problem. Suppose is some separable Banach space. Suppose is a model space. Let be a sequence of elements in the dual space of continuous linear functionals on . Lastly, suppose , 0}' title='{\sigma >0}' class='latex' /> is a positive number, and is a random variable defined on . Then we can define the statisical problem as a mapping where the observations are for each or equivalently . Hence, our goal is to make some inference about given an observation .

Generally speaking, there is usually another layer of formalization that encapsulates what we mean by `some inference.’ In particular, suppose we have a space and define a set of mappings . Then, we consider the elements of as models and either the space as the parameter space.

Now, the goal is to make a model , a set of functionals , a parameter of interest and an observation and develop some estimator that is a - measureable function from to .

Protected: Inverse Problem Formalization (Working)

Darren Warren — Fri, 27 Aug 2010 17:17:31 +0000

Weak-ly Update