For certain mixture models, improper priors are undesirable because they yield improper posteriors. On the other hand, proper priors may be undesirable because they require subjective input. We propose the use of specially chosen data dependent priors. We show that in some cases, data dependent priors are the only priors that produce intervals with second order correct frequentist coverage. The resulting posterior also has another interpretation: it is the product of a fixed prior and a pseudo-likelihood.