Tracy Morrison Sweet

After my meeting a few things:

Grant and MM paper deadlines are 1/31.  Once these things are off my plate, I can focus on other things for my thesis.

For the next 20 days, see if I can model something using the P&S data to incorporate the effect of teaching the same grade.  We brainstormed three ideas today during the meeting…hopefully one of those will work.

I’m rerunning the 15 individual MMSBM fits with stronger priors on gamma (1,5).  And running the observational simulation with weak and strong priors….just to see.

And I have a simulation to see if data simulated from a HLSM with mixture model LS positions can be fitted using HMMSBMs.

Deadlines!!!

December 21, 2011 – Mailing draft of MMSBM paper!

January 6, 2011 – AERA invited IES student poster:

• Submit a 1-2 description of my work (MMSBM stuff)

January 15, 2011 – RAND Summer Internship

• Refine CV and cover letter
• Ask Steve to be a reference

**January 31,2011 is a realistic deadline for the MM chapter for Steve’s book.

I can talk about some results….that I might be able to add to the paper.  I have this sticky issue of the theta’s moving to the edges of my simplex, yet the other papers I’ve read have results where the points do lie on the edges of the simplex.  So I’m assuming they’ve encountered those issues as well.

Here are some results for the monk data.  The simplex doesn’t look exactly like the Airoldi paper, but they used different data.  An email was sent, but I think actually the plots are confirmation enough.

Another option for the paper is fit a simulated model and plot the logposterior for a single network?

Monk Data Airolidi

Monk Membership Medians

Monk Pretty Network Results

And now for the Hierarchical Stuff:

Intervention Network Data

Int Sim Networks Fitted Pretty

Int Alpha Traceplot

Int Alpha Density

The Observation stuff is worse.  But somehow also not worse.  The predicted networks look similar – and the estimates for Beta and Tau are really accurate…except that I used pretty strong priors.  I’m backing off that now and I trying to use more diffuse priors on the second run.  But the question is, if I use more diffuse priors – won’t I kind of have an identifiability issue with tau and beta?

For example, gamma’s are being estimated as ~Gamma(10, 50) but with only 20 gammas, they could have come from ~Gamma(1,5) distribution.  So should I somehow include this dependency in my MCMC?  Not sure yet what to do.

These results are from with the following priors:

beta ~ G(50,1) and tau~G(10, 1)  (I’m running (5, .1) and (1, .1) now.)  And note that the data were generated using gamma~G(10,50).  So I could try using a more diffuse prior to generate the data as well.

Gamma Traceplots

Obs Fitted Networks Pretty

ObservationalSimsNetworksPretty

Tau and Beta Traceplots

Gamma Densities

There are many and I have a feeling it will prove necessary to have them all in one place.

December 15, 2011 – the JSM paper competition.  Just need Brian to sign his letter, scan it and email the following:

• Government: Paper, cover letter, and letter from Brian
• SBSS: paper, letter from Brian, CV, and cover letter

January 6, 2011 – AERA invited IES student poster:

• Submit a 1-2 description of my work

January 15, 2011 – RAND Summer Internship

• Refine CV and cover letter
• Ask Steve to be a reference

**January 31,2011 is a realistic deadline for the MM chapter for Steve’s book.

Goals:

(1) Start coding a simple HMMSBM where the Dirichlet hyperparameters are estimated and come from the same distribution whose hyperparameters are also estimated.  We will hold \xi fixed to be 1/g for now, and try to do just estimate \gamma_k for each network now.

(2) Make sure things for the single network guy are working out.  Maybe keep \xi fixed?  See how that works maybe…

Lots of goals, but lots of progress!  Ask Brian about writing me a letter for a JSM travel award.

stud12an

MMSBM – I coded the single network sampler to estimate the Dirichlet hyperparameter and so far things look okay.

***Found a pretty major mistake, so I think the gammas won’t go flying off into space – hooray!***

I have 3 different networks I’m working with, which I’m going to change to 2.  More explanation below.

(1)  membership probabilities generated with c(0.05, 0.05, 0.05) lambda.  With fairly flat priors (\gamma ~ Gamma(1,1) and \xi ~ Dir(1,1,1)), we recovered these nicely and in 5000 iterations.

(2)  membership probs generated with c(0.25, 0.25, 0.25) was much more of a challenge.  For some reason, with \gamma ~ Gamma(1,1), our gamma parameters were just going off into oblivion.  I actually couldn’t tell which was going off first, the \xi or the \gammas, but a prior of Gamma (1,2) was enough to keep things on track.  Oops, spoke too soon.  It keeps things on track about 30% of the time.

**I think the solution might be to bound the gamma, but I’m sure there’s a better explanation***

(3)  I think it doesn’t really make sense to focus on networks generating with anything higher, actually.  But the third network is generated with c(0.5, 0.5, 0.5) and I anticipate the same issue with gamma and xi.    We’ll see if a Gamma(1,2) prior works on this one.

Hopefully Brian and Andrew can help me figure that out tomorrow.

HLSM – results have been put in the final paper.  It’s been proofed and now the only things I need to do are (a) proof read that last section and (b) ready it for submission!!!

And as far as paper writing is concerned, I’ve “started” the MM chapter, but I have a ways to go.  I am going to write the paper in more detail later this week…but I’m having a hard time jumping in to write when I don’t actually have results yet.  But now that things are looking a little better, I think I might be ready to do a bit more.  An introduction will be last, but that’s not too terribly taxing.  I’ll probably need to dig up a few more papers depending on whether I focus on a blockmodel entrance or mm entrance…or something else entirely.

On my To Do list from Wednesday.

(1) Find a network clustering algorithm.  Ask Andrew for one.

Update: Using Mclust instead, but I have to deal with a warning b/c of the dimensionality:data ratio.  Uses BIC to choose clusters which is better than what I was doing before.

Got one – it’s not great, but uses Hierarchical clustering and plot to choose number of clusters.  I wrote a function to “choose” the optimal number of clusters based one slopes (to find the “elbow”).  It’s slow, but usable for now.

(2)  Incorporate rho into the MCMC by using the estimate in the Ado paper.  Not the MLE, but the “dumb” estimate.

Yes, it was another headache, but it’s done.  Will post about that below.

(3)  Run clustering algorithm on original Y, newY from model with rho, newY from model without rho.  Compare plots too.

I realized that I was making my life difficult.  Instead of using the estimates for the sender and receiver parameters, I was regenerating them from the theta’s.  So in fact things really aren’t *that* bad.

Here are networks generated with and without rho. We can see using Rho we do much better.

New Networks

Also reconstruct the adjacency matricies like Airoldi did.

Look, how nice! And both seem fine.  It’s odd then that the other plots were weird.  Looking into this now….

Original vs 2 threshold plots – with rho

Orignal vs 2 thresholds no rho

(4)  How well do these models fit?  How can we tell?  See 5 and 6.

(5)  Posterior Predictive Checks: simulate theta from posterior (i.e. step i in mcmc), then generate new Y using sampling distribution.  Calculate T(newY) for each step – this is the null distribution of T.  Compare with T(Y).

Three values of T: density, density of outside group ties, number of networks.  Upon first glance, things look GOOD, but the number of networks was being determined so I’m running that now…and it’s terribly slow.  So I’ll post plots tomorrow.

Posterior Predictive Measures – No Rho

*******

Putting analyses into the JEBS paper now that Brian is finished with his edits.  I’m happy to almost be done with that paper (or at least until it gets returned with a revise and resubmit fingers crossed).  I’m just redoing some of the plots now and tomorrow I plan on dropping them into the latex file, rewriting some of the analyses, and proofing the entire thing.

Also on board for this week is finally coding the HMMSBM.  But in light of all of this, the question remains…where should the intervention coefficient lie?  Perhaps the idea is to go from dirichlet (0.2, 0.2, 0.2, 0.2) to (1000,1,1,1)?  Maybe, we’ll see what happens.

And then we’ll be able to start writing the MMSBM paper. I just need to keep the momentum for a few more weeks (like 5-6) and then I can breathe for the 2 full weeks that I’ll be taking off.

(1) Find a network clustering algorithm.  Ask Andrew for one.

(2)  Incorporate rho into the MCMC by using the estimate in the Ado paper.  Not the MLE, but the “dumb” estimate.

(3)  Run clustering algorithm on original Y, newY from model with rho, newY from model without rho.  Compare plots too.

Also reconstruct the adjacency matricies like Airoldi did.

(4)  How well do these models fit?  How can we tell?  See 5 and 6.

(5)  Posterior Predictive Checks: simulate theta from posterior (i.e. step i in mcmc), then generate new Y using sampling distribution.  Calculate T(newY) for each step – this is the null distribution of T.  Compare with T(Y).

Ideas for T: density, entry of B matrix, number of clusters, take 1 nodes and determine the proportion of ties to each of the other clusters.

(6)  When the time comes, we might include the rho parameter in the mcmc…as a latent trait.  Ties are marked by a latent variable either as being forced to be 0 or having a blockmodel edge (prob of ZBZ).

(7)  Another idea for model comparison is to look at Bayes factors.  See notes from meeting and read Kass & Raftery paper.

Things never seem to work out when you need them to.  /sigh

My log posterior traceplots are not working out.  For some reason my log posterior traceplots (data set 1) are converging to each other (even though the individual parameter posteriors converge to different values).

And my tuning function is not working out.  I’m afraid I’m going to have to adjust the tuning parameter for each latent space position separately.  But I’m not even sure that’ll work.   I really don’t understand why things aren’t working.

I think I need to test my one of my simulated simple data sets with this sampler and see if I can tune things. (Possible “to do” for this weekend while I’m traveling?)

***So I’m not sure what’s going on now.  At the airport and working on the log posterior traceplots.  First of the first – if I work with just my medians, the log posterior is close to the true log posterior.  If I look at a traceplot, not so much.

I can’t seem to figure out if it’s a coding issue (it must be, right?)

So it seems a stronger prior for the entries of the B matrix is needed when either the network is larger or there are more groups.  Recall for 3 groups, I used a uniform prior and things looked fine.

Test Network

And a prior of Beta(1, 3) and Beta(10,1) worked for the diagonal and off-diagonal entries when N=50 and G=4.

Test1Network

Now, it seems we need priors of Beta(1,10) and Beta(30, 1) for the other 3 situations.

Test2Network-new

Test3Network-new

Test4Network-new

But this is enough to start thinking about true mixed membership models (i.e. allowing theta to vary a bit more).  And get back to coding my HMMSBMs!

Update: HMMSBM without hierarchical structure is working!

******************

Regardling HLSMs, my adjust my tune function is problematic but it is ridiculous to have to adjust every single tuning parameter by hand.  We are still fine tuning it so that it will have at least some utility.  More time now = less time later (hopefully).

Thesis Meeting at 1:30:  Talk about identifiability issue/priors vs truncated distributions.  Talk about hopefully getting the HLSM out by next week for the JEBS paper.

A couple of things to think about/do:

(1) Do I really need a strong prior or can I just use a flat prior on the B’s.  Check convergence using the posterior (i.e. evaluate the likelihood and prior at each step and compare to actual value).

(2) Tuning parameters for the HLSM – it might not be as simple as changing everything.  Try marginalizing what I change – change one parameter at a time and see how that affects everything else.

(3) Continue coding the HMMSBM and see if I can get some data.

(4) Once I have determined that the HLSM fits my simulated data, I will move on the Pitts & Spillane data.

Deadlines: HMMSBM paper in December 2011, NSF grant deadline is Jan 2012.