Subject: Re: introducing Nesra, special faculty in HCI (TIME-SENSITIVE)
From: Brian Junker <[email-masked]>
To: Nesra Yannier <[email-masked]>, "Ann B. Lee" <[email-masked]>
Cc: Kenneth Koedinger <[email-masked]>, Joel Greenhouse <[email-masked]>
Date: Sun, 29 Aug 2021 21:23:57 -0400

A quick answer may be the "optimal design (od)" software spearheaded by 
Steve Raudenbush.  I think the current web link is:

http://wtgrantfoundation.org/resource/optimal-design-with-empirical-information-od

this does power calculations for balanced cluster-randomized trials 
(among other things).  I haven't used the "od" software for several 
years, but I recall it being a good starting point for such 
calculations.  I probably also have an older copy of this (free) 
software lying around somewhere if this or a similar link is not helpful.

If the design is significantly imbalanced or otherwise deviates from 
what the "od" software can deal with, then rolling your own 
simulation-based power calculations may be the most direct approach; but 
probably not quick enough if your proposal is due in a week.

This article

https://www.stats.ox.ac.uk/~snijders/PowerSampleSizeMultilevel.pdf

has a brief discussion as well as an alternate link to "od" and to some 
other software that might handle at least some imbalanced cases (see 
especially PinT, I think).

I'd be happy to chat briefly later this week (thu or fri) to clarify 
anything above that seems muddy, though it's been a while since I've 
done such calculations myself.

best,

-BJ

On 8/29/2021 5:15 PM, Nesra Yannier wrote:
> Thanks for the introduction Ann!
>
> Dear Brian,
>
> As Ann mentioned we (my colleague Ken Koedinger cc'ed and I) are 
> working on an IES grant and a specific statistics question came up 
> that we weren't sure about (details explained below). Would it be 
> possible to take 15-20 minutes of your time in the next few days to 
> talk about it? We are on a tight timeline as the grant is due soon so 
> any help would be greatly appreciated.
>
> Thanks a lot in advance!
>
> Best,
> Nesra
>
> Nesra Yannier, PhD
>
> Special Faculty/Systems Scientist, Carnegie Mellon University
>
> Founder, NoRILLA
>
> _www.norilla.com <http://www.norilla.com/>_
>
> _https://twitter.com/norillalearning 
> <https://twitter.com/norillalearning>_
>
> EarthShaking Learning Fosters Curiosity | Nesra Yannier | TEDx 
> <https://www.ted.com/talks/nesra_yannier_earthshaking_learning_fosters_curiosity>
>
> ᐧ
>
> On Sun, Aug 29, 2021 at 2:18 PM Ann B. Lee <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     Hi Brian and Joel,
>
>     Let me introduce you to Nesra Yannier, a family friend and special
>     faculty in HCI at CMU.
>
>     Nesra is applying for an IES Department of Education grant, and a
>     question specific to statistic came up. This is what Nesra wrote:
>
>     > It is about doing sample size calculations (with power analysis)
>     for cluster randomized experiments. We had suggested doing a pilot
>     field study as our last study comparing a classroom using our
>     system versus another one doing business as usual, and they
>     suggested doing a power analysis to figure out how many classes
>     would be needed. We're not sure exactly how these types of cluster
>     power analyses are done so thought it might be good to talk to you
>     and see if you had any experience about it.
>     > We do have a tight timeline (it's due in a week) so any help or
>     pointer would be greatly appreciated!
>
>     I know that both of you have plenty of experience with designing
>     similar studies in education and social sciences. Would you be
>     able to help out here? Nesra is cc:d here so that you can follow
>     up directly.
>
>     Thanks much!
>
>     Ann 
>

-- 
Brian Junker                    (412) 268 - 2718
Department of Statistics        [email-masked]
232 Baker Hall                  FAX: (412) CMU-STAT
Carnegie Mellon University        or (412) 268-7828
Pittsburgh PA 15213 USA

WWW: http://www.stat.cmu.edu/~brian/