Subject: IFDA meeting note 04/05/2021
From: Lanyi Xu <[email-masked]>
To: Brian Junker <[email-masked]>
Cc: Yanxi Zhou <[email-masked]>, Malik Khan <[email-masked]>, Xiaofan Zhu <[email-masked]>, Zixia Luan <[email-masked]>
Date: Mon, 5 Apr 2021 16:14:20 -0400

Dear Professor,
Thank you for spending time with us and giving us suggestions on using statistical method to pick important variables! The meeting note today is attached below.
Hope you have a good day!

Best,
Lanyi 

Update on meeting with Annika
Master File
This week: work with data we have (merging & combining data and analysis across company)
Merging: figure out a set of values we can use to compare across companies 
Writing report & presentation: Be CONSISTENT with words
Think about how to link reason for termination to applicant pool
Be specific on level of importance for variables (how to determine importance):
Have a statistical question first
How can I predict whether someone will be hired (logistic ) (Merged Data)
Among the people who are hired, how can I predict who will last for 6 months (logistic ) (Merged Data)
Among the people who are hired, how can I predict how long employees will stay (logistic/ poisson) (Merged Data)
How can I predict who will get promoted (Individual Data)
Among people whose jobs were terminated, how can I predict whether this person left the company voluntarily / involuntarily (Saladions)
Second question as example: Variable selection, scatterplot matrix, boxplot, may use Lasso to encourage some of the coefficients to be zero. 
Which group of applicants get hired: (unsupervised) clustering problem, want to figure out which one matters/ add noise (PCA might be more useful for clustering than prediction, might have difficulty with interpretation)
Last question as example: If you build a model and show that it is helpful, incentive for other company that they need to collect the same data (Saladinos has reason for termination)
When thinking about questions, important to consult with Annika about what questions are important (Even if she asks questions that we can’t answer, it is still good because we will get respective insight and have something we can write about in our report)
Define enough data: training set usually larger than test test, if work with regression models: need enough observations that at least some of the models look they are doing reasonable predictions (with-in sample confusion matrix, variance, other strong evidence that the model has strong prediction power) - at least 10 observations per parameter for largest model built 
Explore differences between analysis when include/exclude 2020
Two dataset, one with 2020 and one without, separately analyze two dataset and see differences (strongly recommended)
In the larger dataset that includes 2020, include a dummy variable (1 if observation comes from 2020, 0 if not), see interaction. 
This Friday: will show Annika the master file
Next week: focus on data we think important but lacks