Forum: open-discussion

RE: NLSY sample weights [ Reply ] By: Will Beasley on 2013-11-22 05:16	[forum:40093]
(I'm posting this mostly for the sake of my memory in the future. ) If I understand correctly, John and Mike discussed this privately and worked out whatever the immediate issue were. Thanks for contacting us with the question, John.

RE: NLSY sample weights [ Reply ]
By: Joe Rodgers on 2013-11-01 16:12

[forum:39977]

Thanks for your interest, John. These are good questions, and some don’t have clear answers. We would be glad to have a phone call about this, and our responses might be better if we asked you some questions before then.

- Is your primary goal to generalize to the US population? If not, if internal validity is your entire goal, then weights are of course irrelevant. But assuming that you are at least partially concerned with external validity ...

- Then we need to know the level of your analysis. If it's all or primarily between-family, then the weights can be useful -- but you are apparently interested in an ACE decomposition. So ...

- What has been done in the past is lots of things, none seem to me to be either entirely correct, but most are defensible in some sense of another. I think the problems of using weights is much more complex than most realize -- which is why the majority of researchers using the NLSY -- including for biometrical purposes – just punt on using weights. Not only are there different weights for each sibling in each family -- there are different weights for each sibling at each time. Given that we often use longitudinal data -- and they may be for 1996-2002 for one sibling, and 1992-1998 for another -- then there are not only multiple sibs, there are multiple time points, and each has its own sampling weight reflecting design changes, natural attrition, etc. And then note that the weights actually derive from the NLSY79 for the NLSY-Children, and that imposes another level and set of assumptions on the analysis. I've known people who have averaged weights, as you imply in the first post. I've known people who used weights in descriptive analysis -- e.g., tables of correlations and means/stdevs, and then portrayed both weighted and unweighted statistics to give the reviewer/reader a sense of how much difference it makes -- but then ran the models without weights. Finally, I've seen (and a number of times actually conducted) analyses using several different versions of weighting -- including no weighting -- and I have yet to find an analysis where the results were substantially different at the interpretational level.

- That last statement is my "take home message," and is the reason that these days I often don't even report weighted results -- though I usually run them, using one or more seemingly reasonable approach -- and there's sometimes (but not always) a footnote in my articles indicating that.

If you are still interested in discussing this by phone, please email your number to Mike Hunter (mhunter@ou.edu). Mike may not have time to discuss weights for two weeks or so. As Will has mentioned, we’ve recently finished a new version of the links, and we’re completing the accompanying documentation and vignettes (including a SAS vignette, and the cross generational vignette example, like you requested in a different thread).

Thanks, Joe Rodgers

NLSY sample weights [ Reply ]
By: John Fuerst on 2013-10-25 23:40

[forum:39953]

Hello,

I was wondering how I should deal with sample weights. I emailed around and some advised not to use them at all because siblings were not sampled within families in any of the NLSY samples. Some, though, argued that I should weight (to get a semi-representative kinship sample); this led to the question of how I might go about doing this. Of those who argued for weighting, one individual suggested that I average the weights per kin pair and then run a weighted ACE analysis in the sense that a pair with a weight of 1 would count as 1 pair. Another fellow suggested that I create weights by fitting a mixed model to get estimates and then bootstrap resample using the household sampling weights and, after, create average per kin pair weights. The easiest way, of course, would be to not weight, but I would like to make sure that this is the best way given the data available. Hence, I inquire. Of course, if weighting is preferable, then the question arises: How might I apply weights in Links?

Thanks for the help.

John