Joint Program in Survey Methodology

Permanent URI for this communityhttp://hdl.handle.net/1903/2251

Browse

Now showing 1 - 20 of 40

Adjustments for Nonresponse, Sample Quality Indicators, and Nonresponse Error in a Total Survey Error Context
(2012) Ye, Cong; Tourangeau, Roger; Survey Methodology; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
The decline in response rates in surveys of the general population is regarded by many researchers as one of the greatest threats to contemporary surveys. Much research has focused on the consequences of nonresponse. However, because the true values for the non-respondents are rarely known, it is difficult to estimate the magnitude of nonresponse bias or to develop effective methods for predicting and adjusting for nonresponse bias. This research uses two datasets that have records on each person in the frame to evaluate the effectiveness of adjustment methods aiming to correct nonresponse bias, to study indicators of sample quality, and to examine the relative magnitude of nonresponse bias under different modes. The results suggest that both response propensity weighting and GREG weighting, are not effective in reducing nonresponse bias present in the study data. There are some reductions in error, but the reductions are limited. The comparison between response propensity weighting and GREG weighting shows that with the same set of auxiliary variables, the choice between response propensity weighting and GREG weighting makes little difference. The evaluation of the R-indicators and the penalized R-indicators using the study datasets and from a simulation study suggests that the penalized R-indicators perform better than the R-indicators in terms of assessing sample quality. The penalized R-indicator shows a pattern that has a better match to the pattern for the estimated biases than the R-indicator does. Finally, the comparison of nonresponse bias to other types of errors finds that nonresponse bias in these two data sets may be larger than sampling error and coverage bias, but measurement bias can be bigger in turn than nonresponse bias, at least for sensitive questions. And postsurvey adjustments do not result in substantial reduction in the total survey error. We reach the conclusion that 1) efforts put into dealing with nonresponse bias are warranted; 2) the effectiveness of weighting adjustments for nonresponse depends on the availability and quality of the auxiliary variables, and 3) the penalized R-indicator may be more helpful in monitoring the quality of the survey than the R-indicator.
ANALYSIS OF COMPLEX SURVEY DATA USING ROBUST MODEL-BASED AND MODEL-ASSISTED METHODS
(2006-09-19) Li, Yan; Lahiri, Partha; Survey Methodology; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
Over the past few decades, major advances have taken place in both model-based and model-assisted approaches to inferences in finite population sampling. In the standard model-based approach, the finite population is assumed to be a realization from a superpopulation characterized by a probability distribution, and that the distribution of the sample is identical to that of the finite population. The model-based method could lead to a misleading inference if either assumption is violated. The model-assisted estimators typically are consistent or at least approximately unbiased with respect to the sampling design, and yet more efficient than the customary randomization-based estimators in the sense of achieving smaller variance with respect to the design if the assumed model is appropriate. Since both approaches rely on the assumed model, there is a need to achieve robustness with respect to the model selection. This is precisely the main theme of this dissertation. This study uses the well-known Box-Cox transformation on the dependent variable to generate certain robust model-based and model-assisted estimators of finite population totals. The robustness is achieved since the appropriate transformation on the dependent variable is determined by the data. Both Monte Carlo simulation study and real data analyses are conducted to illustrate the robustness properties of the proposed estimation method using two different ways: (i) design-based, and (ii) model-based, wherever appropriate. A few potential areas of future research within the context of transformations in linear regression models, as well as linear mixed models, for analysis of complex survey data are identified.
The Bayesian and Approximate Bayesian Methods in Small Area Estimation
(2008-11-20) Pramanik, Santanu; Lahiri, Partha; Survey Methodology; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
For small area estimation, model based methods are preferred to the traditional design based methods because of their ability to borrow strength from related sources. The indirect estimates, obtained using mixed models, are usually more reliable than the direct survey estimates. To draw inferences from mixed models, one can use Bayesian or frequentist approach. We consider the Bayesian approach in this dissertation. The Bayesian approach is straightforward. The prior and likelihood produce the posterior, which is used for all inferential purposes. It overcomes some of the shortcomings of the empirical Bayes approach. For example, the posterior variance automatically captures all sources of uncertainties in estimating small area parameters. But this approach requires the specification of a subjective prior on the model parameters. Moreover, in almost all situation, the posterior moments involve multi-dimensional integration and consequently closed form expressions cannot be obtained. To overcome the computational difficulties one needs to apply computer intensive MCMC methods. We apply linear mixed normal models (area level and unit level) to draw inferences for small areas when the variable of interest is continuous. We propose and evaluate a new prior distribution for the variance component. We use Laplace approximation to obtain accurate approximations to the posterior moments. The approximations present the Bayesian methodology in a transparent way, which facilitates the interpretation of the methodology to the data users. Our simulation study shows that the proposed prior yields good frequentist properties for the Bayes estimators relative to some other popular choices. This frequentist validation brings in an objective flavor to the so-called subjective Bayesian approach. The linear mixed models are, usually, not suitable for handling binary or count data, which are often encountered in surveys. To estimate the small area proportions, we propose a binomial-beta hierarchical model. Our formulation allows a regression specification and hence extends the usual exchangeable assumption at the second level. We carefully choose a prior for the shape parameter of the beta density. This new prior helps to avoid the extreme skewness present in the posterior distribution of the model parameters so that the Laplace approximation performs well.
BAYESIAN METHODS FOR PREDICTION OF SURVEY DATA COLLECTION PARAMETERS IN ADAPTIVE AND RESPONSIVE DESIGNS
(2020) Coffey, Stephanie Michelle; Elliott, Michael R; Survey Methodology; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
Adaptive and responsive survey designs rely on estimates of survey data collection parameters (SDCPs), such as response propensity, to make intervention decisions during data collection. These interventions are made with some data collection goal in mind, such as maximizing data quality for a fixed cost or minimizing costs for a fixed measure of data quality. Data quality may be defined by response rate, sample representativeness, or error in survey estimates. Therefore, the predictions of SDCPs are extremely important. Predictions within a data collection period are most commonly generated using fixed information about sample cases, and accumulating paradata and survey response data. Interventions occur during the data collection period, however, meaning they are applied based on predictions from incomplete accumulating data. There is evidence that the incomplete accumulating data can lead to biased and unstable predictions, particularly early in data collection. This dissertation explores the use of Bayesian methods to improve predictions of SDCPs during data collection, by providing a mathematical framework for combining priors, based on external data about covariates in the prediction models, with the current accumulating data to generate posterior predictions of SDCPs for use in intervention decisions.This dissertation includes three self-contained papers, each focused on the use of Bayesian methods to improve predictions of SDCPs for use in adaptive and responsive survey designs. The first paper predicts time to first contact, where priors are generated from historical survey data. The second paper implements expert elicitation, a method for prior construction when historical data is not available. The last paper describes a data collection experiment conducted using a Bayesian framework, which attempts to minimize data collection costs without reducing the quality of a key survey estimate. In all three papers, the use of Bayesian methods introduces modest improvements in the predictions of SDCPs, especially early in data collection, when interventions would have the largest effect on survey outcomes. Additionally, the experiment in the last paper resulted in significant data collection cost savings without having a significant effect on a key survey estimate. This work suggests that Bayesian methods can improve predictions of SDCPs that are critical for adaptive and responsive data collection interventions.
Beyond Response Rates: The Effect of Prepaid Incentives on Measurement Error
(2012) Medway, Rebecca; Tourangeau, Roger; Survey Methodology; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
As response rates continue to decline, survey researchers increasingly offer incentives as a way to motivate sample members to take part in their surveys. Extensive prior research demonstrates that prepaid incentives are an effective tool for doing so. If prepaid incentives influence behavior at the stage of deciding whether or not to participate, they also may alter the way that respondents behave while completing surveys. Nevertheless, most research has focused narrowly on the effect that incentives have on response rates. Survey researchers should have a better empirical basis for assessing the potential tradeoffs associated with the higher responses rates yielded by prepaid incentives. This dissertation describes the results of three studies aimed at expanding our understanding of the impact of prepaid incentives on measurement error. The first study explored the effect that a $5 prepaid cash incentive had on twelve indicators of respondent effort in a national telephone survey. The incentive led to significant reductions in item nonresponse and interview length. However, it had little effect on the other indicators, such as response order effects and responses to open-ended items. The second study evaluated the effect that a $5 prepaid cash incentive had on responses to sensitive questions in a mail survey of registered voters. The incentive resulted in a significant increase in the proportion of highly undesirable attitudes and behaviors to which respondents admitted and had no effect on responses to less sensitive items. While the incentive led to a general pattern of reduced nonresponse bias and increased measurement bias for the three voting items where administrative data was available for the full sample, these effects generally were not significant. The third study tested for measurement invariance in incentive and control group responses to four multi-item scales from three recent surveys that included prepaid incentive experiments. There was no evidence of differential item functioning; however, full metric invariance could not be established for one of the scales. Generally, these results suggest that prepaid incentives had minimal impact on measurement error. Thus, these findings should be reassuring for survey researchers considering the use of prepaid incentives to increase response rates.
Big Data and Official Statistics
(Wiley, 2022-10-02) Abraham, Katharine G.
The infrastructure and methods for developed countries' economic statistics, largely established in the mid-20th century, rest almost entirely on survey and administrative data. The increasing difficulty of obtaining survey responses threatens the sustainability of this model. Meanwhile, users of economic data are demanding ever more timely and granular information. “Big data” originally created for other purposes offer the promise of new approaches to the compilation of economic data. Drawing primarily on the U.S. experience, the paper considers the challenges to incorporating big data into the ongoing production of official economic statistics and provides examples of progress towards that goal to date. Beyond their value for the routine production of a standard set of official statistics, new sources of data create opportunities to respond more nimbly to emerging needs for information. The concluding section of the paper argues that national statistical offices should expand their mission to seize these opportunities.
Clarifying Survey Questions
(2011) Redline, Cleo D.; Tourangeau, Roger; Survey Methodology; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
Although comprehension is critical to the survey response process, much about it remains unknown. Research has shown that concepts can be clarified through the use of definitions, instructions or examples, but respondents do not necessarily attend to these clarifications. This dissertation presents the results of three experiments designed to investigate where and how to present clarifying information most effectively. In the first experiment, eight study questions, modeled after questions in major federal surveys, were administered as part of a Web survey. The results suggest that clarification improves comprehension of the questions. There is some evidence from that initial experiment that respondents anticipate the end of a question and are more likely to ignore clarification that comes after the question than before it. However, there is considerable evidence to suggest that clarifications are most effective when they are incorporated into a series of questions. A second experiment was conducted in both a Web and Interactive Voice Response (IVR) survey. IVR was chosen because it controlled for the effects of interviewers. The results of this experiment suggest that readers appear no more capable of comprehending complex clarification than listeners. In both channels, instructions were least likely to be followed when they were presented after the question, more likely to be followed when they were placed before the question, and most likely to be followed when they were incorporated into a series of questions. Finally, in a third experiment, five variables were varied to examine the use of examples in survey questions. Broad categories elicited higher reports than narrow categories and frequently consumed examples elicited higher reports than infrequently consumed examples. The implication of this final study is that the choice of categories and examples require careful consideration, as this choice will influence respondents' answers, but it does not seem to matter where and how a short list of examples are presented.
Classifying Mouse Movements and Providing Help in Web Surveys
(2013) Horwitz, Rachel; Conrad, Frederick G; Kreuter, Frauke; Survey Methodology; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
Survey administrators go to great lengths to make sure survey questions are easy to understand for a broad range of respondents. Despite these efforts, respondents do not always understand what the questions ask of them. In interviewer-administrated surveys, interviewers can pick up on cues from the respondent that suggest they do not understand or know how to answer the question and can provide assistance as their training allows. However, due to the high costs of interviewer administration, many surveys are moving towards other survey modes (at least for some respondents) that do not include costly interviewers, and with that a valuable source for clarification is gone. In Web surveys, researchers have experimented with providing real-time assistance to respondents who take a long time to answer a question. Help provided in such a fashion has resulted in increased accuracy, but some respondents do not like the imposition of unsolicited help. There may be alternative ways to provide help that can refine or overcome the limitations to using response times. This dissertation is organized into three separate studies that each use a set of independently collected data to identify a set of indicators survey administrators can use to determine when a respondent is having difficulty answering a question and proposes alternative ways of providing real-time assistance that increase accuracy as well as user satisfaction. The first study identifies nine movements that respondents make with the mouse cursor while answering survey questions and hypothesizes, using exploratory analyses, which movements are related to difficulty. The second study confirms use of these movements and uses hierarchical modeling to identify four movements which are the most predictive. The third study tests three different of providing unsolicited help to respondents: text box, audio recording, and chat. Accuracy and respondent satisfaction are evaluated for each mode. There were no differences in accuracy across the three modes, but participants reported a preference for receiving help in a standard text box. These findings allow survey designers to identify difficult questions on a larger scale than previously possible and to increase accuracy by providing real-time assistance while maintaining respondent satisfaction.
Collinearity Diagnostics for Complex Survey Data
(2010) Liao, Dan; Valliant, Richard; Survey Methodology; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
Survey data are often used to fit models. The values of covariates used in modeling are not controlled as they might be in an experiment. Thus, collinearity among the covariates is an inevitable problem in the analysis of survey data. Although many books and articles have described the collinearity problem and proposed strategies to understand, assess and handle its presence, the survey literature has not provided appropriate diagnostic tools to evaluate its impact on the regression estimation when the survey complexities are considered. The goal of this research is to extend and adapt the conventional ordinary least squares collinearity diagnostics to complex survey data when a linear model or generalized linear model is used. In this dissertation we have developed methods that generally have either a model-based or design-based interpretation. We assume that an analyst uses survey-weighted regression estimators to estimate both underlying model parameters (assuming a correctly specified model) and census-fit parameters in the finite population. Diagnostics statistics, variance inflation factors (VIFs), condition indexes and variance decomposition proportions are constructed to evaluate the impact of collinearity and determine which variables are involved. Survey weights are components of the diagnostic statistics and the estimated variances of the coefficients are obtained from design-consistent estimators which account for complex design features, e.g. clustering and stratification. Illustrations of these methods are given using data from a survey of mental health organizations and a household survey of health and nutrition. We demonstrate that specialized collinearity diagnostic statistics are needed to account for survey weights and complex finite population features that are reflected in the sample design and considered in the regression analysis.
A COMPARISON OF EX-ANTE, LABORATORY, AND FIELD METHODS FOR EVALUATING SURVEY QUESTIONS
(2014) Maitland, Aaron; Presser, Stanley; Survey Methodology; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
A diverse range of evaluation methods is available for detecting measurement error in survey questions. Ex-ante question evaluation methods are relatively inexpensive, because they do not require data collection from survey respondents. Other methods require data collection from respondents either in the laboratory or in the field setting. Research has explored how effective some of these methods are at identifying problems with respect to one another. However, a weakness of most of these studies is that they do not compare the range of question evaluation methods that are currently available to researchers. The purpose of this dissertation is to understand how the methods researchers use to evaluate survey questions influence the conclusions they draw about the questions. In addition, the dissertation seeks to identify more effective ways to use the methods together. It consists of three studies. The first study examines the extent of agreement between ex-ante and laboratory methods in identifying problems and compares the methods in how well they predict differences between questions whose validity has been estimated in record-check studies. The second study evaluates the extent to which ex-ante and laboratory methods predict the performance of questions in the field as measured by indirect assessments of data quality such as behavior coding, response latency and item nonresponse. The third study evaluates the extent to which ex-ante, laboratory, and field methods predict the reliability of answers to survey questions as measured by stability over time. The findings suggest (1) that a multiple method approach to question evaluation is the best strategy given differences in the ability to detect different types of problems between the methods and (2) how to combine methods more effectively in the future.
Design and Effectiveness of Multimodal Definitions in Online Surveys
(2020) Spiegelman, Maura; Conrad, Frederick G; Survey Methodology; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
If survey respondents do not interpret a question as it was intended, they may, in effect, answer the wrong question, increasing the chances of inaccurate data. Researchers can bring respondents’ interpretations into alignment with what is intended by defining the terms that respondents might misunderstand. This dissertation explores strategies to increase response alignment with definitions in online surveys. In particular, I compare the impact of unimodal (either spoken or textual) to multimodal (both spoken and textual) definitions on question interpretation and, indirectly, response quality. These definitions can be further categorized as conventional or optimized for the mode in which they are presented (for textual definitions, fewer words than in conventional definitions with key information made visually salient and easier for respondents to grasp; for spoken definitions, a shorter, more colloquial style of speaking). The effectiveness of conventional and optimized definitions are compared, as well as the effectiveness of unimodal and multimodal definitions. Amazon MTurk workers were randomly assigned to one of six definition conditions in a 2x3 design: conventional or optimized definitions, presented in a spoken, textual, or multimodal (both spoken and textual) format. While responses for unimodal optimized and conventional definitions were similar, multimodal definitions, and particularly multimodal optimized definitions, resulted in responses with greater alignment with definitions. Although complementary information presented in different modes can increase comprehension and lead to increased data quality, redundant or otherwise untailored multimodal information may not have the same positive effects. Even as not all respondents complied with instructions to read and/or listen to definitions, the compliance rates and effectiveness of multimodal presentation were sufficiently high to show improvements in data quality, and the effectiveness of multimodal definitions increased when only compliant observations were considered. Multimodal communication in a typically visual medium (such as web surveys) may increase the amount of time needed to complete a questionnaire, but respondents did not consider their use to be burdensome or otherwise unsatisfactory. While further techniques could be used to help increase respondent compliance with instructions, this study suggests that multimodal definitions, when thoughtfully designed, can improve data quality without negatively impacting respondents.
Effects of Acoustic Perception of Gender on Nonsampling Errors in Telephone Surveys
(2012) Kenney McCulloch, Susan; Kreuter, Frauke; Survey Methodology; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
Many telephone surveys require interviewers to observe and record respondents' gender based solely on respondents' voice. Researchers may rely on these observations to: (1) screen for study eligibility; (2) determine skip patterns; (3) foster interviewer tailoring strategies; (4) contribute to nonresponse assessment and adjustments; (5) inform post-stratification weighting; and (6) design experiments. Gender is also an important covariate to understand attitudes and behavior in many disciplines. Yet, despite this fundamental role in research, survey documentation suggests there is significant variation in how gender is measured and collected across organizations. Variations of collecting respondent gender may include: (1) asking the respondent; (2) interviewer observation only; (3) a combination of observation aided by asking when needed; or (4) another method. But what is the efficacy of these approaches? Are there predictors of observational errors? What are the consequences of interviewer misclassification of respondent gender to survey outcomes? Measurement error in interviewer's observations of respondent gender has never been examined by survey methodologists. This dissertation explores the accuracy and utility of interviewer judgments specifically with regard to gender observations. Using the recent paradata work and linguistics literature as a foundation to explore acoustic gender determination, the goal of my dissertation is to identify implications for survey research of using interviewers' observations collected in a telephone interviewing setting. Organized into three journal-style papers, through a survey of survey organizations, the first paper finds that more than two-thirds of firms collect respondent gender by some form of interviewer observation. Placement of the observation, rationale for chosen collection methods, and uses of these paradata are documented. In paper two, utilizing existing recording of survey interviews, the experimental research finds that the accuracy of interviewer observations improves with increased exposure. The noisy environment of a centralized phone room does not appear to threaten the quality of gender observations. Interviewer and respondent level covariates of misclassification are also discussed. Analyzing secondary data, the third paper finds there are some consequences of incorrect interviewer observations of respondents' gender on survey estimates. Findings from this dissertation will contribute to the paradata literature and provide survey practitioners guidance in the use and collection of interviewer observations, specifically gender, to reduce sources of nonsampling error.
Enhancing the Understanding of the Relationship between Social Integration and Nonresponse in Household Surveys
(2015) Amaya, Ashley Elaine; Presser, Stanley; Survey Methodology; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
Nonresponse and nonresponse bias remain fundamental concerns for survey researchers as understanding them is critical to producing accurate statistics. This dissertation tests the relationship between social integration, nonresponse, and nonresponse bias. Using the rich frame information available on the American Time Use Survey (ATUS) and the Survey of Health, Ageing, and Retirement in Europe (SHARE) Wave II, structural equation models were employed to create latent indicators of social integration. The resulting variables were used to predict nonresponse and its components (e.g., noncontact). In both surveys, social integration was significantly predictive of nonresponse (regardless of type of nonresponse) with integrated individuals more likely to respond. However, the relationship was driven by different components of integration across the two surveys. Full sample estimates were compared to respondent estimates on a series of 40 dichotomous and categorical variables to test the hypothesis that variables measuring social activities and roles would suffer from nonresponse bias. The impact of nonresponse on multivariate models predicting social outcomes was also evaluated. Nearly all of the 40 assessed variables suffered from significant nonresponse bias resulting in the overestimation of social activity and role participation. In general, civic and political variables suffered from higher levels of bias, but the differences were not significant. Multivariate models were not exempt; beta coefficients were frequently biased. Although, the direction was inconsistent and often small. Finally, an indicator of social integration was added to the weighting methodology with the goal of eliminating the observed nonresponse bias. While the addition significantly reduced the bias in most instances compared to both the base- and traditionally-weighted estimates, the improvements were small and did little to eliminate the bias.
Errors in Housing Unit Listing and their Effects on Survey Estimates
(2010) Eckman, Stephanie; Kreuter, Frauke; Survey Methodology; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
In the absence of a national population or housing register, field work organizations in many countries use in-field housing unit listings to create a sampling frame for in-person household surveys. Survey designers select small geographic clusters called segments and specially trained listers are sent to the segments to record the address and/or description of every housing unit. These frames are then returned to the central office where statisticians select a sample of units for the survey. The quality of these frames is critically important for the overall survey quality. A well designed and executed sample, efforts to reduce nonresponse and measurement error, and high quality data editing and analysis cannot make up for errors of undercoverage and overcoverage on the survey frame. Previous work on housing unit frame quality has focused largely on estimating net coverage rates and identifying the types of units and segments that are vulnerable to undercoverage. This dissertation advances our understanding of the listing process, using sociological and psychological theories to derive hypotheses about lister behavior and frame coverage. Two multiple-listing datasets supporttests of these hypotheses. Chapter 1 demonstrates that two well-trained and experienced listers produce different housing unit frames in the same segments. Chapter 2 considers listing as a principal-agent interaction, but finds limited support for the ability of this perspective to explain undercoverage in traditional listing. Chapter 3 has more success explaining the mechanisms of error in dependent listing. Listers tend not to correct the errors of inclusion and exclusion on the frame they update, leading to undercoverage and overcoverage. Chapter 4 tests for bias due to the observed undercoverage, but finds little evidence that lister error would lead to substantial changes in survey estimates. Housing unit listing is a complex task that deserves more research in the survey methods literature. This work fills in some of the gaps in our understanding of the listing process, but also raises many questions. The good news for survey researchers is that the listers' errors appear to be somewhat random with respect to the household and person characteristics, at least for the variables and datasets studied in this work.
Global trends and predictors of face mask usage during the COVID-19 pandemic
(Springer Nature, 2021-11-15) Badillo-Goicoechea, Elena; Chang, Ting-Hsuan; Kim, Esther; LaRocca, Sarah; Morris, Katherine; Deng, Xiaoyi; Chiu, Samantha; Bradford, Adrianne; Garcia, Andres; Kern, Christoph; Cobb, Curtiss; Kreuter, Frauke; Stuart, Elizabeth A.
Guidelines and recommendations from public health authorities related to face masks have been essential in containing the COVID-19 pandemic. We assessed the prevalence and correlates of mask usage during the pandemic. We examined a total of 13,723,810 responses to a daily cross-sectional online survey in 38 countries of people who completed from April 23, 2020 to October 31, 2020 and reported having been in public at least once during the last 7 days. The outcome was individual face mask usage in public settings, and the predictors were country fixed effects, country-level mask policy stringency, calendar time, individual sociodemographic factors, and health prevention behaviors. Associations were modeled using survey-weighted multivariable logistic regression. Mask-wearing varied over time and across the 38 countries. While some countries consistently showed high prevalence throughout, in other countries mask usage increased gradually, and a few other countries remained at low prevalence. Controlling for time and country fixed effects, sociodemographic factors (older age, female gender, education, urbanicity) and stricter mask-related policies were significantly associated with higher mask usage in public settings. Crucially, social behaviors considered risky in the context of the pandemic (going out to large events, restaurants, shopping centers, and socializing outside of the household) were associated with lower mask use. The decision to wear a face mask in public settings is significantly associated with sociodemographic factors, risky social behaviors, and mask policies. This has important implications for health prevention policies and messaging, including the potential need for more targeted policy and messaging design.
GRICEAN EFFECTS IN SELF-ADMINSTERED SURVEYS
(2005-10-31) Yan, Ting; Tourangeau, Roger; Survey Methodology; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
Despite the best efforts of questionnaire designers, survey respondents don't always interpret questions as the question writers intended. Researchers have used Grice's conversational maxims to explain some of these discrepancies. This dissertation extends this work by reviewing studies on the use of Grice's maxims by survey respondents and describing six new experiments that looked for direct evidence that respondents apply Grice's maxims. The strongest evidence for respondents' use of the maxims came from an experiment that varied the numerical labels on a rating scale; the mean shift in responses to the right side of the rating scale induced by negative numerical labels was robust across items and fonts. Process measures indicated that respondents applied the maxim of relation in interpreting the questions. Other evidence supported use of the maxim of quantity -- as predicted, correlations between two highly similar items were lower when they were asked together. Reversing the wording of one of the items didn't prevent respondents from applying the maxim of quantity. Evidence was weaker for the application of Grice's maxim of manner; respondents still seemed to use definitions (as was apparent from the reduced variation in their answers), even though the definitions were designed to be uninformative. That direct questions without filters induced significantly more responses on the upper end of the scale -- presumably because of the presuppositions direct questions carried -- supported respondents' application of the maxim of quality. There was little support for respondents' use of the maxim of relation from an experiment on the physical layout of survey questions; the three different layouts didn't influence how respondents perceived the relation among items. These results provided some evidence that both survey "satisficers" and survey "optimizers" may draw automatic inferences based on Gricean maxims, but that only "optimizers" will carry out the more controlled processes requiring extra effort. Practical implications for survey practice include the need for continued attention to secondary features of survey questions in addition to traditional questionnaire development issues. Additional experiments that incorporate other techniques such as eye tracking or cognitive interviews may help to uncover other subtle mechanisms affecting survey responses.
HIERARCHICAL BAYES ESTIMATION AND EMPIRICAL BEST PREDICTION OF SMALL-AREA PROPORTIONS
(2009) Liu, Benmei; Lahiri, Partha; Survey Methodology; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
Estimating proportions of units with a given characteristic for small areas using small area estimation (SAE) techniques is a common problem in survey research. The direct survey estimates, usually based on area-specific sample data, are very imprecise or even unavailable due to the small or zero sample sizes in the areas. In order to provide precise estimates, a variety of model-dependent techniques, using Bayesian and frequentist approaches, have been developed. Among those, empirical best prediction (EBP) and hierarchical Bayes (HB) methods relying on mixed models have been considered for estimating small area proportions. Mixed models can be broadly classified as area or unit level models in SAE. When an area level model is used to produce estimates of proportions for small areas, it is commonly assumed that the survey weighted proportion for each sampled small area has a normal distribution and that the sampling variance of this proportion is known. However, these assumptions are problematic when the small area sample size is small or when the true proportion is near 0 or 1. In addition, normality is commonly assumed for the random effects in area level and unit level mixed models. However, this assumption maybe violated for some cases. To address those issues, in this dissertation, we first explore some alternatives to the well-known Fay-Herriot area level model. The aim is to consider models that are appropriate for survey-weighted proportions and can capture different sources of uncertainty, including the uncertainty that arises from the estimation of the sampling variances of the design-based estimators. Then we develop an adaptive HB method for SAE using data from a simple stratified design. The main goal is to relax the usual normality assumption for the random effects and instead determine the distribution of the random effects adaptively from the survey data. The Jiang-Lahiri type frequentist's alternative to the hierarchical Bayesian methods is also developed. Finally we propose a generalized linear mixed model that is suitable for binary data collected from a two-stage sampling design.
Improving External Validity of Epidemiologic Analyses by Incorporating Data from Population-Based Surveys
(2020) Wang, Lingxiao; Li, Yan; Survey Methodology; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
Many epidemiologic studies forgo probability sampling and turn to volunteer-based samples because of cost, confidentiality, response burden, and invasiveness of biological samples. However, the volunteers may not represent the underlying target population mainly due to self-selection bias. Therefore, standard epidemiologic analyses may not be generalizable to the target population, which is called lack of “external validity.” In survey research, propensity score (PS)-based approaches have been developed to improve representativeness of the nonprobability samples by using population-based surveys as references. These approaches create a set of “pseudo-weights” to weight the nonprobability sample up to the target population. There are two main types of PS-based approaches: (1) PS-based weighting methods using PSs to estimate participation rates of the nonprobability sample; for example, the inverse of PS weighting (IPSW); (2) PS-based matching methods using PSs to measure similarity between the units in the nonprobability sample and the reference survey sample, such as PS adjustment by subclassification (PSAS). Although the PS-based weighting methods reduce the bias, they are sensitive to propensity model misspecification and can be inefficient. The PS-based matching methods are more robust to the propensity model misspecification and can avoid extreme weights. However, matching methods such as PSAS are less effective at bias reduction. This dissertation proposes a novel PS-based matching method, named the kernel weighting (KW) approach, to improve the external validity of epidemiologic analyses that gain a better bias–variance tradeoff. A unifying framework is established for PS-based methods to provide three advances. First, the KW method is proved to provide consistent estimates, yet generally has smaller mean-square error than the IPSW. Second, the framework reveals a fundamental strong exchangeability assumption (SEA) underlying the existing PS-based matching methods that has previously been unknown. The SEA is relaxed to a weak exchangeability assumption that is more realistic for data analysis. Third, survey weights are scaled in propensity estimation to reduce the variance of the estimated PS and improve efficiency of all PS-based methods under the framework. The performance of the proposed PS-based methods is evaluated for estimating prevalence of diseases and associations between risk factors and disease in the finite population.
INVESTIGATION OF ALTERNATIVE CALIBRATION ESTIMATORS IN THE PRESENCE OF NONRESPONSE
(2017) Han, Daifeng; Valliant, Richard; Survey Methodology; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
Calibration weighting is widely used to decrease variance, reduce nonresponse bias, and improve the face validity of survey estimates. In the purely sampling context, Deville and Särndal (1992) demonstrate that many alternative forms of calibration weighting are asymptotically equivalent, so for variance estimation purposes, the generalized regression (GREG) estimator can be used to approximate some general calibration estimators with no closed-form solutions such as raking. It is unclear whether this conclusion holds when nonresponse exists and single-step calibration weighting is used to reduce nonresponse bias (i.e., calibration is applied to the basic sampling weights directly without a separate nonresponse adjustment step). In this dissertation, we first examine whether alternative calibration estimators may perform differently in the presence of nonresponse. More specifically, properties of three widely used calibration estimations, the GREG with only main effect covariates (GREG_Main), poststratification, and raking, are evaluated. In practice, the choice between poststratification and raking are often based on sample sizes and availability of external data. Also, the raking variance is often approximated by a linear substitute containing residuals from a GREG_Main model. Our theoretical development and simulation work demonstrate that with nonresponse, poststratification, GREG_Main, and raking may perform differently and survey practitioners should examine both the outcome model and the response pattern when choosing between these estimators. Then we propose a distance measure that can be estimated for raking or GREG_Main from a given sample. Our analytical work shows that the distance measure follows a Chi-square probability distribution when raking or GREG_Main is unbiased. A large distance measure is a warning sign of potential bias and poor confidence interval coverage for some variables in a survey due to omitting a significant interaction term in the calibration process. Finally, we examine several alternative variance estimators for raking with nonresponse. Our simulation results show that when raking is model-biased, none of the linearization variance estimators under evaluation is unbiased. In contrast, the jackknife replication method performs well in variance estimation, although the confidence interval may still be centered in the wrong place if the point estimate is inaccurate.
Model-Assisted Estimators for Time-to-Event Data
(2017) Reist, Benjamin Martin; Valliant, Richard; Survey Methodology; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
In this dissertation, I develop model-assisted estimators for estimating the proportion of a population that experienced some event by time t. I provide the theoretical justification for the new estimators using time-to-event models as the underlying framework. Using simulation, I compared these estimators to traditional methods, then I applied the estimators to a study of nurses’ health, where I estimated the proportion of the population that had died after a certain period of time. The new estimators performed as well if not better than existing methods. Finally, as this work assumes that all units are censored at the same point in time, I propose an extension that allows units censoring time to vary.