Stretched thin: asking too much of an establishment survey

Surveys are an important tool to learn about population characteristics in the social sciences. To make them reliable, survey institutes and methodologist put a lot of effort into ensuring their representativity. Still, every survey has its limits. Nonetheless, applied researchers often leave the safe waters of pre-defined sampling and weighting schemes to study very specific subpopulations or to analyze the data at a level of detail that was not intended by the creators of the survey. In this paper, I use the IAB Establishment Panel, a prominent and widely used German establishment survey, to show the consequences of pushing a survey beyond its limits. Despite not being representative at fine regional or sectoral scales, the survey is often used like that in applied research. I show that the approximation error from using aggregated survey responses can be massive and that using such aggregates can lead to very different conclusions than the ‘true’ population data in plausible empirical applications. C43,C80