Gilens and a small army of research assistants29 gathered data on a large, diverse set of policy cases: 1,779 instances between 1981 and 2002 in which a national survey of the general public asked a favor/oppose question about a proposed policy change. A total of 1,923 cases met four criteria: dichotomous pro/con responses, specificity about policy, relevance to federal government decisions, and categorical rather than conditional phrasing. Of those 1,923 original cases, 1,779 cases also met the criteria of providing income breakdowns for respondents, not involving a Constitutional amendment or a Supreme Court ruling (which might entail a quite different policy making process), and involving a clear, as opposed to partial or ambiguous, actual presence or absence of policy change. These 1,779 cases do not constitute a sample from the universe of all possible policy alternatives (this is hardly conceivable), but we see them as particularly relevant to assessing the public’s influence on policy. The included policies are not restricted to the narrow Washington “policy agenda.” At the same time – since they were seen as worth asking poll questions about – they tend to concern matters of relatively high salience, about which it is plausible that average citizens may have real opinions and may exert some political influence.30
...
Gilens, using a modified version of this simple count of the number of “powerful” interest groups favoring (minus those opposing) each proposed policy change, developed a measure of Net Interest Group Alignment. To the set of groups on the “Power 25” lists (which seemed to neglect certain major business interests) he added ten key industries that had reported the highest lobbying expenditures. (For the final list of included industries and interest groups, see Appendix 1.) For each of the 1,779 instances of proposed policy change, Gilens and his assistants drew upon multiple sources to code all engaged interest groups as “strongly favorable,” “somewhat favorable,” “somewhat unfavorable,” or “strongly unfavorable” to the change. He then combined the numbers of groups on each side of a given issue, weighting “somewhat” favorable or somewhat unfavorable positions at half the magnitude of “strongly” favorable or strongly unfavorable positions. In order to allow for the likelihood of diminishing returns as the net number of groups on a given side increases (an increase from 10 to 11 groups likely matters less than a jump from 1 to 2 does), he took the logarithms of the number of pro groups and the number of con groups before subtracting. Thus:
Net Interest Group Alignment = ln(# Strongly Favor + [0.5 * # Somewhat Favor] + 1) - ln(# Strongly Oppose + [0.5 * # Somewhat Oppose] + 1).38
...
Our dependent variable is a measure of whether or not the policy change proposed in
each survey question was actually adopted, within four years after the question was asked. (It turns out that most of the action occurred within two years). Of course there was nothing easy about measuring the presence or absence of policy change for each of 1,779 different cases; Gilens and his research assistants spent many hours poring over news accounts, government data, Congressional Quarterly publications, academic papers and the like.39