Surveys

In Richmond, California, a community with a large oil refinery, community organizers needed to generate knowledge about how a range of residents had experienced the effects of the refinery, what they believed about its benefits and harms, and how they felt about the possibility of the refinery closing in the future. 

To do this, they created a dozen questions with multiple choice and short answer responses, put the questions into an online and print format, and carried out a survey of several hundred residents through in-person encounters, social media posts, emails, and word of mouth. They then held a community event to present the findings back to the community and plan actions to advance environmental justice and for a just transition.51

What Type of Knowledge Surveys Generate

Surveys generate knowledge about how widely felt something is and what portions of a community have specific experiences or beliefs. Surveys are useful for building knowledge grounded in the experience of people directly impacted by an issue and showing with numbers how widespread an experience is. Too often in community advocacy, a person’s testimony is disregarded by people in positions of power who argue that the community member’s experience is just one anecdote and doesn’t reflect a wider problem worthy of public action. In the example above, individual testimonies about the impact of the refinery are often ignored because the refinery is perceived as benefiting a “silent majority” through providing jobs and charitable grants. A survey can speak back to this perception of a silent majority.

Surveys allow you to see who else in the community shares the experience and create evidence showing it is not just one person’s view. A group of formerly incarcerated residents was faced with this challenge as they advocated for public investments in housing, transitional employment, and other resources for people coming home from incarceration. The Safe Return Project designed a survey to document the challenges facing people recently released and found that 78% were unemployed, among other alarming findings.52

In another example, organizers were hearing complaints of restaurant workers being unpaid in San Francisco, something notoriously difficult to document and prove. Organizers worked with restaurant workers and academics to create a PAR process led and administered by the workers, reaching over four hundred workers and leading to workers winning back pay.53

At the height of stop-and-frisk policies in New York City, a PAR collective in the South Bronx formed to understand how invasive—and (un)successful—the policy was in preventing violent crime. The group of residents, academics, and organizers developed a survey and administered it to over one thousand residents of blocks most impacted by the policy. The collective designed a “back pocket report” that contained high-level findings and could be handed out easily. Their data was part of the legal challenge to the policy and helped to shift the narrative that the policy itself created a sense of unsafety rather than the residents of the area.54

How Surveys Can Build Relationships and People Power

Surveys give you the chance to have many conversations with the people whose experience you want to document and amplify. This is an opportunity for a new relationship, but survey interviews are highly structured, so they aren’t the best way to build a deeply felt connection. Surveys rely on the survey questions being asked the same way each time without the survey interviewer saying how they feel about the questions. In a written survey, the answerer may not have any opportunity to build connections at all. 

Surveys allow you to ask questions that focus on a deeply felt and otherwise silenced experience or perspective. For instance, the Safe Return Project team asked this question to gather information about people’s experience getting support before their transition back to the community:

Thinking about the information or services you received prior to your release (from incarceration), how would you rate them on a scale of 1 to 10, where 10 is extremely helpful and 1 is not helpful at all for your transition back to the community.

CIRCLE ONE: 1  2  3  4  5 6  7  8  9  10

Internalized oppression and isolation can lead to a belief that the harm and challenges a person faces are unique to their individual situation and caused by their actions alone. A survey—and especially a group-based process of analysis—can show that many people also have had a particular experience, affirming that people who have had the experience are not alone and busting the myth that their struggle is caused by their own individual actions.

Time, Capacity and Resources, and Tools Needed

Surveys seem simple but typically take a lot more time than expected (even with Google Forms). Really, they do! There are several steps in a survey research process:

1. Survey questionnaire design: It is worth taking time to do several drafts and try out your survey questionnaire before you start using it. Once you start conducting a survey, you can’t go back and change survey questions without compromising the responses you’ve gathered. If you are doing research to support advocacy, it is especially important that all respondents are asked the same questions, in the same way, to ensure consistency in your findings because your opposition will try to cast doubt on your survey methods. 
2. Conducting the survey: Participatory surveys have the strength that the people carrying out the survey have lived experiences and identities similar to the people being surveyed. This can help establish trust and openness. Conducting effective surveys requires strong skills in listening, following a protocol about how questions are asked, taking notes, and keeping records of the responses. Training by doing dry-run practice activities is valuable.
3. Processing the data: Once you have survey responses gathered, you need to put them all into a spreadsheet to be able to analyze them. Care must be taken not to mistakenly enter the wrong information or delete information from the spreadsheet. If you use an online survey tool for the survey, this step is easier because it is automated. It’s a good idea to save a copy of the raw data, unedited, just in case something goes wrong in the analysis and some data gets erased. 
4. Analyzing the data: There is a joke that you can make any number say what you want it to say—the same is true for survey data. Analyzing data in a group can create opportunities to build meaning and cross-check different interpretations of your findings. It may also provide ideas for advocacy, narrative efforts, or future survey iterations. At a minimum, you should plan for more than one review of the data by multiple people.

More detail on each of these steps is available in Related Resources at the end of this section.

Online survey tools: Tools such as Survey Monkey and Google Forms can be great for ease of processing and analysis, for sharing information digitally, and even for surveys administered in person, with your team entering responses as they survey someone. Increasingly, QR codes are a great way for people to answer surveys on their phone wherever they are. Make sure to create and test your draft in a separate document before transferring it to your survey tool for ease of updating and sharing. In general, the free options can get you pretty far (you can use image questions in Google Forms for example, or create a spreadsheet or graphs quickly while analyzing). But depending on the number of your survey takers and the back-end analysis options you want, you should decide early in your process if you want to work with a paid tool like Survey Monkey.

Pitfalls, Challenges, and Myths

Underestimating: Surveys are more time consuming (and often more expensive) to develop, administer, and analyze than most people expect. It can take many iterations to get your questions right through testing and editing, hours and hours to administer it (especially if it’s an in-person survey), and if you have a lot of responses, it takes time to go through them. 

Identifying how many people to survey: Your team should develop a clear idea of how many takers you will need to give your findings the oomph needed to impact change. For example, impacting a city-wide policy that affects all residents will need hundreds if not a thousand respondents who represent what the city looks like. If you’re working on something specific to a portion of a community to inform the next steps in your campaign or organizing, you may only need a dozen or more. If you can look at comparable, successful surveys to the one you are developing, this can help give a sense of the numbers needed. 

Not clearly identifying who you want to take your survey: Say you are doing a survey about community support or lack of support for policing, but you only survey young people in the neighborhood. Unless you are specifically looking to understand young people’s perspectives, your opposition will poke holes in your survey saying you ignored adults and seniors. Designing a way to have a “representative” set of survey takers (for example, mirroring the demographic data of the population who takes your survey) can strengthen the “validity” and power of your survey.

Weaving in Cultural Strategy

Surveys represent one of the most impersonal research tools, but this can be shifted in several ways, adding both rigor and resonance to the process. In designing surveys, authors often take on a certain “voice” and tone. Paying attention to this in how your group writes the questions and answers can create resonance for survey takers while also paying attention to nuance in language and meaning that might create misinterpretations. And while surveys are often designed to give us a quick overview and quantitative data through scales or multiple-choice questions, you can also design in spaces in the “margins” for people to add in their own notes or drawings. Questions also don’t have to only be written: survey makers can use photos or visuals that they take or make themselves, questions that use metaphors, or even audio. 

Another key part of the process is creating an interpretation and analysis process that welcomes people in and allows for trusting dialogue around this. Discussing in groups (projecting results or printing out multiple copies) is a simple way to open up understanding and build connections. Finally, it is important to have accessible and rapid ways to share back findings, even if they are in process. The Morris Justice Project, a PAR collective in the South Bronx, did this periodically throughout their process by hosting “sidewalk science” installations, putting up data and results in the areas where the data was collected, and sharing back survey findings in a report that could be folded to fit in your pocket.

Related Resources

• How to Conduct Your Own Survey, by Priscilla Salant and Don A. Dillman, is a book with practical guidance on designing and carrying out a survey. 
• “Power to Our People, Participatory Research Kit: Creating Surveys,” by DataCenter, is a kit for organizers that gives criteria for deciding whether a survey will be useful given your goals, time, and people-power capacity. It is geared toward participatory research and centers impacted people as the drivers of the research process.
• “Community Toolbox: Conducting Surveys,” by the Center for Community Health and Development, describes techniques for designing and carrying out a survey.