Survey Design

In this section, we do not intend to fully expand on the best practices of creating questions. There are already excellent resources available such as those below.

This overview is a starting point and focuses on issues specifically relevant to the PIEL Survey app.

Question Quality

The questions should be clear and reasonably brief.

This is good research practice but is also important with the PIEL Survey app. The app is designed with large fonts so that users can easily read it when interrupted to do a survey. The easier it is to read the survey without finding glasses or light source, the higher the completion rate is likely to be.

Ideally the question and responses should fit on one page without the need for scrolling.

Number of Responses

This point is relevant only for for list questions and checkbox questions.

Responses need to be comprehensive. This is not only good research practice, it is built into the PIEL Survey App. A participant must respond to each question to be able to navigate to the next question.

As with the number of questions, the PIEL Survey app does not have a limit on the length and number of responses. Ideally the response text should be brief and be visible without scrolling. This is easy with "Yes/No" questions but with broader questions, a limit of 5-7 response options is common.

If the potential list of responses is too large you have 2 choices.

  • Add a choice such as "Other" followed by branching logic that presents a text question asking for more information.
  • Ask an initial question using broad categories then use branching logic to ask for sub-categories.

One Question at a time

The PIEL Survey app is designed in a way that it is not possible to add sub-questions or related questions on one page. A paper or web based survey can have many questions on one page.

This forces the researcher to split questions. For example a matrix "table-like" question will need to be spit into separate questions, even though they are similar.

Number of Questions

The PIEL Survey app does not have a limit on the number of questions, the limit is really a research decision. Too many questions will reduce response rate and data quality.

Your survey should be focused on the research question. If the survey becomes long, it is likely that the quality of the responses will drop off, both during the survey and as the survey is repeated multiple times.

In most cases, a very long survey can be dramatically reduced in length for a participant if good use is made of branching logic. This way, you can ensure that only relevant questions are asked.

The PIEL Survey app shows a progress bar as a motivator for the participant. However, if the survey is too long, this will not show rapid progress.

Schedule

If you are conducting a scheduled survey, you should take great care not to run it too often or a times that are too intrusive.

If you annoy your participant, they may respond by disabling notifications or even worse, deleting the app.

You could make the survey more interesting and relevant by adapting the question flow to a participant's situation. For example, an early question could be: "Have you finished work?" then use branching logic to ask relevant questions.

Test it

We highly recommend that you test your survey with colleagues and even do a pilot study.