ah teaches interface design (Surveying the Land lecture)

Surveying the Land

Lecture outline

Surveying users and deriving meaning from research results.

Lecture recording available for viewing at https://334.ah1.ca/lectures/08/video/.


An overview

An efficient way to collect a wide range of opinions from a large sample.


  • Inexpensive to run
  • Large sample


  • Time consuming to prepare
  • Unpredictable response rate
  • Possibly low quality of response

Preparing surveys

Putting together a good survey requires effective consideration of a number of aspects:

  • Purpose
  • Participants
  • Questions
  • Running the survey


With the survey you will want to define what is the goal, and how are you going to achieve it?

Types of surveys

We can take different approaches to structuring our surveys to achieve different results.


  • Start with open-ended questions
  • End with close-ended questions
  • Good for sensitive topics

Starting with a really sensitive question can be awkward for participants.

Reverse funnel

  • Start with close-ended questions
  • End with open-ended questions
  • Good for non-sensitive topics

Survey formats

Each method of delivering a survey comes with its own advantages/disadvantages:

  • In person requires your time, but offers the best completion rate.
  • Take home does not require your time, but means a potential lack of completion.
  • Email has less time constraints for respondents but response rates will vary.
  • Web-based means you can build in scripts to ensure completeness.

Participants and bias

Much like with interviewing, we need to be cautious of sampling bias — when our 'sample' of participants is not sufficiently representative of those we are designing for.

Types of questions

When designing survey questions we have flexibility in the types of questions we can ask.

Consider as part of this:

  • What questions will you use the answer to?
  • How are you going to analyze the results?

Types of questions

Open and closed

Much like in interviews, open or close-ended questions can be used:


  • Can help us understand opinions
  • Collects subjective responses
  • More difficult to analyze

For example: "How would you improve the interface?"


  • Can help us see patterns
  • Supplies a set of answers
  • Easier to analyze, but loses details

For example: "How understandable do you find the interface?"

(not at all)

  • 1
  • 2
  • 3
  • 4
  • 5

(entirely understandable)

Types of questions

Ranked order

In this style of question we ask participants to rank up to ten items to understand their preferences. For example:

Rank the order of importance of the following (1 = most important, 3 = least important)

  • _____ Color of buttons
  • _____ Location of buttons
  • _____ Size of buttons

Types of questions


These are meant to be 'checks' to ensure that the participant is paying attention and not just rushing answers.

For example: What is the fifth letter in the name of Canada's capital city, Ottawa?

Types of questions


In this style of question we are measuring opinions, attitudes and beliefs using a 5 or 7 point scale. For example:

I like the presentation of interface controls

Strongly agree

  1. 1
  2. 2
  3. 3
  4. 4
  5. 5

Strongly disagree

Types of questions

Multiple choice

Here we can collect very explicit answers but may be unable to consider cases that do not match our set. For example:

Where is the main computer in your home? (select one)

  • Living room
  • Office
  • Bedroom
  • Kitchen

Running the survey

Before running the survey make sure to test the wording, timing and analysis with someone not affiliated with your project.

Running this test as a speak-aloud may help you see potentials for error.

Non-response bias

It can be difficult to get people to respond or complete a survey. To combat 'non-response' bias, consider:

  • Do you have enough representation from your audience in the responses?
  • When would delivering the survey makes the most sense?
  • Keeping the survey short
  • Trying other methods of distributing the survey
  • Is anonymity important in the survey responses?


We are going to take a look at two examples of analysis: A ranked order quantitative example, and a survey response qualitative example.

A collection of data in Excel showing the results of a ranked order question about features for a pre-recorded lecture
Affinity diagram sample
P3: Interface Proposal
Sketching exercise
Reading reflection

For lecture...

Back to critiquing your sketches!

Next week's lecture


How we go about prototyping testable interfaces and interactions. Pre-recorded lectures and slides will typically become available the day of the lecture.