P2: Heuristics (due June 19)
Introduction
Assessing user interfaces using heuristics enables you to see issues or items in your own designs that you may not otherwise find. Heuristic evaluations tend to be a more efficient way of locating errors.
In this project you will be using heuristic evaluations to assess the usability of the Canvas conversations feature. You may evaluate other interfaces or use alternative frameworks with permission of Andrew.
This project is completed individually.
P2: Heuristics is worth 20% of your final grade.
Weekly instructions
This project spans multiple weeks. Please read the weekly instructions carefully.
From May 29 to June 5
This week is for planning your heuristic evaluation.
Please complete the following:
- Define two tasks for your heuristic evaluation. These are the tasks that those completing the evaluation will perform. Make sure the tasks are:
- Important or typical tasks a user might do — for example: "The user is going to archive a collection of old conversations."
- The task should be realistic.
- Allow for enough complexity or length in the task that your heuristic evaluation can be thorough.
- Select which heuristics are relevant. Working with Nielsen's heuristics and your tasks, determine which heuristics are relevant. Each task requires four heuristics be evaluated.
- Prepare a data sheet for collecting your evaluations. An example is given in the table below. You may pursue a different structure if you prefer but it will need to clearly communicate the same pieces.
| Task and location in task | What heuristic was broken? | Description of usability problem | Visual example | Severity rating | Recommendations |
|---|---|---|---|---|---|
| Name or description of task and where/when heuristic is broken. | Name of heuristic broken. | Description of the problem that occurred. | Insert a screenshot (or screenshots) of the problem in action. | Assess the severity of the usability problem using the severity ratings for usability problems. | Make a recommendation based on the problem and severity. |
Bring to your June 5 class
A copy of the following:
- Two clearly defined tasks for evaluation.
- A set of eight selected heuristics (4 per task).
- An empty data sheet.
We will chat about your deliverables in-class.
From June 5 to June 12
This week is for completing the heuristic evaluation.
- Working with your two tasks, four selected heuristics (per task), and data sheet complete the heuristic evaluation. Find at least eight issues. An example of filling out the data sheet is provided in the table below.
| Task and location in task | What heuristic was broken? | Description of usability problem | Visual example | Severity rating | Recommendations |
|---|---|---|---|---|---|
| "Archiving a collection of old conversations": After selecting which conversations to archive when you press the 'Archive' button. | Consistency and standards. | The interface for confirming the action taken (the archive action) is inconsistent with the Canvas interface as it uses the browser's built-in styling for the pop-up rather than something that is consistent with the Canvas interface. This problem does not prevent the user from completing the given task but it can lead to confusion as the user may see the inconsistency as a problem having occurred with their action, rather than a confirmation as it is intended. |  Clicking on the 'Archive' button resulted in Firefox showing a modal asking if the user wanted to archive the conversation. | Minor usability problem (2). |  A sample pop-up from the Canvas calendar interface. The Canvas interface has examples of pop-ups being used that are consistent with the visual treatment of the interface. Pictured above is an example from the Canvas calendar interface which presents a pattern that could be used again within the conversations feature when archiving messages. |
- Once done the evaluation, find at one 'theme' of problems — where is there something common across the usability issues you have uncovered? Write down a brief (maximum 200 words) summary of why you believe this is a 'theme' in problems with the interface.
Bring to your June 12 class
A copy of the following:
- Your completed heuristic evaluation data sheet.
- Your brief summary of a 'theme' of problems.
We will chat about your deliverables in-class.
From June 12 to June 19
This week is for completing a report on the heuristic problems.
- Assemble all project materials into a document that is designed for easy and concise understanding. The document should include:
- Background: A brief overview of what you were evaluating and why.
- Methodology: Explain the tasks you chose, why you chose them, and the heuristics used in the evaluation.
- Results: Summarize the findings of the evaluation. This should not be a direct copy of the data sheet and should instead focus on collecting and connecting issues.
- Recommendations: Describe changes that should be made to correct the errors identified in the results and why they should be made.
- Appendices: All the raw data including original data sheets and summaries from the heuristic evaluation.
- Write this document as if it is intended for a non-designer (such as management). The format (PDF, website, etc.) is of your choosing but should demonstrate a refined and well-designed document. Maximum 500 words (excluding appendices).
You are allowed to improve your earlier weekly deliverables for your final submission.
Grading rubric
Your project will be graded on the following criteria:
| A | B | C | D/F | |
|---|---|---|---|---|
| Use of supporting artifacts (3 points): 'Do I have to guess what is referred to? (in background, methods, results)' |
|
|
|
|
| Reasonably concise writing (6 points): 'Can I follow the process? Does it stay on topic?' | The writing presents a clear narrative from beginning to end of the document without extra, un-related materials. | Each section of the document presents a clear narrative without extra, un-related materials. | Each section of the document presents a clear narrative with some extra, un-related materials. | The document presents an unclear narrative with many extra, un-related materials. |
| Effective argumentation (6 points): 'Am I convinced? (by recommendations)' | The recommendations use artifacts effectively in combination with written arguments to make a strong case for why a particular direction or conclusion is appropriate. | The recommendations use written arguments to make a strong case for why a particular direction or conclusion is appropriate. | The recommendations use written arguments that do not make a strong case for why a particular direction or conclusion is appropriate due to gaps in knowledge or the rationale. | The recommendations use written arguments that do not explain why a particular direction or conclusion is appropriate. |
| Audience (3 points): 'Would a client understand this?' | Document uses a language understandable by a lay — naive or new-to-the-topic — audience. | Document uses a language understandable by a professional or design audience. | Document uses a language understandable by a design audience. | Document uses a language not understandable by a design audience. |
| Polish (2 points): 'Would a client read this?' |
|
|
| |
Final submission requirements (June 19)
The final submission for P2 is a collection of all weeks of the project into a document that is designed for easy and concise understanding by a non-designer (i.e. manager). The format (PDF, website, etc.) is of your choosing but should demonstrate a refined and well-designed document.
Your project submission is due to Canvas before your June 19 class.
Please make sure double-check all your submitted files and URLs to ensure they can be opened. We want to avoid late or problematic submission penalties whenever possible.