IbnMubarak's Blog

Pre-Test Briefing

abdullah — Sun, 01 Aug 2010 15:57:34 +0000

Description of the room, including camera equipment used to record the actions and words of the evaluator
Description of the observers, either in the room or behind a one-way mirror in the control room. If the observers are in the room, they should be briefly introduced (first names will do)
Explanation of the product being evaluated
Explanation of the process of the evaluation, including scenarios, questionnaires, and the use of telephone (if appropriate)
Explanation of think-out-loud process
Demonstration of the think-out-loud process
Reminder that you are asking them to help you evaluate the product you are testing the product, not the user

taken from: Barnum, Carol “Usability Testing & Research” p234

Usability Yardstick to Measure Findings

abdullah — Sun, 01 Aug 2010 14:56:05 +0000

When using the Top-Down approach to log results from user testing, place them under the following categories.

Clarify the core concepts
Does the design match the user’s mental model (e. g., the desktop metaphor)?
Fit content to customers who use the product
Does the content fit the user’s expectations? Is the information organized or mapped properly and at the right level for the audience? Are there any missing parts?
Plan and maintain consistency
Is there consistency from program to program (external consistency and within a program (internal consistency)?
Provide reassuring feedback
How does the system provide feedback or information to the user (dialog boxes, time to complete tasks etc.)’?
Clarify interaction rules
Does the system follow a standard set of rules For interacting with the product from a design point of view (e.g, drag and drop or buttons)?
Structure navigation clearly
Does the user understand how to get from point A to point B (menu, icon, keyboard use)?
Use plain terminology
Does the language match the user’s own vocabulary of use?
Optimize user assistance
Can users get help when they need it (on1ine help or performance support)?
Optimize visual design
Is the ‘visual design effective and pleasing (e.g,, do radio buttons look like radio buttons; are icons intuitive in design?
Design for the context of use
Can the product he used as designed (are there “bugs” in the system that prevent it or is the context of use poorly conceived

taken from: Barnum, Carol. “Usability Testing and Research” p159. Also, see http://www.t2i.se/pub/papers/SpaceLab_Usability_Evaluation.pdf for real life usage of this.

Evaluation Measures

abdullah — Sun, 01 Aug 2010 11:29:39 +0000

Quantitative Data:

Time to completion of each task
Number of participants completing tasks within allocated time
Number of participants completing tasks with extra time
Number of problems encountered
Number of errors (unsuccessful tries)
Number of local problems
Number of times each participant uses “help” menu
Number of times each participant accesses advertising sites
Length of time each participant spends on browsing advertisements

Qualitative Data:

Facial expressions
Verbal comments when they think out loud
Spontaneous verbal expressions (comments)
Interviews before the test
Interviews after the test
Miscellaneous activities (stretching, wanting breaks, etc.)

taken from: Barnum, Carol. “Usability Testing and Research” p186

Tasks Commonly Tested in User-Testing

abdullah — Sat, 31 Jul 2010 23:00:23 +0000

First impressions (look and feel of the product)
First tasks (so important in fixing in users’ minds whether they will consider the product easy or difficult to use)
Tasks most frequently performed
Critical tasks (even if performed less frequently)
Specific problem areas (typically identified by sponsor or heuristic evaluation)
New tasks added to a product or changes made from an earlier version of the product (including changes made after an earlier usability test of a prototype)

taken from: Barnum, Carol. “Usability Testing and Research” p161

Factors to consider in Persona’s – Chauncey Wilson

abdullah — Sat, 31 Jul 2010 22:45:45 +0000

Motivation – how motivated are users to use the product?
Typing skills – critical for some jobs
Task experience – familiarity with the task that the product supports
Turnover rates – high turnover rates would indicate that ease of learning issues is critical
General education
Learning style
Computer background
Software applications used
Expected training support for the product
Age ranges
Stress level when performing tasks
Domain (subject matter) knowledge
Number of people involved in performing the tasks
Job category
Level of automation
Attitude toward the job
Location of use (home, office)
Physical characteristics
Disabilities – what percentage of the user population has disabilities that could affect product use?
Number of people in user population with English as a second language

taken from: Barnum, Carol. “Usability Testing and Research” p158

User-Centered Design Process in a Nutshell

abdullah — Sat, 31 Jul 2010 22:27:26 +0000

Design
It is the idea of the product transformed into a blueprint, yet it is still abstract.
Build a Prototype
It is the first tangible model of the product and extremely important as testing cannot begin without it. Ranging from paper prototypes to interactive web pages, choose the level of fidelity that takes the least amount of time and resources.
(Further reading: comparison of fidelity levels, benefits of using prototypes)
Self-test & Improve Prototype
Identify as many problems as you can yourself before asking the users. Use heuristics and cognitive walkthroughs. Heuristic evaluations compare product against a set of rules or principles, while cognitive walkthroughs are role-playing sessions where reviewers try to perform tasks with the product as inexperienced users. Improve the prototype based on the results from self-test.
(Further reading: Overview of Cognitive Walkthrough process, comparison of heuristics and cognitive walkthroughs, Other methods (focus groups, interviews, surveys, etc), some heuristics:
- 4 Heuristics for designing minimalist instruction, Hans van der Meij & John M. Caroll
- Nielsen’s 10 usability heuristics
- Gerhardt-Powal’s 10 cognitive design principles
- On Being a Butler Heuristics
- Usage Patterns (O’Reilly) Heuristics.
Do User Testing & Improve Prototype
1. Goal: What do you want to learn from users?
2. Persona’s & Subgroups: Create precise & detailed persona’s of expected users. Give each a name, a car, a job, a life. Don’t design for average user. Design considering each user separately. Divide persona’s into expert and novice users. Have at least 3 participants to test each group. Design for both novice and expert users and remember that novice users will mature into experts.
  (Further Reading: Chauncey Wilson’s user-profile factors)
3. Tasks to be Tested: You can’t test all tasks that could be performed with the product so focus on those that meet the goal of testing. After listing the tasks to test, prioritize them to make sure important ones get done even if time runs out. Time each task based on the evaluation measures (quantitative/qualitative) and keep the test under an hour.
  (Further Reading: Commonly tested tasks, Evaluation Measures)
4. Scenarios: Convert the tasks to be tested into scenarios. For example, the task of filling out online form can be converted into ‘You want free email. Register for an account’. Avoid creating steps or instructions. Focus on the bigger process, not the steps required to complete it. Avoid using the language of the product. Don’t say: click on “check-mark”.
5. Post-test Questionnaire: Having gone through the testing, the participant has become a great resource for user feedback. Don’t let him go without asking his opinions on ease of use, perception and any other input he wants to give.
6. Dry Run: Do a dry-run without actual participant or get a tolerant user. Then do a pilot test with an actual user. You need a briefer, a logger and a cameraman/timekeeper. The briefer will be the only person who talks to the participant. The logger notes down everything. The cameraman/timekeeper keeps track of the time taken for each task.
7. Test: Screen users to put them in the right subgroup before the test. Debrief them about the whole process. Observe them as they complete the test and do post-task and post-test questionnaires.
  (Further Reading: Pre-Test Briefing, Thinking-Out-Loud Procedure, Categories for logging test results)
Repeat process for features that got changed

For samples of how to implement these visit: http://ablongman.com/barnum/

summarized from: Barnum, Carol. “Usability Testing and Research”.

Life-cycle of User-Centered Product-Design

abdullah — Sat, 31 Jul 2010 19:40:53 +0000

Tech-Ed, a usability consulting firm, proposes the following 12-step design process:

Design site
Develop prototype (fidelity levels, M. Shrage’s heuristics)
Perform heuristic evaluation (& cog walkthru p158 C Wilson)
Incorporate feedback into alpha site
Perform usability test of alpha site (focus groups, surveys, tech support, http://ablongman.com/barnum/pdf/bladefinalreport.PDF test plan, scenarios, walkthru, pilot, )
Incorporate feedback into beta site
Perform log analysis of beta user activity
Use log analysis data to identify areas for further testing
Perform usability test of beta site
Incorporate feedback into first release
Perform ongoing log analysis of site
At intervals, use log analysis data to identify areas for further testing and improvement

Taken from: Barnum, Carol. “Usability Testing and Research” p385

Usability Testing

abdullah — Sat, 10 Jul 2010 19:54:53 +0000

Lab Testing

With Lab: put subject in a separate room with one-sided glass and record his reactions to the product
Without Lab: observe subject from a distance and mark down his reactions to the use of the product. Could also ask him to be verbose.

Field Testing

Structured Observation: observe subject in his home
Longitudinal Studies: leave product with subject for some time. ask for feedback when picking up product.

Expert Testing

Heuristic Evaluation: evaluation against a set of principles (e.g. “error messages should be expressed in plain languages without any codes” will be ensured)
Cognitive Walkthroughs: exploration walkthroughs where the problem solving of the user is replicated (e.g. print the page: would the user know how to do it? would he know where to look?)

User Testing

Focus Groups: Traditionally, 6-12 users are moderated by a facilitator who has an agenda of finding out their preferences on certain features of the product & can dig down deeper by asking them their reasons. He ensures everyone gets equal chance to voice their opinions and keeps his questions unbiased, so as not to steer them towards a certain opinion. Online focus groups are also used nowadays.
Survey: A question paper asking users about their preference and experiences with the product. Reliable results if done by more than 65% of the people who were asked. Cannot probe deeper as in a focus group. People usually don’t write much in open-ended questions.
Interviews: One-on-one form of focus group. Can dig deeper for information and interviewees are willing to give more info, but is expensive. Can get good responses to open-ended questions.
Internal Sources of Customer Feedback: Tech Support (gets most questions on hardest part of the product), Trainers (who train others on the use of the product)

taken from: Barnum, Carol. “Usability Testing and Research”. New York. 2002

Designing Visual Interfaces

abdullah — Mon, 28 Jun 2010 00:15:56 +0000

Elegance & Simplicity
- Principles: Unity, Refinement, Fitness
- Techniques:
  - Reduction
    1. determine essential qualities (adjectives)
    2. critique each element (does it relate to the essence of the design?; would design suffer without it? remove if non-essential)
    3. remove essential elements & see effect (if still essential, think of replacements)
  - Regularization
    1. use regular geometric forms, simplified contours & muted colors
    2. make forms identical in size, shape, color, texture, line-weight, orientation, alignment, spacing.
    3. limit topography to few sizes from 1 or 2 families
    4. make critical elements not-regularized, so they stand out
  - Leverage
    1. review functional role of each element
    2. identify if multiple elements are filling (or partially filling) the same role
    3. see if adjacent elements can replace elements (even if it requires minor modifications)
    4. combine redundant elements into single, simpler unit or replace the lot with a higher level idiom
      *Don’t overdo leverage (wrist watch: time & stopwatch in same screen)
Scale, Contrast & Proportion
- Principles: Clarity, Harmony, Activity, Restraint
- Techniques:
  - The Squint Test
    1. close one eye & squint the other
    2. step back & take a look at the whole screen
    3. identify elements that are not apparent (they won’t be readily recognized)
  - Establishing Perceptual Layers
    dimensions that are both associative & selective are to be contrasted (e.g. color)
    1. assign each element to a group; categorize elements based on origin/intended use
      (keep categories few – around 7)
    2. rank groups according to importance in 3-5 levels
    3. differ perceptual variables for each group to form layering effect
      (Hierarchical – size, value, …)
      (Non-hierarchical – hue, …)
    4. maximize difference between groups, but minimize difference within groups
    5. squint test to ensure elements of one group stick together, but whole group is separate from other groups
  - Sharpening Visual Distinctions
    1. Identify ranks of groups/elements
    2. determine range of variation available (in size, color, etc) & use as much of the range as possible
    3. use log-scale rather than linear-scale across visible range to enhance discriminability (skipping a level is usually sufficient)
    4. squint-test to ensure the first 2 or 3 echelons can be seen “at a glance”
  - Integrating Figure & Ground
    1. determine overall size of figure & ground
    2. equalize visual weight (squint test to ensure neither positive, nor negative space dominates
    3. provide sufficient margins to reduce visual tension
    4. center the figure on the ground if possible. squint test to ensure figure looks centered
    5. margins in GUI > margins in print
Organization & Visual Structure
- Principles: Grouping, Hierarchy, Relationship, Balance
- Techniques:
  - Using Symmetry to Ensure Balance
    1. identify axis of symmetry (x or y)
    2. vertical axis is more prevalent
    3. equally distribute content on both sides of symmetry axis
    4. center the axis of symmetry in overall display
    5. squint test to ensure symmetry & other effects
  - Using Alignment to Establish Visual Relationships
    1. identify major boundaries & align items to them to enhance them
    2. elements that are almost aligned, align them completely by changing size, or position of margin/element
    3. look for free-standing elements & align them with major margin or some other element in the display
    4. if can’t be related to anything, try to relate it to proportions of display itself, by positioning it to a regular division of space
  - Optical Adjustments for Human Vision
    1. determine true alignment/spacing required
    2. extend elements beyond the margin according to the sharpness of acute angle (more acute = more extension)
    3. use “close-up” squint test to ensure element equivalence
  - Shaping the Display with Negative Space
    1. review organization of information in a prioritized set of chunks (review prioritized info groups)
    2. separate individual units of info by adding extra white-space between them
    3. determine which elements require additional visual emphasis
    4. increase white-space around critical elements (1) by moving other elements away, or (2) by moving them into margin
    5. remember white-space is not wasted space
Module & Program
- Principles: Focus, Flexibility, Consistent Application
- Techniques:
  - Reinforcing Structure through Repetition
    1. start with a rough sketch of the series of layouts
    2. look for common margins/functional units
    3. look for elements that are visually related but can’t be placed beside each other
    4. use standard locations & consistent style of rules, text, images, …
  - Establishing Modular Units
    1. determine vertical unit (e.g. front height)
    2. should be good to provide distinction from other vertically stacked groups
    3. determine horizontal unit – should be large enough to contain the largest word (at least 3x the vertical unit)
    4. horizontal unit should be 5-7 divisions of screen width & should be easy to factor into multiples & partial units
  - Creating Grid-Based Layout Programs
    1. determine size restrictions of layout
    2. determine basic vertical (groups) & horizontal (labels with options) modules
    3. draft a rough layout sketch
    4. extend smaller elements; shrink bigger ones
    5. for resizable window layouts, design for minimum acceptable width, not arbitrarily small
Image & Representation
- Principles: Immediacy, Generality, Cohesiveness, Characterization, Communicability
- Techniques:
  - Selecting the Right Vehicle
  - Refinement through progressive abstraction
  - Coordination to Ensure Visual Consistency
Style
- Principles: Distinctiveness, Integrity, Comprehensiveness, Appropriateness
- Techniques:
  - Mastering the Style
  - Working Across Styles
  - Extending & Evolving the Style

taken from: Mullet, Kevin et all. “Designing Visual Interfaces”

Be a Butler

abdullah — Sun, 27 Jun 2010 23:45:37 +0000

Do less features, but do them well. Even if its less marketable, satisfied users will spread the word and will be more loyal customers.

Paul Grice’s (1967) Cooperative Principles for Conversation:

Quantity
Quality
Relevance
Manner

Respect Physical Effort

Treat Clicks as Sacred
Remember Where they put things (& where they were)
Remember What They Told You
Stick with a Mode

Respect Mental Effort

Use Visual Elements Sparingly
Make Common Tasks Visible; Hide Infrequent ones
Give Feedback; Show Signs of Progress
Keep Preferences to a Minimum; Give Smart Defaults
Follow Conventions (Even if they’ren’t your ideal design)
Look for Widgetless Features

Be Helpful

Offer Sufficient Information Early & in Context; Prevent Errors
Solve Problems; Don’t Complain or Pass the Buck
Be Predictable
Request & Offer Only Relevant Info; Don’t Mislead
Explain in Plain Language
- Avoid Jargon
- Don’t Blame the User
- Indicate Consequences of Options
- Use Common Sense

taken from: Isaacs, Ellen et. all. “Designing from Both Sides of the Screen”