9 – v2 Prototype and User Testing

There should be one or more design iterations following the initial prototype. Ensure you include screenshots documenting the evolution of your prototypes and explain the rationale for the changes from one revision to the next.

 Production

Based on our personas and task descriptions, a new prototype (v2) was created. This was a more high-fidelity version, created using Photoshop and the Marvel web-app, that allows the screens to be stitched together and pubished as an interactive site. This site could then be accessed on your mobile device, and downloaded as an icon to your home screen – giving the full impression that it is another app on your phone.

The prototype can be viewed and interacted with here.

This updated prototype provides “happy path” functionality for the following:

  • searching for wrecks in a list view via three different criteria
  • searching for a specific wreck in a map view
  • selecting this wreck and viewing the details
  • booking a tour of this wreck

A direct comparison can be seen below, showing the improvement between prototype versions 1 and 2. Click on each image to zoom in.

 

Beyond this, additional screens were added to demonstrate increased functionality. Click each image to zoom in.

The full set of images for this new prototype are available here

User Testing

Five test users were given the following tasks to complete using the updated prototype:

  1. From the home page, search for shipwrecks using the following criteria:
    • of Military type
    • located in Cork
    • of Novice diving difficulty.
  2. Now could you search using the Map View – for a wreck called the S.V. Albion, located off the southern coast, near Rennies, Nohaval, County Cork.
  3. Could you book a tour for this wreck, for the 5th of December, at 3pm, for 2 people.

Each user was asked to follow the Think Aloud Protocol, and describe their thoughts and actions to us as they participated in the test. For our part, we stayed quiet and let each user figure things out for themselves.

All of these user tests were recorded and can be viewed here.

Post-Test Questionnaire and Results

On this occasion we asked each user to fill in a complete System Usability Scale (SUS). In addition, we also selected a number of questions from the Questionnaire for User Interaction Satisfaction (QUIS), which was designed to assess users’ subjective satisfaction with specific aspects of a user interface.

These questions, and the responses given, are presented below. Click on each image to zoom in.

It should always be remembered that the System Usability Scale will not specify individual problems, but will instead give a firm indication of the level of work your usability requires.

An industry-accepted breakdown of the scoring system is as follows:

  • 80.3 or higher is an A. (People love your site and will recommend it to their friends)
  • 68 or thereabouts is a C. (You’re doing OK but could improve)
  • 51 or under is an F. (Make usability your priority now and fix this fast)

As can be seen in our own version of an SUS calculator, our prototype generated a very impressive 91.5 from our test users.

Thoughts:

  • people who had seen the existing Irish Wrecks site gave the highest scores, whereas new users gave lower – were expectations lowered for the repeat users?
  • as our UI is still a basic prototype, with a “happy path” throughout, this score has to be taken with a pinch of salt
  • but it is a great indicator that we are at least on the right track  – a very low score would have informed us that a redesign was urgently necessary

As for the Questionnaire for User Interaction Satisfaction, a good rule of thumb is to aim for scores above the average of the rating scale, in which case we again received very positive feedback.

User Feedback

Test User 1 – Samantha

  • Found the search functionality on the List View very easy to use
  • Slightly confused by our user task requiring her to switch to Map View after getting the list of wrecks
  • Tried to scroll and zoom in when on the first map screen
  • Momentarily confused when the Map View first changed to the close-up view
  • Found the process of booking a tour very easy to use
  • Improvements/Differences from existing Website
    • Overall found prototype very easy to use
    • Layout was good, easy to navigate
    • Couldn’t think of any negatives

Test User 2 – Fiona

  • During the List View search, noted that the dropdown for county is in alphabetical order, so was easy to find the required county
  • Slightly confused by our user task requiring her to switch to Map View after getting the list of wrecks
  • Tried to scroll when on the first map screen
  • Found the required wreck very quickly on the second map screen
  • Improvements/Differences from existing Website
    • Good layout overall
    • Happy that wreck information can be searched using dropdowns rather than links
    • Was nice to have the option to switch to Map View

Test User 3 – Alex

  • Had no problems with the List View
  • Noted that the first map screen indicated there were 11 wrecks off Cork, but after zooming in it seemed that only 3 were returned
  • Able to find the wreck and book a tour very easily
  • Mentioned that the style of the wireframe isn’t great, but understands that it’s just a prototype to test functionality
  • Didn’t feel particularly stimulated while using the prototype (based on post-test questionnaire)
  • Not keen on the font throughout
  • Would like to see some colour (he didn’t realise how much colour influences him)
  • Improvements/Differences from existing Website
    • Original website wasn’t that difficult to navigate once past the opening screens
    • Asked if prototype was a webpage or an app, because it’s very easy to fit a prototype to the screen as it’s a controlled environment
    • Probably took 1/20 of the time to complete the tasks with new prototype
    • Immediately able to access the information

Test User 4 – Eamonn

  • Found it confusing that search bar brought you straight to the list, wanted to be able to search a wreck by name using a search bar
  • List view was clear and easy to use
  • Map was good, little hard to read the writing on the popups after clicking on a wreck marker

Test User 5 – Johnny

  • Found the interface clear and intuitive, quickly used the filters to a find a wreck based on the criteria in the task
  • Found the black and white colour scheme clear and easy to use, no distractions
  • Was able to use the map view easily to find a wreck off the coast of Cork
  • When prompted about recommendations for further features they suggested an ability to book accomodation/get directions to the wreck for their trip.

References

  1. How To Use The System Usability Scale (SUS) To Evaluate The Usability Of Your Website
  2. SUS Calculator Template – usabilitygeek.com
  3. Can You Use The SUS For Websites?
  4. Human Performance repository – Full copy of QUIS

Leave a Reply

Your email address will not be published. Required fields are marked *