10 – Goal Analysis and Future Updates

Goals serve as acceptance criteria during evaluation … Must set goals that we will aim for the product to achieve.

Prototype Testing Results

The results of user testing for our v2 prototype were as follows. A reminder that seconds indicates how long it took to complete the task, while interactions is how many taps/scrolls/swipes the user made while doing so.

Task 1 Task 2 Task 3
seconds interactions seconds interactions seconds interactions
Test User 1 – Samantha 13 6 35 12 22 9
Test User 2 – Fiona 24 6 30 8 13 8
Test User 3 – Alex 14 6 18 5 19 9
Test User 4 – Eamonn 29 8 58 20 14 10
Test User 5 – Johnny 14 6 16 10 15 9

* Experienced Users

* Novice Users

Our Findings

General:

  • It should be noted that the “experienced” users used a different mobile device to the “novice” users – OnePlus One vs Sony Aqua M4.  As the OPO has a larger screen size, it deserves attention that the novice users at times had to scroll more to see options towards the bottom of the screen.
  • It should also be noted that some users were more vocal than others in terms of embracing the Think Aloud Protocol. In some cases this can lead to slower response times compared to those users who focus solely on completing the task.

Task 1:

  • Our estimated time of 14/15 seconds proved to be reasonably accurate for both user types – with just over half of our users matching this time.
  • It should also be noted that Test User 2 took additional time to complete this task as she was fully embracing the Think Aloud Protocol as we had requested.
  • The vast majority of users, regardless of experienced or novice, needed the exact same number of interactions as we did. This value did not need to be adjusted.

Task 2:

  • For experienced users, our estimate of 25 seconds was likely over-optimistic, with only one user bettering this time.
  • For novice users our estimate of 30 seconds was right in the middle of our user scores – with one user well below this time and one user well above it. A result like this proves that a larger pool of test users is definitely required in future iterations to provide a better average result.
  • For experienced users, our estimate of 8 interactions matched the average score perfectly.
  • For novice users, it is likely we will need to increase the interactions estimated value.

Task 3:

  • Of interest here is the fact that our novice users actually bettered the average time of the experienced users. This can likely reflect that the prototype design for the map view has been improved to such a degree that the experienced users hold no advantage over the novice users.
  • Our estimated time of 18 seconds was an exact match for the overall average response, so this value can remain the same for the next iteration.
  • All users bettered our interactions estimate, so this can be lowered.

Future Quantifiable Goal Updates

Based on the findings from the user testing above, the quantifiable times will need to be updated based on the analysed data. Using the initial time as a baseline value and the average quantified time based on the user’s performance during the testing phase, the median value will be the updated time for each of the quantitative goals.

Experienced Users

  • Task 1 (interactions)
    • Baseline 8 interactions
    • User Performance Average: 6 interactions
    • Iterative Quantifiable Interactions: 6 interactions
  • Task 1 (seconds):
    • Baseline 14 seconds
    • User Performance Average: 17 seconds
    • Iterative Quantifiable Time: 15.5 seconds

  • Task 2 (seconds):
    • Baseline 25 seconds
    • User Performance Average: 28 seconds
    • Iterative Quantifiable Time: 26.5 seconds
  • Task 2 (interactions)
    • Baseline 8 interactions
    • User Performance Average: 8 interactions
    • Iterative Quantifiable Interactions: 8 interactions

  • Task 3 (seconds):
    • Baseline 18 seconds
    • User Performance Average: 18 seconds
    • Iterative Quantifiable Time: 18 seconds
  • Task 3 (interactions)
    • Baseline 12 interactions
    • User Performance Average: 9 interactions
    • Iterative Quantifiable Interactions: 9 interactions

Novice Users

  • Task 1 (seconds):
    • Baseline 15 seconds
    • User Performance Average: 21.5 seconds
    • Iterative Quantifiable Time: 18.5 seconds
  • Task 1 (interactions)
    • Baseline 9 interactions
    • User Performance Average: 7 interactions
    • Iterative Quantifiable Interactions: 7 interactions

  •  Task 2 (seconds):
    • Baseline 30 seconds
    • User Performance Average: 37 seconds
    • Iterative Quantifiable Time: 33.5 seconds
  • Task 2 (interactions)
    • Baseline 10 interactions
    • User Performance Average: 15 interactions
    • Iterative Quantifiable Interactions: 13 interactions

  • Task 3 (seconds):
    • Baseline 18 seconds
    • User Performance Average: 14.5 seconds
    • Iterative Quantifiable Time: 14.5 seconds
  • Task 3 (interactions)
    • Baseline 12 interactions
    • User Performance Average: 10 interactions
    • Iterative Quantifiable Interactions: 10 interactions

Leave a Reply

Your email address will not be published. Required fields are marked *