3 – User Research: Questionnaire

Conduct at least one form of user research to gather data about your user group(s).

Introduction

A questionnaire is a simple tool for gathering information. It typically consists of a set of questions used to assess a participant’s opinions, attitudes, preferences and characteristics on a given topic. As a research method, a sample of the wider audience is used. The gained insights can then be applied to a broader population when crafting the next design iteration.

Initial Draft Questionnaire

Based on the collective and individual research we had performed whilst investigating and interacting with the Irish Wrecks website, each group member then worked on their own list of potential questionnaire questions. I personally studied numerous articles and blogs about the art of creating quality UX questionnaires, some of which are listed below.

These questions were our initial attempts to document the thoughts of current website users, potential website users and those individuals who have never thought about wreck-diving before. I then collated the three lists of questions into one draft; removing unsuitable questions, improving grammar and phrasing, removing duplicates, etc. This draft was then transferred to a Google Document that could be shared with the project group, to allow for further discussion and amendments.

A selection of these questions are listed below. Note that all questions provided a multiple choice for the answer – though these are not included here for sake of brevity.

However the full list of draft questions and answers can be seen here.

Gender/Age/Occupation
Have you ever gone scuba diving?
If yes, how often would you dive?
Have you ever visited the site of a wreck before?
If no, how interested would you be in visiting a wreck?
What type of wreck would you be most interested in visiting?

Would you download an App for your phone with information about Irish wrecks and how to visit them?
With whom would you most likely visit a wreck?
Would you pay for a guided tour of an Irish wreck?
Would you pay more for a private tour, or prefer to travel with a group?
Would you like the ability to book accommodation for your trip via our app?

Creating and Publishing our Questionnaire

David had already created an initial questionnaire using Google Forms. Once we were happy with our draft questions, I updated this Form with our content and experimented with the different styles for framing each question; such as multiple-choice, checkboxes, linear scale and multiple-choice grid. This was to ensure we had the best chance possible of helping the user to understand the question and feel comfortable providing an answer. Click on the images below to see some examples.

Once complete, we published the link to our questionnaire via our personal Facebook and LinkedIn accounts to start the process of gathering data.

The completed questionnaire itself can be viewed here.

Questionnaire Results

After leaving our questionnaire open for a few days, we quickly gained over 113 responses – at which point we prevented any more submissions and started analysing the data. Google Forms conveniently has an option to create a spreadsheet with all of the submitted responses, and we started from there.

This spreadsheet can be viewed here.

Our first major decision was to split our data based on one primary question – “have you previously visited a wreck?”. This split our responses into two user groups, 13 people who had previously visited a wreck – and 88 who had not. This immediately allowed us to come up with ideas for personas, task descriptions and prototypes that would be based on these two opposing user groups – an experienced user and a novice user, if you will.

Over a period of a few days, I went through the data in every way possible – creating a number of extra sheets (see tabs at the bottom of the file) to list new subsections of data, and using formulas to calculate every last bit of useful information I could. Just to get across how convoluted these formulas started to get and how long it took to process this spreadsheet, some of them are listed below:

=ARRAYFORMULA(INDEX(B2:B89,MATCH(MAX(COUNTIF(B2:B89,B2:B89)),COUNTIF(B2:B89,B2:B89),0)))

  • display the most commonly occurring value in a column
  • (e.g. did Female or Male appear most often)

=ARRAYFORMULA(MAX(COUNTIF(B2:B89,B2:B89)))

  • count how many times the most commonly occurring value appeared in a column
  • (e.g. 67 Females)

=lookup(X2,{“-“,“Extremely important”,“Moderately important”,“Not at all important”,“Slightly important”,“Very important”},{“”,5,3,1,2,4})

  • assign a numerical value depending on which of the five text-based responses were given for a question
  • (e.g. Extremely important = 5, Not at all important = 1)

=UNIQUE(X2:X88)

  • get all unique values in a column
  • (e.g. across the 88 rows of data, provides just the 5 unique text values from the string above)

=COUNTIF(X$2:X$88,X94)

  • count how many times each of those unique values appeared
  • (e.g. Moderately important = 24, Extremely important = 18)

=MEDIAN(Y2:Y89) >>> =AVERAGE(Y2:Y89) >>> =STDEVA(Y2:Y89)

  • calculate the median, average and standard deviation for a set of values
  • (e.g. provide useful data based on all of the 1-5 values above, such as the average score for this question was 3.19 out of 5, with standard deviation of 1.62)

 

I was also able to create a number of charts based on the data, to help visualize trends more easily. Click the image below to zoom in.

References

  1. Scott Smith, Ph.D. (2013). Survey Questions 101: Do You Make any of These 7 Question Writing Mistakes? – Qualtrics
  2. Diane Loviglio (2012). Writing Surveys That Work – Mozilla UX
  3. Chris Gray (2014). Better User Research Through Surveys – UX Mastery
  4. UXBeginner – How to Get People to Fill Out Your Damn UX Surveys
  5. FluidSurveys – Solving the Mystery of the ‘Survey Questionnaire’
  6. Achilleas Kostoulas (2014). How to Interpret Ordinal Data
  7. http://webapps.stackexchange.com/questions/89063/how-to-include-multiple-if-statements-in-one-cell-in-google-sheets
  8. http://stackoverflow.com/questions/14380882/count-distinct-values-in-spreadsheet
  9. http://stackoverflow.com/questions/18070201/how-to-output-the-most-common-value-and-the-number-of-occurrences-of-that-value

Leave a Reply

Your email address will not be published. Required fields are marked *