Wayfinder Evaluation Experiment Design


We looked at the following tasks for the user to do as part of the test:


We planned to conduct cross-product comparison and within-product comparison:



If we had sufficient time, we would have wanted to experiment on users with varying degrees of blindness to ensure the product was applicable for the whole blind population. Our preferred test group size would be 15-20 people with various backgrounds of age, gender, and occupation. We would use within-subjects experiment so the same users apply to multiple tasks, because we did not have access to large number of users to conduct between-subjects experiment without bias. 

Given the time and resource constraints, however, we planned to test with three users. These users graciously volunteered to evaluate our functional prototype:

Wayfinder Evaluation User Feedback

Users identified several improvement areas in our prototype:

  • One user suggested we put more aural markers to help users keep track of their location in the application. An example of this was to explicitly mark the ends of a menu or a list, either by having an end marker audio notification, or by implementing a rotating menu to cycle back to the first item once the end is reached.
  • All three users thought the side-way swipe gesture was intuitive for browsing through a list, but the up and down swipes were less intuitive and not very consistent, due to limitations of the prototyping environment. So we needed to make sure that our tutorial used easy language to explain the ideas that an up swipe would go back to the previous menu location, and a down swipe would exit the current session and return to the very top of the app menu. 
  • Two of the three volunteers had trouble hearing the voice menu in a relatively noisy outdoor environment, a limitation imposed by the volume of the Google Glass bone conduction speakers.
  • One user expressed confusion with the menu title "Retrieve Nearby Landmarks". The word "nearby" did not give him a clear sense of how close the landmarks would be, and in what order they would appear. This provided us with an opportunity to further refine the design of this functionality in future prototypes.
  • One user pointed out that a richer database that incorporated other navigation resources could enhance the effectiveness of the application. Such resources could include Google Maps street names and FourSquare/Yelp business names.
  • Two users expressed the desire to be able to edit and organize the landmarks more effectively. We should work on enabling edit and deletion of landmarks, as well as creating a multi-layer notification schema based on the importance of a landmark.