Elliott Bay Book Company

User Research Methods

#1 Competitive Analysis

What was the purpose of conducting a Competitive Analysis?

We conducted a competitive analysis as preparatory research. This methodology is used to evaluate and provide the opportunity to assess the strengths and weaknesses of competitors.

What actions were taken?

• Selected two competitors with similar business DNA, target audience, popularity, and are located in most visited cities in the United States.

• Compared the bookstores’ business performance using DemographicsNow.

• Read reviews in Yelp and Google Reviews.

• Compared the UI design, features, and best practices.

• Evaluated the strengths and weaknesses.

What were the findings and recommendations?

• From a business standpoint, Strand Bookstore (SB) in New York generated more revenue. This is followed by EB, and lastly, The Last Bookstore (LB) in Los Angeles.

• In terms of popularity, Strand Bookstore is the most visited bookstore. This is followed The Last Bookstore, then, EB.

• When it comes to the website’s design —

  • SB and LB use clearer labels and taxonomy. EB needs to develop better information architecture and select more informative labels.

  • Book categories can be easily spotted in SB and LB. On the other hand, EB needs to create one.

  • Unlike SB and LB’s events page with a calendar of events, EB populates its event page using books as a primary element. This can be confusing. EB needs to redesign this page with an objective that event information is instantly visible.

What would we do differently?

We would consider changing the variables in selecting competitors. Perhaps, competitors which were located in the same city and established in the same timeframe. We initially thought about this, but there are only select bookstores that instead of being a retail store where one can purchase books, people would visit because they are considered as a tourist attraction. However, we remained curious about what were the important factors or motivations for the customers who would prefer to visit Twice Sold Tales than EB.

#2 Contextual Interview

What was the purpose of Contextual Interviews?

We wanted to further validate our observations and findings from the Competitive Analysis. How do they affect the customer’s behavior? We used this methodology to have a better understanding of the customer’s behavior.

We have two main objectives in conducting this research —

• To know what type of people are going to Elliott Bay Book Company

• To know how the users use the website of Elliott Bay Book Company

What was the approach to conducting this methodology?

• Prepared interview questions using the Activities, Environments, Interactions, Objects, and Users or AEIOU framework.

• Intercepted bookstore visitors on-site and reached out to some students who would like to participate.

• Recorded the interviews and took notes of the highlights.

What were the findings?

The four customers whom we interviewed on-site and the two students like —

• The ambiance of the bookstore

• Their interactions with the staff

• The bookstore as a place to hang out with friends

• The memories they had in the bookstore

The majority of them did not use the EB website as they preferred the in-store experience. While those who attempted using the website find it frustrating and difficult to use.

Key design insights leading to more questions.

Come for the books, stay for the experience.

It became obvious that customers enjoy visiting the bookstore not only because of the books but because of the experiences they offer. This led us to consider browsing travel websites. What if we redesign their website not only as an online retail shop but as a destination? And even if that’s the case, we also wanted to make the website as an additional channel to generate revenue. After all, a website is an expense. Since customers like the staff pick, how do we make the staff pick more visible to increase sales? Furthermore, one of the many ways to encourage visits is to hold events such as book reading, talks by authors, etc. How are we going to improve the usability of the event page so that we increase the store traffic?

What were the lessons learned and the next steps?

During this activity, we reached out to EB personnel to learn more about the business. Also, in this way, we can narrow our design decisions, understand what features to prioritize, gather insights on how they use their website, and so on and so forth.

Unfortunately at that time, according to the person we spoke to, there were already ongoing initiatives to redesign the website.

As for the activity itself, we learned that the more participants we recruited or interviewed, the better the study is. It reduces the risk of the study being biased.

This activity also helped us in creating scenarios and tasks for baseline testing.

#3 Baseline Testing

What is Baseline Testing?

The baseline test is a methodology used to evaluate the website’s current performance by gathering quantitative and qualitative data.

In conducting the baseline test, what were the activities performed?

• Prepared a baseline test script, scenarios, and tasks.

• Develop a Key Performance Indicator (KPI) plan.

• Observed participants’ interaction with the website.

• Gathered and analyzed data.

Here is the snapshot of the quantitive data:

Based on these findings, it could be observed that there was a high failure rate. 3 out of 5 participants struggle to interact and navigate the website.

• Success Rate: On average, 57% (3 out of 5) failed in achieving a satisfactory result for completing each task.

• Failed Distribution: Out of 3 participants who did not achieve a satisfactory rating, 2 - 3 of them were unable to complete the task within the time limit. Furthermore, 1 - 2 gave up or failed to complete the task at all.

• Error Count: Participants make mistakes when using the search bar to look for information. This resulted in a high failure rate.

Now, what are the next steps?

After the contextual interviews, we knew we wanted to focus on changing the aesthetic of the website and narrow our efforts in creating pixel-perfect visual designs. However, given the high failure rate, I insisted that we go back to our first impressions - redesign the information architecture, use clear labels, and improve the taxonomy.

That led us to conduct the card sorting activity.

Any thoughts or reflections?

This was our favorite methodology because it is both qualitative and quantitative. This study would benefit from increasing the number of participants. Also, we’d be tempted to select non-tech savvy participants for baseline testing because the ones we interviewed hardly have a social media presence.

Lastly, we strongly believe that the EB website would benefit more in a redesign where function and information are the primary objectives.

#4 Card Sorting

Why conduct a Card Sorting exercise?

Card sorting is a methodology that is used to generate a category tree or the taxonomy to improve the information architecture, workflows, menu structure, and website navigation paths.

For the redesign process of this website, it was important that we work on the information architecture and menu structure.

How to go about a card sorting exercise?

• Created groups and cards using OptimalSort.

• Asked 10 participants to group the cards.

• Asked supplementary post-questions.

• Gathered and analyzed data.

What were the key findings?

Based on the collective results of the participants, categories are arranged in this logical order: Books, Events, About, More.

• Books constitute with the eight cards. 90% to 100% of the participants responded that these eight cards should be under the Books category. These are All Collections, Audiobooks, Bestsellers, Digital Books, First Editions, What’s Popular, Staff Picks, and What’s New.

• Events constitute three pages. 100% of the participants chose Book Group Meetings, Calendar of Events, and Children’s Events.

• Based on the percentage achieved, only Contact Us is expected to be in the About category. Three other pages were grouped in About section but this is between 10% and 40%.

• With regards to the More category, most participants put the cards that they feel should not be in any of the three categories, or these cards could be in any other categories but are slightly misplaced. An example of this is Media, which was grouped under Books by 40% of the participants. They feel that while Media is not a book, it could be in the Books category assuming these are Audiobooks.

Consider these actions:

• Separating Books and other Media: Many participants expressed that there are too many cards under Books. To break this down, separating Audio Books, Digital books, CDs and DVDs into another category will make the website more straight forward.

• Adding Media to the navigation: Even though Books are technically considered as media as well, to shorten this label, Audio Books, Digital books, CDs and DVDs should be under Media.

• Adding Programs to the navigation: This will constitute Book Consignment, Book Donations, and Maiden Voyage. Rename and Shorten the following labels:

  • What’s New → New Releases;

  • What’s Popular → Popular Now;

  • Digital Books → eBooks;

  • Calendar of events → Calendar.

• Place the Gift Certificates on the website strategically: Add secondary navigation.

What were the next steps?

• Planned to create at least two wireframe designs.

• Planned to conduct a Paper Prototype Test.

• Planned to work on the Final Design.

Closed vs Open vs Hybrid Card Sort

We repeated this activity. The first attempt was a hybrid card sort that seemed to confuse the participants. In the interest of time, we decided to do a closed card sort, that way they could focus on only grouping the cards. The challenge we faced on the hybrid card sort was, everyone seemed to create a new category making it difficult to analyze.

#5 Comparative Paper Prototype

When to use a Comparative Paper Prototype?

A paper prototype is creating low fidelity wireframes using a pen and paper. This is used to test user interfaces quickly and economically. In this activity, we created two designs. The designs were compared to test which one is more usable.

Watch the Comparative Paper Prototype demonstration.

How did we like the activity?

It was really fun! We found a new appreciation in using a pen and paper in creating prototypes. We learned how ideas flow freely. We did an awesome job with the folding of the papers to transition from one page to the next.

Personally, I always keep a pen and paper. Each time I have an aha-moment, I sketch my ideas. It’s effective as part of my design process, especially when I was starting as a designer.

Ideally, this is the last activity. But since we wanted to explore how the different methodologies would aid in designing, we came up with high fidelity prototypes.