Validating Usability for a New Document Library

Packaging Industry | Sr. UX Researcher

Tools Used: Miro | Microsoft Forms | FullStory | Microsoft Teams | Usertesting.com - SUS

Overview

To create a Document Library similar to an older product our customers used for purchasing and accessing documents for audits and product details, we developed a new Document Library on our purchasing platform. The layout and structure closely resembled the older version but included some new requirements. To ensure that the overall experience was clear and consistent for both our valued customers and internal stakeholders, I conducted several rounds of research.

This thorough approach aimed to provide our product team and stakeholders with the most impactful and actionable recommendations to significantly improve usability, as well as overall usage of the new feature. This initiative, in turn, is intended to effectively reduce the number of offline touches that our customer service and sales team encounters.

Research Methods

  • Design QA

  • Internal Feedback Survey

  • Data Analysis | A full data analysis utilizing our FullStory offering.

  • Moderated User Usability Interviews + SUS

  • Unmoderated Customer Service Usability Testing + SUS

Design QA

In addition to ensuring that development recreated an experience based off of our Figma designs, I also took time to document anything that felt clunky in the experience.

I no longer have access to the Miro account, but a thorough review of the experience was conducted.

The Filters

  • Search Tips (Modal) | Update the 1,2,3 font to the one in the Figma designs.

  • Search | From a visual standpoint, ensure that the search area matches our UI elsewhere to include the line underneath extending all the way to the right, the color of the search icon, regular text (not italics), and helper text to include applicable suggestions.

  • Reset Filters | Move to the bottom side by side with the Apply button like we do elsewhere in the product and change to “Clear Filters”.

  • Plant Location (Filter dropdown) | Update format to City, State, Country like we have in our designs.

The Table Header

  • Pills | Remove pills since we are unable to allow customers to “X” them out like this component is supposed to be used. Future UX discussion Mentioned in internal feedback

  • Download (Button) | Update to our button component we use in the platform for inactive download button elsewhere like this . The button should have an outline, not the disabled looking state.

  • Download (Copy upon selecting documents) | Currently we are not using “Files Selected” plural when a customer selects more than one and displaying “File Selected” regardless.

  • Download Limit (Copy & UX) | We currently show 00/50 regardless of whether the customer selects 50 or more which is technically the download limit. I recommend that we only show a number out of 50 if the user selects 50 or more. It currently reads as 50 being a total number of documents and I anticipate this confusing customers.

The Table Content

  • Checkbox

    • Remove the animation that development added to the checkbox behavior.

    • Add the “Select All” tooltip as noted in the Figma files.

  • Version Date (Column) | Update format from 0000-00-00 to 00/00/0000 like we use in the platform.

  • Plant Location (Column) | Update format to City, State, Country like designs & platform.

  • No Documents Found (Graphic) | Update graphic to search icon as seen in Figma designs.

Internal Feedback Survey Insights | 16 Responses

For this portion of the research, I collected initial usability feedback from both the Digital and Customer Service teams about the new Document Library, as many internal users will use it. Users were asked to log in and explore the library before giving their input. The feedback comes from 7 employees in North America (NAM) and 9 in Europe, the Middle East, and Africa (EMEA), including 8 responses each from the Digital Team and Customer Service.

"The library looks fairly straightforward and clear to navigate."

"I found it simple and relatively easy to navigate."

"Not very pleasing to the eye in general. Looks very internally focused and not customer friendly.”

Survey Responses

Internal Feedback Survey | Identified User Pain Points

  • The search feature often failed to find expected documents, frequently showing "No Results Found."

  • The filters didn't work well with the initial search.

  • The filter "(UX pill component) lacked the ability to remove each pill like it should according to UX best practices.

  • Customers were confused that the region defaulted to NAM, despite their actual location. The region filter was too low on the page, making it hard to both find and update.

  • Although the “Plant Location” filter was only relevant to “General Documents,” the column also displayed for “Product Specific Documents,” causing confusion.

  • The language filter did not work properly, giving results in languages that were not selected.

  • Users found the download limit display unclear, thinking 0/50 meant they had 50 documents when in reality it meant that the user could only download 50 documents at a time.

  • Users disliked the "Load More" button at the bottom of the table.

Data Analysis (FullStory) | 30 Days

  • Most of our users interacting were from North America.

  • Our top error click was associated with our checkbox component at 89% of users; most likely due to it’s size.

  • Our top overall click was the “Load More” button at nearly 50% of users, The next top clicks included the apply button, language filter, checkbox, and preview option.

  • Our top overall dead click was the download button at 32% of users, followed by the download icon within the button, filter container, checkbox, and header area.

  • The majority of users searched by a product family, material #, product keyword, or document type such as “ISO” or “SDS”.

  • The majority of users clicked the Document Library directly from the dashboard, followed by either the PDP or Order Details.

 

Usability Interviews + SUS | 8 Customers / 8 Customer Service

For this portion of my research on the MySEE Document Library, I focused on the overall usability of the library for both our customers and internal customer service partners. As requested by the team, I focused my efforts on the Food side of the business for both NAM & EMEA. I conducted eight moderated usability interviews with customers using Teams and eight unmoderated customer service interviews through usertesting.com.

Team Hypotheses

  • Customers may expect to update their region further up on the screen as well as for the region to default based off their location.

  • Customers may not understand the difference between our general and product-specific document types, making it difficult to find what they need.

  • Customers may find it more helpful to see a list of document types broken down much like our legacy e-commerce platform provided vs. how we default to one or the other today.

  • The timing and reasoning behind searching for specific documents is most likely tied to specific needs such as an audit.

  • The ability to reference previously viewed and/or downloaded documents could be helpful.

  • Based off of language being the top 3rd click on the page; users may expect for the language to control the page and not just the documents.

  • The way that our plant location information displays is most likely confusing to users as well as the fact that we display all plant locations regardless of a user’s region selection.

Usability Interviews + SUS | Insights

Overall, both customers and customer service partners were impressed with the new UI and pleased that they now have the ability to quickly access the documents they need in one place.

“If I knew this was available to me instead of reaching out to you...then yeah, I would come here and and get the document(s)... It’s kind of a one stop shop so it’s a good thing!” - Food Packaging Customer

“It’s much easier to operate than it used to be.” - Food Packaging Customer

Could users find the region filter?

Note that the region should ideally dictate much of the user experience, however it did not for our first implementation.

  • The majority of customers interviews; 63%, could not find the region filter when asked to show me where they’d go to update their region and were confused by the fact that our table content often times displayed both regions regardless of their selection.

  • I was pleasantly surprised to see that 100% of customer service partners interviewed were able to successfully locate the region. However, 25% did say they’d prefer to update the region at the top of the page instead.

  • It was interesting to see that 75% of customers sorted the region column in the table when asked to update their region instead of finding the filter.

Did users understand the difference in our document types?

Although customers seemed to understand the difference in our document types, it wasn’t always clear to the user which document type (general vs. product specific) to select in the filters before searching. This realization supports the break down of document types.

“If I was looking for your SQF or the food safety report, that's what I would look for ...vs. a general document...So yeah, I mean environment... I don't know what that means.” - Food Packaging Customer

  • Customers described General Documents as viewing “everything”, “all documents”, “Basic documents”, and those related to audits.

  • Customers described Product Specific Documents as viewing documents related to the products they order including product spec sheets.

  • Customer Service described General Documents & Product Specific Documents as viewing documents related to customer audits & those they’ve seen in our legacy e-commerce platform.

What type of documents did users expect to see within the Document Library?

Below I’ve tagged the documents based off which types were available in the library vs. those that were not users mentioned.

 

Were there trends in what users searched for and/or by within the library? Yes!

It was interesting to see that the majority of customers; 63%, searched by a “Product Family” to find the documents that they needed. The remainder searched by either a material # or a document keyword like “safety” or “letter of guarantee”. Similarly, customer service searched by Product Family and/or keywords. For clarity, a product family at this company was a combination of letters and numbers that tied to specific brands aka product families.

These findings aligned with the FullStory analysis where I observed the same type of search trends.

How often do users search for documents and why?

  • The majority of customers; 63% said they search for documents for audit purposes.

  • The remainder said they often times need documents for customer inquiries, internal sales request(s) for product info, and/or competitor comparison.

  • Customer service partners said they access documents daily to several times a week depending on their customer base and whether or not their customer(s) are on our e-commerce platform or not. Likewise, customer service said they are most often pulling documents related to audit purposes.

The data above represents customer responses; not customer service.

Is there anything that would make finding the same documents easier again?

Both customers and customer service said it could be helpful to have a way of quickly accessing previously searched for or downloaded documents. However, customers (50%) had more thoughts around the concept.

“Have a favorites folder...” - Food Packaging Customer

“A history of searches or something?” - Food Packaging Customer

“Last documents viewed” - Food Packaging Customer

“It wouldn’t have to be such a broad search...this is what you looked for last year and the year before and the year before; it’d be kind of nice.” - Food Packaging Customer

How do users feel about the download experience? Were users confused by the download limit UI?

  • 100% of both customers and customer service interviewed felt like the download experience was clean and easy to understand and were able to easily find and preview documents.

  • Customers did mention the need for greater visibility regarding document expiration dates.

  • Yes, in fact 50% of customers found the current download limit UI confusing and associated 0/50 with the total number of documents shown instead of the total number they can download at once. I didn’t receive feedback on the download limit UI since the CS interviews were unmoderated but expect that i would have if moderated.

“So be nice to know if it's good for a year, expires here or doesn't expire and then don't have a date on them at all.”

Did users have any thoughts around the number of documents they see by default before clicking “Load More” at the bottom?

  • Both customers (63%) and customer service (43%) suggested adding pagination to the experience. The remainder were okay with the current experience.

What were users' expectations when using the language filter? Is this filter one they would use often? If not, why?

  • It was encouraging to hear that the majority; 75% of customers and 71% of customer service expected for the language filter to display documents in their selected language(s). customers; The remainder expected for both the library & the documents to translate to the language they selected.

  • As far as usage, 85% of customers and 71% of customer service said no, they wouldn’t use this filter often. This was interesting since we anticipated this filter being more highly valuable to customer service partners.

Was the plant location important and/or relevant to users when looking for documents? If not, why?

  • For customers, they said it’s not useful to have the plant location content listed or available to filter down from since they don’t typically know where their products are being produced. It’s not surprising then that 87% of customers said they wouldn’t use this filter.

  • However for customer service, 71% said yes the plant location filter is both useful and important when narrowing down a document search. Likewise, 71% said they would use this filter often. This data points to a need for better defined internal vs. external customer facing experiences.

Comparing SUS Results | Customers vs. Customer Service

Both customers and customer service were asked to complete a system usability scale (SUS) survey at the end of each usability interview.

Customers scored the Document Library at a 65.9 (C).

Customer Service scored the Document Library at a 73.8 (B).

Recommendations for Improving Usability & Usage of the new Document Library

  1. Consider the value prop of adding some of the documents that are not currently available to the library. This would include analyzing how often and how many customers request documents like COA, COC, COI, Prop 65 Warning, and PFAs documents.

  2. Provide a way to see the total (#) of documents available per language. This should be possible from the back end and give EMEA customers a feel for how many documents are available in the language that they need. Also, consider future state allowing customers to see horizontally all available languages for a specific document vs. repeating the document over and over.

  3. Add a tooltip to the product family filter to provide clarity on what this is for external users.

  4. Provide clarification on what we mean by plant location to our customers by updating the header on the filter and table to something like “[Company] Mfg. Plant”. Another consideration would be to remove plant location from the external experience based off the findings above.

  5. Consider surfacing the expiration date for documents at a table content level vs. having to preview each one first. Improve overall transparency between differing dates on documents.

  6. Maximize the search functionality; ensuring that users can search by keywords with expected results.

The following validated recommendations were already in progress with the UX team based off the first few rounds of user research!

It’s encouraging that these active updates were validated by both customers and customer service feedback.

  1. Create an experience that's driven by the user’s region.

  2. Ensure that the filter pills are displaying as intended by the UX team.

  3. Create a global horizontal search component with helper text rather than a tiny left aligned search input.

  4. Rather than showing General & Product Specific Documents as the document types users can bounce between, allow users to see a breakdown of available document types in the library with the ability to select all for both general and product specific documents.

  5. Regarding the current download limit UX experience, users will no longer see the 0/50 UI which should provide clarity.

  6. Pagination will be included to allow users the ability to see a max of 50 per page which also eliminates the need for any type of error alert related to a user trying to download 51 or more documents at once.