Feeback Session @ customer conference
Executive Summary
-
The UI console was only available via the VPN, which caused a lot of problems, thus we had only 5 participants.
-
All participants rated ease of use either a 4 or 5 (where 1 is very difficult and 5 is very easy), even when tasks were not accomplished properly the first time
-
Participants expressed that the UI was very learnable
-
Based on feedback, no major design changes are needed, but consider addressing some of the observations
-
Everyone raved about the graph visualization
-
Although people thought that the routing workflows were great, many said they wouldn't be able to use them in their environment since they prefer to go through source control
Background
OpenShift Service Mesh was not yet generally available when this testing was performed (Spring 2019).
The UX team was not proactively involved in the design of this UI, but was brought in at times to provide expert reviews. This was the first feedback session UX was involved in.
Goals and Methodology
Two usability tests were offered: (1) focused on the graph (2) focused on routing & validation
Participants were given tasks to be performed, actions were recorded and participants were asked post task question
Graph Tasks
Average ease of use ratings, prompts and notable observations
4
View graph for bookinfo namespace
Observation: No one was able to successfully achieve this task on their first try
5
Explore different ways to view traffic on the graphic
4
Which edge displays tcp traffic?
Observation: No one was able to successfully achieve this task on their first try
5
How many versions does the view application have?
5
Get information for the reviews application
5
Get additional information about the reviews application
5
Hide the product page application
5
Overall graph flows
Routing and Validation Tasks
Average ease of use ratings, prompts and notable observations
4
Determine the traffic distribution between the 3 versions of the Reviews service
Observation: No one was able to successfully achieve this task on their first try
4
Create routing configuration for Reviews service with specified distribution
Observation: No one was able to successfully locate this action on their first try
5
Delete the virtual service & destination rules created for the Reviews service
Observation: No one went to the service and found the Delete ALL item in the actions menu
5
Overall graph flows
Observations
-
When validation errors occur, they are shown in the YAML editor, but nothing is shown in the notification drawer
-
Participants tried to click on the graph labels, but they are not selectable. Only shapes are selectable.
-
One participant asked if they could see the details of more than one thing at a time, to either compare or aggregate the data
-
When viewing details in the side details panel and changing context, the content in the side panel should scroll to the top
-
The title of the side details panel is not clear
-
Find/hide help modal changes size based on the selected tab
-
Find/hide help modal can be moved almost off the screen
-
Find/hide syntax is very complex
-
On the overview page, it takes a really long time for the first tooltip to be viewed. This seemed to factor in the initial failure of going to the graph of the bookinfo application
Suggestions
-
Redesign the side details panel
-
Add a notification when theirs a validation error
-
Scroll to the top of the details panel when the context is changed
-
Use PF tooltips when hovering over icons on the overview cards for (1) consistency and (2) timing
-
Keep find/hide help modal height consistent at all times, and add a vertical scrollbar as needed
-
Improve the find/hide help modal
-
Prevent the find/hide help modal from being moved off the screen
-
Investigate making graph labels selectable
-
Validate use case --> comparing or aggregating details of multiple objects
-
Consider a follow up service to the community
-
Verify that RBAC can disable all management functions in the UI
-
Should management workflows be de-emphasized
-
Explore infra as code. Where all config changes that are applied to a system are also kept in source control. This provides the ability to see all changes, who made them and when, and the ability to rollback
-
-
Consider additional testing in a more stable environment