Skip to content

Remote Usability Tests - August/September 2021

Introduction đź‘‹

SprintCraft is an internal application whose goal is to provide an intuitive way to plan our sprints, and to monitor how our hours and budgets are allocated. We would like to improve the SprintCraft user experience so that both internal, and external users enjoy using it. But first we need your help! That’s where this usability test comes in. Our goal is to get feedback from you so that we can determine what needs to be improved and how. Thank you for taking part!

A few things before you begin

  • Please follow the test instructions below and record yourself and your screen whilst doing so
  • You can use any service you like for the recording. The 14 day free trial on the Loom Business Plan is a good option if you don’t already have an account (note: Loom’s "Starter" plan only allows recordings up to 5 minutes long, so please use the free trial on the Business Plan instead)
  • Please read each test instruction out loud before completing it
  • Speak your thoughts out loud so that we know what you’re thinking/feeling through each step
  • When answering a question, try to elaborate on your thoughts. We want as much information as possible!
  • Don’t worry if you struggle to complete any of the tasks or if you don’t know the answer to a question; our goal is to uncover which parts of the app are confusing
  • Your recording should be at least 15 minutes long
  • When your recording is ready, please upload it to your Jira ticket (videos will contain real budget data so can't be made public)
  • Ping @ali_hugo on your Gitlab issue in the list below if you have any questions
  • Nizar
  • Giovanni
  • Paulo
  • Piotr
  • Cassie

Test Instructions

Please complete the steps below and answer the associated questions. Remember to read each test instruction out loud before completing it:

  1. Create a new account on SprintCraft by adding the +test suffix to your OpenCraft email address
  2. Log in to SprintCraft
  3. Describe your first impression of the interface
  4. Evaluate the sustainability of OpenCraft during the month of July 2021. What do the numbers displayed in the table mean to you?
  5. Give a brief explanation of each of the column titles in the Budget dashboard
  6. Evaluate the time spent on the “Opencraft - Firefighting Management Role” during July 2021. What adjustments would you suggest for this account’s budget over the coming months?
  7. Next, evaluate the time spent on the "OpenCraft - Instances DevOps" during July 2021. What adjustments would you suggest for this account?
  8. As an epic owner or client owner, which parts of the Budget dashboard would be most meaningful to you?
  9. How would you view a month-by-month breakdown of an account’s budget for the selected period?
  10. How do you think these account budgets are set in SprintCraft?
  11. Is there anything you particularly like, dislike, or find confusing about the OpenCraft Sustainability dashboard (the dashboard used to display the sustainability of the OpenCraft team as a whole)?
  12. Is there anything you particularly like, dislike, or find confusing about the OpenCraft Budget dashboard (the dashboard used to display all OpenCraft budgets)?
  13. Do you think it is useful to see the Sustainability and Budget dashboards at the same time? Why, or why not?
  14. Evaluate your cell’s commitments for the upcoming sprint
  15. Which columns of the “Commitments for Upcoming Sprint” dashboard would be the most useful to you?
  16. Would you use the “spillover” and “new work” toggles above the cell commitments dashboard? If so, what would you use them for?
  17. Give a brief explanation of each of the column titles in your cell’s Sustainability dashboard
  18. Is there anything you particularly like, dislike, or find confusing about the cell Sustainability dashboard (the dashboard used to display the sustainability of your cell)?
  19. Briefly evaluate your individual commitments, sustainability, and budgets (if you're not a member of this cell, you may evaluate any random cell member)
  20. Would you find this individual overview useful? Why, or why not?
  21. Create a new sprint for your cell
  22. Did you find it easy to create a new sprint? Why, or why not?
  23. Imagine that you have just completed a sprint for your cell and need to provide reasons for any spillovers. How do you think the current process of adding a [~crafty]: <spillover>(.*)<\/spillover> comment could be improved?
  24. Have you ever used the "clean sprints feature"? Do you think it is useful? Why, or why not?
  25. Which part of the SprintCraft app would you expect to use most often, or find to be the most useful to you?
  26. Is there anything else you’d like to mention about the SprintCraft app?
  27. One last thing before you stop your recording; we need a distinct, SaaS-friendly, Googleable name for the SprintCraft app. The team has already made some suggestions. Do any of the below names catch your eye? (positive and negative feedback welcome!)
    • AgileCraft
    • AgilePlanner
    • AsSprint (for painless sprints)
    • Circuit
    • Loop
    • Oasis (OpenCraft Asynchronous Sprint Is Simple)
    • Oasis (Opencraft Asynchronous SImple Sprints)
    • RemoteSprintPlanner
    • ScrumCraft
    • Scrumtious
    • Sprint Coach
    • Sprint Keeper
    • Sprint Master
    • SprintCraft
    • SprintFromHome
    • SprintVector
    • Sprintability
    • Sprintagility
    • Time Planner

Thank you! đź‘Ź

A big THANK YOU for taking part in this usability test! Your insight will help to make the SprintCraft application the best it can be.

Please remember to add a link to your video on your Jira ticket (you can find the link in the introduction above).