Motivation
- To evaluate the Eclipse PASS interface against established usability principles before public launch
- To identify and prioritize usability issues by their real impact on users and effort required to fix them
- To produce actionable, developer-ready recommendations rather than abstract critique
The team
I conducted this heuristic evaluation independently as the UX researcher, working closely with stakeholders, the IT development team, and accessibility experts to review the system before launch. Because Eclipse PASS is a collaborative open-source platform, some fixes were resolved internally by UVA’s own developers, while others required coordination with the system’s external development partner, Johns Hopkins University.
Methodology
The evaluation used Jakob Nielsen’s ten usability heuristics as a structured framework — covering principles like visibility of system status, error prevention, and consistency and standards — applied against nine core user tasks, including viewing and creating submissions, associating publications by DOI, submitting on behalf of someone else, and managing grant-linked submissions.
Each identified issue was rated on two independent scales:
- Severity (S0-S4): how much the issue affects users, from no impact to critical, task-blocking failures
- Ease of Resolving (E0-E3): how much effort the fix requires, from a quick tweak to a significant backend/architectural change
This dual-rating approach matters because severity and effort don’t move together — some of the most user-damaging issues were also the cheapest to fix, and prioritizing on severity alone would have missed that.
Findings — Critical issues (S4)
1. Journal search inconsistency (S4, E2, External) Typing an even number of characters into the journal search field returned “no results” and silently failed to search at all — while an odd number of characters worked correctly and triggered the expected autocomplete. This meant journals with exactly even-length names (like “SAGE”) appeared not to exist at all on a user’s first attempt, a deeply misleading failure with no visible cause to the user.
2. Page refresh error (S4, E2, External) Refreshing the page during a submission triggered a vague, technical error message (“Cannot read properties of undefined”) that, when dismissed, immediately reloaded the page and re-triggered the same error — trapping the user in an infinite loop with no way out except abandoning the submission via the browser’s back button.brary at all yet.
Findings — Significant issues (S3)
3. Unclear messaging in error messages (S3, E1, External) The generic “Something went wrong” error lacked any actionable guidance: no explanation of what happened, why, or how to get help. I drafted and got approval from the ACE division’s Content department for replacement language with clear structure: what happened, who to contact, and what happens next, using line breaks to separate each piece of information rather than a dense paragraph.
Findings — Moderate issues (S2)
Eight issues fell into this category, each a real but non-blocking source of friction:
- Proxy submission radio buttons too close together (Internal, remediated) — violated Gestalt’s principle of proximity, risking users selecting the wrong option
- Form navigation bar’s current step blocked out rather than highlighted — prevented users from confirming which step they were on
- Redundant Back/Next buttons after selecting a grant — two identical button pairs on screen serving the same purpose
- Unclear “Cancel” vs. “Abort” language — a confusing pair of contradictory-sounding buttons, worsened by an “Abort” button colored like a safe, primary action. I proposed replacement copy (“Discard Draft” / “Yes, discard draft” / “Continue editing”), approved by the ACE division’s Content department
- Difficult-to-parse Review page — vertically-centered section headers made it hard to tell where one section ended and the next began; file type shown only as an unexplained icon
- “Add new” author button lacking visual affordance (Internal, remediated) — didn’t look interactive, and the instructional text explaining it didn’t reappear for a second entry
- Unclear file upload guidance (External) — no indication of supported file formats, and the “at least one file required” error appeared as an easy-to-miss popup in the corner rather than in the page body
Findings — Minor issues (S1)
Two lower-severity but still real issues: a row quantity modifier with no visual indication it was interactive (recommended renaming “Rows” to “Rows per page” with a dropdown arrow), and a navigation bar that didn’t highlight the current tab (Dashboard/Grants/Submissions), leaving users unsure where they were in the system.
Summary of all 12 issues
| No. | Issue | Severity | Ease |
| 1 | Journal search inconsistency | S4 | E2 |
| 2 | Page refresh error loop | S4 | E2 |
| 3 | Unclear error messaging | S3 | E1 |
| 4 | Proxy radio button proximity | S2 | E1 |
| 5 | Nav bar step blocked out | S2 | E1 |
| 6 | Redundant grant nav buttons | S2 | E1 |
| 7 | Cancel/Abort clarity | S2 | E1 |
| 8 | Hard-to-parse Review page | S2 | E1 |
| 9 | Hidden “Add new” author affordance | S2 | E2 |
| 10 | File upload clarity | S2 | E2 |
| 11 | Row quantity modifier | S1 | E1 |
| 12 | Nav bar tab indication | S1 | E1 |
Conclusion and Outcome
This evaluation gave the PASS project team a structured, prioritized roadmap ahead of launch, distinguishing issues UVA could fix immediately in-house from those requiring coordination with Johns Hopkins University as the external development partner. Of the 12 issues identified, 3 (the proxy radio buttons, the redundant navigation buttons, and the hidden “Add new” affordance) were confirmed fixable internally and remediated before launch, alongside a fourth pending a CSS-only fix. The two most critical issues: the journal search failure and the page-refresh error loop, were flagged for deeper investigation given their potential to fully block a user’s task, ensuring the most damaging problems received the most urgent attention rather than being addressed in the order they were found.
← Back to Selected Work