Europe PMC is an open, life sciences literature discovery tool that supports the open access policies of 37 funders. The primary users are life sciences researchers (academic and industry), clinicians and biocurators.
The challenge
When I joined the team they had limited knowledge of user-centred design, no user research insights or behavioural analytics and lots of assumptions.
Articles in Europe PMC are linked to research outputs such as supporting data, code, protocols and other resources including peer review reports. Biological terms and concepts such as gene-disease relationships and biological samples are text mined from the full text and highlighted on the article text.
The aim of the redesign was to better support users’ literature search goals and tasks and surface linked research outputs and value-added resources that are the unique selling point of Europe PMC.
My role as User Researcher
To understand the users’ goals, tasks and behaviours when literature searching I planned and conducted a generative research study in 2017 which included:
- A diary study with 13 participants: participants were asked to record their literature searches in a Google Docs template during one working week. Working with the Junior UX Designer, we interviewed the participants before their journalling activity. We then analysed their diary study responses and interviewed participants again at the end of the week. I collated all the search queries in one spreadsheet and categorised the types of search.
- Usability testing: participants’ were asked about their literature searching behaviour and recent literature searches. They were then asked to complete tasks related to recent literature searches using the search tool they use most frequently, and then to repeat the tasks on Europe PMC.
- Competitor analysis: to learn what features were available in other literature search tools.
85 searches were analysed in the diary study. Five main search types were identified including: exploring a new topic; finding new articles on a familiar topic, finding evidence within articles, finding a specific article and finding information about a methodology.
I added all the insights to a Trello board with participant quotes by theme, for quick reference and sharing with the team.
I created a search experience map and a research report to share with the team and stakeholders.
I created a task model to break down the researcher goal: finding evidence to inform a research question. This helped to analyse the thought processes and behaviours of a researcher doing a literature search. Which in turn informed our search functionality and results page design.
I also created personas and search scenarios to help the team understand different motivations and pain points of researchers and biocurators, when searching the literature.
The research insights led to a complete redesign of the article and search pages. I collaborated with the UX Designer to develop several different designs and we tested them iteratively with users.
Some of the design improvements included:
- Improved search results design including labelling of different article types and enhanced search filter functionality
- Article page redesign including:
- Figure previews under the abstract
- A data section containing links to supplemental and cited or curated data
- Re-organising links to useful resources elsewhere under sections that made more sense to users including reviews, protocols,
- A citations and impact section including data citations
My role as Product Manager
Once the designs had been through a few rounds of usability testing, I led an Agile team of 5 full stack and front-end developers, 1 QA / DevOps and the UX Designer to deliver the project.
I created and managed the product backlog of user stories in JIRA and facilitated sprint planning meetings, sprint retrospectives and release planning meetings. I worked with our QA to define our manual testing approach.
The project provided an opportunity to modernise the front-end tech stack which was previously based on the Wicket framework. After some technical investigation and evaluation we decided to use Vue.JS and a design system created in Storybook. The front-end was re-built in stages. We also used Rendertron to render all the site content without CSS for search engines.
I worked collaboratively with the new UX Designer and front-end developer on developing the page designs and interactions further, creating a design system and iterative testing with users.
Due to the scale of the project I planned a phased release schedule. We started with a beta release so that we could get early feedback on the new design and features before the general release.
Anonymous usage data from Europe PMC (e.g. user interactions with links, buttons or form elements), were tracked using Piwik. I defined several custom events and worked with the front-end developer to implement them. I analysed quantitative usage data and defined metrics to measure the impact of the redesign.
Outcomes
Satisfaction ratings for regular users increased by 11% from 66.8% in 2017 to 77.3% in 2021. Engagement by repeat users increased and the number of unique users per year tripled from 2017 to over 30 million.
Evidence from the user research shifted conversations with stakeholders and changed priorities on the roadmap.
“There is huge praise across the Scientific Advisory Board for the new features of the user interface and the approach to systematically and continuously collect user feedback that informs product development.”
Europe PMC Scientific Advisory Board 2020 report
Learnings
The biggest learning curve from this project was how to manage the release of a major site redesign. We tested the functionality of the site thoroughly, using bug bashes as well as manual and some automated testing by our team QA. I’ve written about bug bashes here.
I planned four different phases of release. As part of this schedule, a beta version of the site was released and linked from the site header so that we could get early feedback on the new design and functionality. The launch date of the new site was publicised on Twitter. We felt confident about our plans.
One of the underlying technologies we were using was Rendertron, which renders the entire site content for search engines on the fly, without CSS. It worked well on the staging environment. However when we moved the site to the live environment there was a performance issue. We had page loading speed KPIs to meet, so we couldn’t go publicly live until we had fixed the issue and improved page loading speed.
It took the technical team around a week to resolve the issuess at launch. I ran a retrospective with the team to reflect on what we could have done to avoid the issues. A key lesson learned was that better load testing with more realistic session data would have helped. We took that learning forward for future projects.
A new UX Designer joined the project after the initial user research had been completed. It was challenging for her to get up to speed with the findings of the research. Her understanding improved over time as she was involved in more of the usability tests.
References
Ferguson C, Araújo D, Faulk L, et al. Europe PMC in 2020. Nucleic Acids Research. 2021 Jan;49(D1):D1507-D1514. DOI: 10.1093/nar/gkaa994. PMID: 33180112; PMCID: PMC7778976.
PMC, Europe; Ide-smith, Michele (2017). Europe PMC Literature Search User Research Report. figshare. Journal contribution. https://doi.org/10.6084/m9.figshare.4789744.v1
PMC, Europe; Ide-smith, Michele (2017). Europe PMC Search Experience Map. figshare. Poster. https://doi.org/10.6084/m9.figshare.4789738.v1
Europe PMC Team. The new Europe PMC is here. 2019 Dec.