Europe PMC is an open access archive of 46.5M scientific articles, that supports the open access policies of 37 funders. The primary users are life sciences researchers (academic and industry), clinicians and biocurators.
The challenge
When I joined the team they had limited knowledge of user-centred design, no user research insights or behavioural analytics and lots of assumptions.
The unique selling point of Europe PMC is the comprehensive linking of articles to other research ouputs including data, protocols and peer review reports. And the annotation of biological entities in the text (extracted using machine learning), for example proteins, organisms or gene-disease relationships.
The aim of the redesign was to better support users’ literature search goals and tasks and surface linked research outputs and value-added resources that are the unique selling point of Europe PMC.
I needed to understand whether our assumptions about user needs were correct and what to prioritise on the roadmap. My starting point was to plan and run a product discovery.
Product discovery
To understand the users’ goals, tasks and behaviours when literature searching I planned and conducted an in-depth research study in 2017 which included:
- A diary study with 13 participants: participants were asked to record their literature searches in a Google Docs template during one working week. I interviewed participants before and after their journalling activity.
- Usability testing: participants’ were asked about recent literature searches and completed tasks using the search tool they used most frequently, then repeated the tasks on Europe PMC.
- Competitor analysis: to learn what features were available in other literature search tools.
85 searches were analysed in the diary study. Five main search goals were identified including: exploring a new topic; finding new articles on a familiar topic, finding evidence within articles, finding a specific article and finding information about a methodology.
I identified differences in search behaviour between novice searchers, who expected search to work like Google, and information specialists who use Boolean logic, and build multi-line, complex search queries, for systematic reviews.
By observing how people searched I discovered what information was important to them, when deciding what articles to view or when scanning an article page. I defined a hierarchy of information for both the search results and article pages.
Critically, I found that our USP – useful data and resources linked from the article page -were not easily discoverable. They were buried in the interface and not labelled in a way that made sense to users.
I added all the insights to a Trello board with participant quotes by theme, for quick reference and sharing with the team.
I created a search experience map and a research report to share with the team and stakeholders.
I created a task model to break down the researcher goal: finding evidence to inform a research question. This helped to analyse the thought processes and behaviours of a researcher doing a literature search. Which in turn informed our search functionality and results page design.
I also created personas and search scenarios to help the team understand different motivations and pain points of researchers and biocurators at different career levels, when searching the literature.
The research insights led to a complete redesign of Europe PMC to better support user behaviours. I collaborated with the UX Designer to develop several different designs and we tested them iteratively with users.
Some of the design improvements included:
- Improved search functionality and search results design including changing the information hierarchy of results, labelling of different article types, changes to search filter functionality, search snippets and search suggestions.
- Article page redesign including:
- Combining the separate abstract and full text pages into one article page.
- Figure previews under the abstract.
- Changes to the categorisation of linked resources based on user needs. This resulted in new sections within the article page for Data (links to supplemental, cited or curated data), Reviews & Recommendations, Protocols and Citations and Impact (including data citations).
Delivery
I led an Agile team of 5 full stack and front-end developers, 1 QA / DevOps and the UX Designer to deliver the project.
I created and prioritised the product backlog of user stories in JIRA using an impact/effort framework. I facilitated sprint planning meetings, sprint retrospectives and release planning meetings. I worked with our QA to define our manual testing approach.
The project provided an opportunity to modernise the front-end tech stack which was previously based on the Java Wicket framework. After some technical investigation and evaluation we decided to use Vue.JS and a design system created in Storybook. The front-end was re-built in stages. We also used Rendertron to render all the site content without CSS for search engines.
I worked collaboratively with the new UX Designer and front-end developer on developing the page designs and interactions further, creating a design system and iterative testing with users.
Due to the scale of the project I planned a phased release schedule. We started with a beta release so that we could get early feedback on the new design and features before the general release.
Anonymous usage data from Europe PMC (e.g. user interactions with links, buttons or form elements), were tracked using Piwik. I defined several custom events and worked with the front-end developer to implement them. I analysed quantitative usage data and defined metrics to measure the impact of the redesign.
Outcomes
Satisfaction ratings for regular users increased by 11% from 66.8% in 2017 to 77.3% in 2020. Engagement by repeat users increased and the number of unique users per year tripled from 2017 to over 30 million.
Evidence from the user research shifted conversations with stakeholders and changed priorities on the roadmap.
“There is huge praise across the Scientific Advisory Board for the new features of the user interface and the approach to systematically and continuously collect user feedback that informs product development.”
Europe PMC Scientific Advisory Board 2020 report
Learnings
The biggest learning curve from this project was how to manage the release of a major site redesign. We tested the functionality of the site thoroughly, using bug bashes as well as manual and some automated testing by our team QA. I’ve written about bug bashes here.
I planned four different phases of release. As part of this schedule, a beta version of the site was released and linked from the site header so that we could get early feedback on the new design and functionality. The launch date of the new site was publicised on Twitter. We felt confident about our plans.
One of the underlying technologies we were using was Rendertron, which renders the entire site content for search engines on the fly, without CSS. It worked well on the staging environment. However when we moved the site to the live environment there was a performance issue. We had page loading speed KPIs to meet, so we couldn’t go publicly live until we had fixed the issue and improved page loading speed.
It took the technical team around a week to resolve the issuess at launch. I ran a retrospective with the team to reflect on what we could have done to avoid the issues. A key lesson learned was that better load testing with more realistic session data would have helped. We took that learning forward for future projects.
A new UX Designer joined the project after the initial user research had been completed. It was challenging for her to get up to speed with the findings of the research. Her understanding improved over time as she was involved in more of the usability tests.
References
Ferguson C, Araújo D, Faulk L, et al. Europe PMC in 2020. Nucleic Acids Research. 2021 Jan;49(D1):D1507-D1514. DOI: 10.1093/nar/gkaa994. PMID: 33180112; PMCID: PMC7778976.
PMC, Europe; Ide-smith, Michele (2017). Europe PMC Literature Search User Research Report. figshare. Journal contribution. https://doi.org/10.6084/m9.figshare.4789744.v1
PMC, Europe; Ide-smith, Michele (2017). Europe PMC Search Experience Map. figshare. Poster. https://doi.org/10.6084/m9.figshare.4789738.v1
Europe PMC Team. The new Europe PMC is here. 2019 Dec.