Challenge
The Royal College of Occupational Therapists (RCOT) is the professional body and charity representing over 36,000 occupational therapy professionals across the United Kingdom.
Like most purpose-led organisations, RCOT manages a sprawling content estate built over many years, spanning clinical guidance, continuing professional development resources, and event information. This high-value material sat alongside transactional pages, creating a cluttered ecosystem where critical assets were easily buried.
The site search was a problem. It was keyword driven, intolerant of typos and synonyms, and reliant on members already knowing the exact phrasing of the content they wanted. High-value resources were being buried. Members were giving up.
The pressure intensified as Google rolled out AI Overviews, causing charity sector organic traffic growth to plummet and domain rankings to drop by 28.9%. Visitors now arrived with new habits, expecting synthesised answers to conversational questions. Staying the same meant risking member frustration, disengagement, and a complete breakdown in digital resource discovery.
Solution
We adapted the initial proposed ‘hybrid search’ approach to break up the rollout into two distinct phases.
-
The (AI-enabled) Foundation: connecting an LLM and making simple enhancements to search experience to quickly enable more intuitive natural language searches that better understand intent and enhance accuracy.
-
The ‘Hybrid’ (UI-enabled) Experience - introducing better filters (reflecting more intuitive internal tags) and considering deeper UX / UI enhancements to onward journeys that enhance search relevance and drive specific actions.
Within this project we focused on launching and optimising phase 1, and collaborating with RCoT to learn from this experiment to further optimise for phase 2.
At this stage we have applied AI to the search function to test its ability to deliver accurate results based on a user’s intent.
Impact
To test the value of this change, we included a measurement framework that would tell us, in honest terms, whether the change was worth making. This incorporated concurrent A/B tests comparing results from the new AI search function against the old version. So far we have identified three metrics constantly moving in the right direction:
-
Average click position in search results fell from 4.6 to 3.1 - this means fewer clicks needed to find the right thing.
-
Click-through rate per search rose from 34 percent to 38 percent - when members searched, they were more likely to find something worth clocking.
-
Search exit rate fell from 80 percent to 74 percent - fewer members leaving frustrated.
Now that we have established a baseline for AI-enabled search to understand user intent we are moving into the second phase. Now our focus is on improving onward journeys via UX and UI. Our aim is to deliver search results that are not just accurate, but highly relevant and actionable.
To achieve this we are building an interface designed to actively guide users toward completing high-value tasks.
