Findings

Findings by theme:

The review included assessment of regional SDEs' academic, clinical and industry engagement. 

Regional SDEs are working to establish data sharing agreements and engage stakeholders but face challenges such as securing support from all NHS trusts and GP practices. 

Key lessons and good practices identified across the SDE Network include: 

• building on existing data sharing agreements
• using existing governance frameworks
• seeking standardised data provider agreements
• defining clear data controllership
• using 'bridging agreements' for transparency (such as to ensure that GPs are fully aware of the SDE and purposes when data is shared)
• employing strategies such as centralised negotiation for efficiency
• the SDE being embedded at a strategic level with healthcare and academic partners, for instance through representation at board level
• developing clinical engagement and a data provider engagement strategy
• using clinical advocates who both promote and influence SDE design
• producing a joint controller relationship for agreement across an SDE footprint with a documented set of onboarding procedures
• clinicians and academics forming part of the SDE senior leadership team. 

Proactive stakeholder engagement through strategic embedding, using clinical advocates, establishing user feedback mechanisms, and creating formal advisory committees were also highlighted as effective approaches.

The review evaluated the status of Patient and Public Involvement and Engagement (PPIE) across the SDE Network. All regions delivered PPIE as part of their core activity. Some regions showed strong PPIE integration, with public insights influencing decision-making, while others had developing or mixed practices. Some regions exhibited limited or insufficient PPIE, lacking integrated insights and dedicated resources. 

The review identified learning and good practice in PPIE, highlighting the necessity of integrating public insights into decision-making processes, allocating dedicated resources, and ensuring transparency in how public input is best used. Effective PPIE extends beyond formal groups to include diverse community engagement, supported by metrics for monitoring and continuous improvement. 

Good practice observed included:

• embedding public members in governance structures
• co-producing PPIE strategies
• using diverse and deliberative engagement methods
• forming partnerships with organisations like Healthwatch
• actively using public insights in decision-making
• transparently publishing engagement findings and how public input shapes decisions
• developing actionable public insights
• dedicated PPIE resources and structural embedding of PPIE
• inclusive engagement strategies targeting marginalised groups and targeted outreach to seldom-heard communities
• activities that build public understanding of data use for research
• creating safe environments where public members feel empowered to challenge thinking

The review revealed significant variations in the operational maturity and processes of members of the SDE Network, as well as in data discovery methods, and technical infrastructure. Some had well-established processes while others had processes that still required testing with live projects. Common themes included the need to transition manual processes to scalable, automated systems, address recruitment challenges for permanent staff, and ensure robust validation processes for users and organisations.

Findings highlighted the importance of testing operational models with live projects for effectiveness and scalability, implementing dedicated systems like customer relationship management (CRM) for efficient pipeline management, and using existing resources to avoid duplication.

Good practice observed includes:

• reusing existing resources
• establishing clear operating procedures
• using legacy experience
• maintaining comprehensive data listings with robust validation policies

Key learning

Testing is crucial: several SDEs had documented operating models that remained untested due to a lack of live projects. This risks potential scalability and effectiveness issues once projects commence.

Automation gaps: while many regions use CRM systems, others use manual processes for project management and project onboarding which limit efficiency and scalability.

Data discoverability and accessibility: there is a consistent challenge in ensuring full data accessibility and detail on national platforms like the HDRUK Gateway. Some data has been published but is not fully accessible due to pending provider agreements, necessitating project-by-project negotiation.

Clear governance: in some SDEs a functional operating model is needed to unite disparate legacy systems under a single SDE governance structure.

The review of the SDEs identified good practice across the SDE Network and found a number of shared challenges on the journey towards a federated network.

SDEs highlighted good practice such as adopting a data mesh architecture, using existing expertise, using cloud platforms for scalability, implementing robust security measures, use of standardised data formats like Observational Medical Outcomes Partnership (OMOP) and fostering collaboration for interoperability.

Key learning included:

• the challenges of relying on external data curation
• issues with inconsistent and fragmented infrastructure
• the risk of non-scalable platforms
• the inefficiency of manual data processing

SDEs also demonstrated the importance of scalable architectures, balancing data utility with privacy, addressing interoperability challenges for multi-SDE collaboration, and ensuring legal compliance alongside technical readiness.

Good practice identified amongst the SDEs included:

• automating processes for efficiency
• use of standardised data formats like OMOP
• implementing robust multi-layered security architectures
• using cloud platforms and third-party tools for scalability and management
• establishing comprehensive data flow processes
• fostering collaboration and shared learning
• securely handling researcher-provided data through standardised processes

All regional SDEs are working with providers to develop greater depth and breadth of data available, informed by researcher priorities.

SDE areas of good practice include prioritising structured data and using existing national and regional datasets while planning for unstructured data later. Effective metadata management involves automating publishing via APIs, linking to authoritative sources, using user feedback to refine information, and providing user portals for data discovery.

Early engagement with data providers, establishing data linkage solutions with common pseudonymisation, and developing reusable open-source tools are important collaborative strategies.

Challenges include:

• API issues
• reliance on manual processes for metadata management
• uneven geographic data coverage
• dependencies on provider staff for data delivery
• difficulties in federating data across different local systems
• ensuring comprehensive data coverage across a region
• securing appropriate commercial agreements for data usage

SDEs have developed governance structures which involve patients and the public in decision-making and oversight. 

The review found that best practice for data governance includes:

• establishing a single data access committee with patient and public representation
• adopting standardised national documentation (data access agreements and data access request form, for example)
• proactively securing required legal and ethical approvals for processing patient data
• defining clear controller roles and responsibilities
• using overarching framework agreements to manage data sharing

Key learning indicates that project-by-project agreements and ambiguous decision-making processes may create inefficiency at project level, and that complex controller arrangements require significant coordination.

As the SDE Network develops its products and services for researchers, it is also increasingly pro-active in raising awareness of its role in enabling secure access to NHS data for research purposes.

The review highlighted key learning and best practices for business development, sustainability, and marketing. Effective strategies include establishing structured pipelines with robust management and tracking systems, developing clear and affordable pricing models, securing dedicated resources for managing leads, using existing relationships, and adopting or aligning with network-wide commercial artefacts. 

Learning indicates that premature marketing is inefficient without a clear proposition. Formal strategies are essential for long-term sustainability, reactive approaches limit growth, lack of resources hinders growth, financial modelling is essential for long-term sustainability, and targeted market segmentation is more effective than broad marketing efforts. A diversified income model beyond academic projects is crucial.