2021-10-12 P&E Agenda - Item 10 - Attachment 1 - Social Return on Investment Evaluation

5.3 Data validity
A fundamental part of any qualitative evaluation such as an SROI assessment is to ensure the data which is
relied upon is valid and reliable. Before the financial proxies where devised and the SROI was calculated, data
validity was tested, including through peer review.

The evaluators consider the collected and collated data has high validity based on:

     •   Highly informed participants – contribution from people with direct involvement in the Digital Hub
     •   Involvement since inception in 2019 – a range of respondents with mature and new experiences
     •   High participation rate and large data set (helping ensure the sample size is representative, with least
         bias) (this helped address potential weaknesses based on self-selection, rather than random sample)
     •   Neutrality of survey (participants were able to express negative and neutral values, not guided
         towards positive values)
     •   Anonymity and no recourse (the survey included an explanation that individual responses would not
         be provided to Council, ensuring participant anonymity to freely express their opinion)
     •   A suitable timeframe was allowed (3 weeks) to ensure sufficient time to participate (reducing
         potential impact and bias of early adopters)
     •   The data was provided directly to the independent evaluators, not through Council (no filtering of
         data)
     •   Some respondents chose to express no benefit (demonstrating confidence among respondents to
         openly express their views and experience)
     •   Survey design allowed respondents to skip questions (ensuring no ‘forced’ responses in order to
         progress and complete survey)
     •   Data from the three (3) key stakeholder types has been kept separate to maintain integrity.




12