Original source · versioned page text
2020-09-14 GC Agenda - Item 7 - Attachment 2 - Independent Scientific Review of Monitoring and Evaluation Plan
Text sections have not been matched to the original PDF pages. Use the original document to check page numbers.
Plans and drawings
This document appears to be plans or drawings. Its page text below holds only the words printed on the drawings, without the drawings themselves, so it may be jumbled or hard to follow. To see the plans as drawn, choose Original above or download the file (both need you to log in).
Log in to see the plans as drawn General Committee Agenda
14 September 2020
Item 7 - Attachment 2
Review of TNC monitoring and
evaluation Plan provided for the
‘Bring back the fish’ project
Prepared for Noosa Shire Council
August 2020
See the original
The page text is free to read. Viewing or downloading an original file needs an account.
Log in to see the originalSearchable page text hides email addresses. Original files are unchanged and may show email addresses.
Full text of the other 23 sections
Section 2
Acronyms
MER Monitoring, Evaluation and Reporting
TAG Technical Advisory Group
TNC The Nature Conservancy
USC University of the Sunshine Coast
2
Section 3
TABLE OF CONTENTS
EXECUTIVE SUMMARY .................................................................................................................................. I
1 INTRODUCTION ................................................................................................................................... 1
2 BACKGROUND ..................................................................................................................................... 1
2.1 NOOSA AND TNC PARTNERSHIP ................................................................................................................... 1
2.2 BRING BACK THE FISH PROJECT ..................................................................................................................... 2
2.3 MONITORING, EVALUATION AND REPORTING (MER) PLAN 2020-2022 ............................................................. 3
2.4 SCOPE OF REVIEW ...................................................................................................................................... 6
3 REVIEW METHODS ............................................................................................................................... 6
3.1 LIMITATIONS ............................................................................................................................................. 6
4 FINDINGS ............................................................................................................................................. 7
4.1 APPROPRIATENESS OF PROJECT GOALS TO THE FOCUS OF THE PARTNERSHIP ........................................................... 7
4.2 EFFECTIVENESS OF THE PROPOSED INDICATORS AND MONITORING METHODS ......................................................... 8
5 RECOMMENDATIONS........................................................................................................................... 8
5.1 RECOMMENDATIONS OR SUGGESTIONS FOR IMPROVEMENT TO THE MONITORING & EVALUATION PLAN ..................... 8
5.2 RECOMMENDATIONS FOR LONG-TERM MONITORING OF THE STATED GOALS AND OBJECTIVES OF THE PROJECT, BEYOND
THE END OF THE CURRENT AGREEMENT TERM ON 30 SEPTEMBER 2022. .......................................................................... 10
6 REFERENCES ...................................................................................................................................... 11
APPENDIX A: MERI REVIEW CHECKLIST ....................................................................................................... 12
Section 4
Executive Summary
Noosa Shire Council and The Nature Conservancy (TNC) have signed a three-year Partnership Agreement
that sets a bold vision for an innovative and regionally significant alliance to improve the health and
resilience of the Noosa River, and the industries and local communities that rely on its long-term wellbeing.
Under the terms of the Partnership Agreement, TNC is leading the Bring Back the Fish project that will pilot
the restoration of oyster reefs within the Noosa River estuary, to improve environmental health, engage
the Noosa community in conservation and support economic and community wellbeing.
TNC has prepared a Project Plan and Monitoring, Evaluation and Reporting (MER) Plan that has been
presented to Council. Council has requested a peer review of the plan. This report is provided to Council to
address the review requirements as specified:
1. Briefly comment on the appropriateness of the goals of the project in relation to the stated focus of
the partnership (the first four dot points identified on p4 of the Project Management Plan),
identifying any major gaps. There are 11 goals listed in the MER component of the plan (page 68-
73).
2. An assessment of the effectiveness of the proposed indicators and monitoring methods to
effectively measure the success of the stated Goals and Objectives of the plan.
3. Any further recommendations or suggestions for improvement to the Monitoring & Evaluation
Plan, including any further monitoring that might support determination of achieving success in the
project.
4. Provide recommendations for long-term monitoring of the stated Goals and Objectives of the
project, beyond the end of the current agreement term on 30 September 2022.
Appropriateness of project goals to the focus of the Partnership
The oyster reef restoration project is an appropriate project to commence collaborative restoration
activities under the Partnership agreement. The project will contribute to all of the focal areas of the
Partnership (but particularly focal area 1).
Effectiveness of the proposed indicators and monitoring methods
A good practice MER checklist was developed based on a number of reference documents. Twenty criteria
were then used to test the effectiveness of MER planning and design, monitoring, evaluation and reporting
components.
Sixteen good practice criteria were found to be met. Four of the criteria were partly met or unclear, but
none of these raised any significant concerns with the MER plan.
Overall, the review concludes that the MER plan is robust and comprehensive, and the proposed indicators
and monitoring methods likely to be effective. The MER Plan obviously reflects TNC’s experience with
similar oyster restoration projects in Australia and overseas.
Recommendations:
1. Resolve conflicting terminology and different versions of MER framework.
There are some inconsistencies in terminology between the Project Plan and the MER Plan. In particular,
the terms goals and objectives are used differently each document, and as used in the MER Plan have
multiple versions with some differences. This may simply reflect the timing and evolution of both
documents. We recommend that the relationship between the documents are clarified, that the
terminology is modified to avoid confusion and multiple versions rationalised.
i
Section 5
2. Incorporate recommendations for longer term monitoring as part of the final project report.
Legacy is the extent to which project impacts will continue over time after funding ceases (Roberts et al.,
2018). The Project Plan and MER Plan don’t explicitly address legacy but there is evidence that legacy has
been considered in the overall project design. Monitoring of project legacy (impacts) should be addressed
by incorporating recommendations for ongoing monitoring as part of the final project report, when the
performance of the current project and its MER are known and future resources and capability understood.
3. Consider further development of socio-economic monitoring and assessment methods.
Biophysical and ecological monitoring and assessment methods are well established and clearly described
in the MER Plan The monitoring and assessment methods provided for the socio-economic target are fairly
generic and could be further developed. If socio-economic outcomes are important then the project should
engage someone with appropriate skills to further develop this component of the MER Plan.
4. Collect, maintain and store provenance records of any species introduced to the site.
It is good practice for the provenance records of any species introduced to the site to be kept, if oyster spat
or mature oysters are translocated from elsewhere (directly or via a nursery).
5. Evaluate the partnership and share learnings
The Partnership agreement aspires to demonstrate a ‘leading example’ of a collaborative alliance that
‘achieves superior outcomes’. As well as learning about oyster restoration, the project should also evaluate,
and reflect on the learnings about the Partnership itself. Partners should contract a third party to facilitate a
partnership evaluation near the end of the current project (June 2022). Findings should inform future
partnering arrangements between Noosa Council and TNC, and may also be helpful to other organisations
considering similar arrangements.
Longer-term monitoring, beyond the life of the current project.
The objectives and design of future monitoring efforts beyond the current three-year project will depend
upon:
The success of the project (whether ecological and other targets have been met);
The likely trajectory of ecological measures beyond that time (whether restored oyster beds are still
developing, stabilised or at risk);
Whether additional oyster restoration sites or relevant estuarine management activities are
planned or ongoing;
Whether the partnership with TNC is expected to continue; and
The availability of resources, scientific and community monitoring capacity.
Recommendation 2 above suggests that the final report include a set of recommendations for longer-term
monitoring (e.g. in the phase 2 technical report). TNC, as the project manager, will be best placed to make
those recommendations at that time, using the information, models and other decision support tools
available from the Noosa project and other, related TNC restoration projects.
ii
Section 6
1 Introduction
Noosa Council entered into a three-year partnership with The Nature Conservancy (TNC), starting on 25 July
2019, with the primary project aim to restore oyster reefs in the Noosa River. The scope of the partnership
is outlined within the Alliance & Funding Agreement, which identifies clear outputs for the Bring Back the
Fish Project that is being implemented under the partnership agreement.
Project Deliverable 5 (of the partnership agreement) is that A Project Implementation Plan presented to
Noosa Council by TNC in conjunction with Council officers for its approval. This was reported to the Ordinary
Meeting of Noosa Council on 16 July 2020. The result was the following Council resolution:
“Approve the Project Management Plan (provided as Attachment 1 to the report) as
meeting the requirements of the Project Implementation Plan outlined in the Alliance &
Funding Agreement subject to Council appointing an independent scientific expert to
provide a review of the project’s Monitoring Evaluation and Reporting Plan. The review is
to include an assessment of the effectiveness of the methodology, project’s goals,
measures, outputs and outcomes within the context of ecological restoration and the
terms of the three year contract, and recommendations (if any) for improvement to be
included in the Monitoring and Evaluation Plan.”
As a result of this resolution, Noosa Council approached Eberhard Consulting to undertake and
independent review of the monitoring, evaluation and reporting plan. This document reports the results of
that review.
2 Background
2.1 Noosa and TNC Partnership
The scope of the partnership between Noosa Council and the TNC is outlined within the Alliance & Funding
Agreement (Noosa Shire Council and The Nature Conservancy, 2019). The focus of the Partnership is to:
“Improve the health and resilience of Noosa’s marine and estuarine environment through
innovative restoration and coastal resilience projects;
Capitalize on the expertise of The Nature Conservancy’s global networks and experience (through
knowledge brokering, mentoring, study tours and access to subject matter experts) to improve
Noosa River management and strengthen the long-term social, environmental and economic health
and resilience of the Noosa River and surrounding marine environment;
Demonstrate a leading example of a replicable, environmentally-focused, collaborative alliance that
achieves superior outcomes for the environment and local communities compared to existing river
and marine management models; and,
Increase government (State and Federal), private, industry and community support for restoration
and conservation-focused activities that improve the long-term social, economic and environmental
health and resilience of the Noosa River and surrounding marine environment.” (Noosa Shire
Council and The Nature Conservancy, 2019, p. 19)
1
Section 7
The Partnership agreement will be reviewed after three years (July 2022). The total operating budget is
$2.4M, inclusive of $1.2M from TNC and $1.2M from Noosa Council, with the expectation that TNC will
raise additional funds to support further oyster bed restoration.
An Executive Level Forum from each party is responsible for the cooperative arrangements under the
agreement. A Technical Advisory Group (TAG) (including independent experts) has been established to
advise on the shellfish restoration project.
The agreement commits to the restoration of oyster reefs in the lower estuary, and technical support for
the Noosa River Plan. Schedule A to the agreement sets out a series of objectives, deliverables (outputs),
timeframes, measures and delivery responsibilities. Each objective is further developed in the project
management plan for the Bring Back the Fish Project (The Nature Conservancy, 2020).
2.2 Bring Back the Fish Project
The goal of the Bring Back the Fish Project (‘the project) is to:
“improve the environmental health of the Noosa River Estuary through active
restoration and conservation activities that engage the Noosa Community in meaningful
conservation and support economic and community wellbeing” (The Nature
Conservancy, 2020, p. 10)
The Project has a number of objectives, each with multiple activities. The primary objective is the
restoration of shellfish ecosystem, initially at two locations and then (subject to successful establishment at
these sites) at other locations throughout the estuary. The objectives described in the Project Plan
(paraphrased details in brackets) are as follows:
A-1 Project establishment and management (project governance, management, communication
and reporting);
A-2 Site selection (identify restoration sites for phase 2 and 3);
A-3 Community engagement (strengthen community interest, support and participation);
A-4 Oyster ecosystem restoration; and
A-5 Noosa River Plan (provide technical and expert support to Noosa Council).
The Project Plan notes (on pg. 9) that more recent dialogue between TNC and Noosa Council has identified
additional initiatives to explore potential for seagrass restoration in Lake Cooroibah, and opportunities for
sustainable commercial and recreational fisheries in the Noosa River. These additional activities will
incorporated into the current objectives (A2 and A2).
The Project Plan specifies a set of deliverables, with timeframes, measures and responsible parties
identified. Table 1 below provides a summary version of the project objectives and deliverables.
Table 1. Project objectives and deliverables
Objectives Deliverables1
A1 Establish effective project governance, A1.1 Establish Technical Advisory Group (TAG)
management, communication and
A1.2 Appoint Project Manager
reporting sufficient to successfully
implement shellfish restoration A1.3 Project Implementation Plan and MER
project Plan
A1.4 Participate in public forums
A1.5 Annual project reports & final report
1
Deliverables have been paraphrased for brevity. For full description, refer to Table 1 in the Project Plan pg. 10-15
2
Section 8
Objectives Deliverables1
A1.6 Six monthly progress reports
A2 Identify suitable restoration sites for A2.1 Shellfish restoration suitability model
Phase II and Phase III and appropriate
A2.2 Permitting
shellfish substrate design and
configurations that minimize estuary- A2.3 Community consultation
user conflict whilst optimizing
rehabilitation
A3 Strengthen community interest, A3.1 Community consultation to identify
support and participation in Noosa volunteering opportunities
River restoration by establishing a
A3.2 Establish volunteering program
community volunteering program to
support oyster restoration
A4 Restore oyster ecosystems across the A4.1 Restore two sites
lower estuary
A4.2 Restore multiple additional sites
A4.3 MER study
A5 Provide technical and expert support A5.1 Workshop with Noosa Council
to Noosa for planning,
A5.2 Technical review x 5
implementation and evaluation
associated with appropriate elements A5.3 Study tours x 3
of the Noosa River Plan and other
A5.4 Conservation planning
coastal and marine management plans
A5.5 Access to TNC networks
A5.6 Promote project at conferences
A5.7 Promote project for further support
2.3 Monitoring, Evaluation and Reporting (MER) Plan 2020-2022
The MER Plan is embedded within the Project Plan, and provides substantial additional detail, including:
Project team and project governance;
Targets, goals and objectives;
Indicators and data collection methods and responsibilities; and
Evaluation metrics and reporting commitments.
Project targets, goals and objectives presented in the MER Plan are outlined in Table 2 overleaf.
3
Section 9
Table 2. Project targets, goals and objectives2
Targets Goals Objectives
ECOLOGICAL TARGET: Within the timeframe of the project build a Deploy substrate to meet tolerance outlined in design
1 resilient structure for the “Living Shellfish 1 and project outputs
To re-establish a self-
Ecosystem
sustaining population of S.
glomerata that will create a Within the timeframe of the project create a self- 2 Demonstrate survival of target shellfish
resilient reef consisting of sustaining shellfish population
3 Demonstrate natural recruitment of target shellfish
diverse biological 2
communities, fish Demonstrate a density of target shellfish similar to
4
populations and their pre- defined reference system
interactions.
Within the timeframe of the project demonstrate To demonstrate more fish post ecosystem restoration
3 5
the creation of habitat that benefits fish
Within the timeframe of the project demonstrate To demonstrate an increase in biodiversity
4 that construction of the reef enhances marine 6
biodiversity
Monitor water quality to demonstrate that Explore the capacity for the developing ecosystem to
5 construction of the reef enhances marine 7 reduce water turbidity
ecological health
SOCIO-ECONOMIC TARGET: Within the timeframe of the project demonstrate To qualitatively demonstrate benefits to the local
8
6 the benefit of shellfish reefs to local economy economy
To create opportunities for
the local community, 9 To demonstrate delivery of jobs
recreational users and
Within the timeframe of the project engage the 10 Demonstrate engagement by the local community
businesses in the Noosa
7 community in long-term stewardship of the
11 Demonstrate media engagement
shellfish reef
2
These are taken from the MER Plan Annex 1 pages 68-73, noting that there are inconsistencies with the summary table provided in Annex 2, page 74 (and Figure 6 of the
Project Plan pg.32).
4
Section 10
Targets Goals Objectives
Shire through shellfish reef Within the timeframe of the project improve Demonstrate improved knowledge and education for
12
restoration. knowledge, education and practical skills in marine individuals in marine restoration
restoration for practitioners, users and community
Demonstrate improved practical skills for individuals
8 members 13
to undertake marine restoration
Demonstrate involvement opportunities for
14
community members to undertake marine restoration
PROJECT EFFICIENCY Demonstrate responsible fiscal management 15 Financial investment been leveraged
TARGET: 9 throughout the project
16 Project delivered within budget
To deliver the project on
Demonstrate Technical Advisory input throughout Demonstrate effective technical advice provided
time and within budget 17
the project throughout project
and additional funding is 10
leveraged to support Restore each site to meet environmental conditions
18
project deliverables set in approvals
Demonstrate effective project management Demonstrate timely and effective project
11 19
throughout project delivery management and delivery
5
Section 11
2.4 Scope of Review
The purpose of this scope is to engage a qualified person to undertake a peer review of the plan, and
provide feedback in writing to Noosa Council, that meets the requirements of this Council resolution. The
review is to focus primarily on the Monitoring & Evaluation component only, which is pages 53-83 of the
overall Project Management Plan. The first component of the scope listed below does require an
understanding of the broader project.
The review should consist of the following components;
1. Briefly comment on the appropriateness of the goals of the project in relation to the stated focus of
the partnership (the first four dot points identified on p4 of the Project Management Plan),
identifying any major gaps. There are 11 goals listed in the MER component of the plan (page 68-
73).
2. An assessment of the effectiveness of the proposed indicators and monitoring methods to
effectively measure the success of the stated Goals and Objectives of the plan.
3. Any further recommendations or suggestions for improvement to the Monitoring & Evaluation
Plan, including any further monitoring that might support determination of achieving success in the
project.
4. Provide recommendations for long-term monitoring of the stated Goals and Objectives of the
project, beyond the end of the current agreement term on 30 September 2022.
3 Review methods
A simple monitoring and reporting checklist was developed as an assessment framework. The checklist
drew on reference materials for good practice monitoring, evaluation and reporting, including:
National standards for the practice of ecological restoration in Australia (Standards Reference
Group SERA, 2018);
National Natural Resource Management Monitoring, Evaluation, Reporting and Improvement
Framework (Australian Government, 2009); and the
Queensland Reef Water Quality Program Evaluation Framework (Roberts et al., 2018).
The checklist includes 20 questions that have been used to judge the adequacy and likely effectiveness of
the monitoring, evaluation and reporting plan provided by TNC.
Two documents form the primary source of evidence for the review:
Bring Back the Fish Project Management Plan 25th July 2019 to 30th September 2022 (which includes
the MER Plan) (The Nature Conservancy, 2020); and
Alliance & Funding Agreement between Noosa Shire Council and The Nature Conservancy (Noosa
Shire Council and The Nature Conservancy, 2019).
The completed checklist (Appendix A) documents the assessment process and supports the findings and
recommendations presented in the following sections.
3.1 Limitations
This review was conducted over a period of about three days. It is heavily reliant on the key documents
provided by Noosa Council (particularly the Project Management Plan). The reviewer is independent of the
project so free from conscious or unconscious bias, but also not familiar with the project details. The review
6
Section 12
takes a broad view of the overall monitoring and reporting plan, comments on its effectiveness and makes
recommendations to improve or complement this work. In this way, the review seeks to add value to the
work already undertaken by TNC.
4 Findings
This section presents the review findings, addressing the first two components of the project brief: the
appropriateness of the project goals to the focus of the Partnership, and the likely effectiveness of the
proposed monitoring and evaluation methods.
4.1 Appropriateness of project goals to the focus of the Partnership
The Partnership agreement sets a bold vision for an innovative and regionally significant alliance to improve
the health and resilience of the Noosa River, and the industries and local communities that rely on its long-
term wellbeing.
Four focal areas are identified3:
1. Health and resilience of Noosa’s marine and estuarine environment, through innovative restoration
and coastal resilience projects;
2. Improve Noosa River management by engaging with the expertise of TNC’s global networks and
experience;
3. Demonstrate a collaborative alliance that achieves superior outcomes to existing management
models; and
4. Increase government, private, industry and community support for conservation and restoration
activities etc.
There is some confusion in the hierarchy of goals and objectives in the Project plan and MER plan (where
those terms are used in several different ways). The review brief specifies the goals as those presented in
the MER Plan (and reproduced as Table 2 above). The 11 goals clearly articulate how the Bring Back the Fish
Project will deliver ecological, socio-economic and project efficiency targets associated with the oyster reef
restoration project.
The oyster reef restoration project is an appropriate project to commence collaborative restoration
activities under the Partnership agreement. The project will contribute to all of the focal areas of the
Partnership (but particularly focal area 1). Oyster reef restoration was prioritised at an expert workshop
held in 2014, and is a good fit with TNC’s expertise and international networks.
The oyster reef restoration project is the core of the document hierarchy of Partnership agreement, Project
Plan and MER Plan. Focal area 3, the alliance itself, is enacted by the restoration project, but is not explicitly
evaluated. Later recommendations (as part of the MER review) suggest that the partners should also
actively consider how they will assess, evaluate and learn about the partnering initiative itself.
3
The focus areas are paraphrased. The full text is provided in the section 2.1
7
Section 13
4.2 Effectiveness of the proposed indicators and monitoring methods
A good practice MER checklist was developed based on a number of ‘good MER practice’ reference
documents. Twenty criteria were then used to test the effectiveness of MER planning and design,
monitoring, evaluation and reporting components presented in the Project Plan and MER Plan. .
Sixteen of those criteria were found to be met. Four of the criteria were partly met or unclear, but none of
these raised any significant concerns with the MER plan. Details of the assessment are provided as
Appendix A.
The four criteria that were considered to be partly met or unclear are:
7 Are the proposed indicators and monitoring methods clearly described? Met for biophysical
and ecological indicators. Qualitative methods for social indicators could be improved.
9 If any species are introduced to the site, will provenance records be kept? Unclear.
15 Are assessment methods identified or described? Met for biophysical and ecological indicators.
Assessment methods for socio-economic indicators could be improved.
20 Are processes in place to identify and share learnings into future partnership and beyond?
Partially met. The partnering process and outcomes should also be evaluated.
Six recommendations are made to improve the MER Plan.
Overall, the review concludes that the MER plan is robust and comprehensive, and the proposed indicators
and monitoring methods likely to be effective. The MER Plan obviously reflects TNC’s experience with
similar oyster restoration projects in Australia and overseas.
5 Recommendations
We offer the following recommendations to Noosa Council and TNC for their consideration in moving
forward with the Bring Back the Fish project. The recommendations were developed in response to the
review findings (Appendix A).
5.1 Recommendations or suggestions for improvement to the Monitoring &
Evaluation Plan
Recommendation 1. Resolve conflicting terminology and different versions of MER framework.
There are some inconsistencies in terminology and relationship between the Project Plan and its MER Plan.
This may simply reflect the timing and evolution of both documents. The MER Plan should:
Clarify the relationship between the MER Plan, the Project Plan and the Partnership Agreement (the
MER Plan appears to address A1-A4 but not A5 in the Project Plan
Modify the terminology between the Project Plan and the MER Plan to remove confusion. Table 3
highlights that that some terms (goals and objectives in particular) are used inconsistently between
the Project Plan and the MER Plan. There are many versions of project terminology that can be
adopted. What is desirable is that the Project Plan and MER Plan use a consistent terminology that
allows the relationship between the Project Plan and the MER plan to be clear and unambiguous.
Rationalise the multiple versions of goals and objectives that occur in the Project and MER Plan
(Annex 1, Annex 2 and Figure 6) (refer footnotes below).
Table 3. Suggested changes to terminology
8
Section 14
Document Current terminology Alternative Explanation
(# of items) terminology
Project Plan Project goal (1) Project goal An overarching project goal is
appropriate.
Objectives (5) Project phases or These are operational elements
elements or phases of the project
Deliverables (21) Deliverables or Outputs of the project activities.
outputs
Measurable Measures These are not outcome measures,
outcomes (27) but output measures.
MER Plan Targets (3) Targets The targets are clear and helpful.
They could also be considered
outcomes.
Goals (11)4 Objectives Steps on the way to the targets /
outcomes, achieved in the life of
the project
Objectives (19)5 Performance More detail on measures and
measures assessment.
Indicators Indicators
Recommendation 2. Incorporate recommendations for longer term monitoring as part of the final project
report.
Legacy is the extent to which project impacts will continue over time after funding ceases (Roberts et al.,
2018). The Project Plan and MER Plan don’t explicitly address legacy but there is evidence that legacy has
been considered in the overall project design. Monitoring of project legacy (impacts) should be addressed
by incorporating recommendations for ongoing monitoring as part of the final project report, when the
performance of the current project and its MER are known and future resources and capability understood.
Recommendation 3. Consider further development of socio-economic monitoring and assessment
methods if desired.
Biophysical and ecological monitoring and assessment methods are well established and clearly described.
The monitoring and assessment methods provided for the socio-economic target are fairly generic.
Evaluation of qualitative socio-economic data is not described, and many of the socio-economic indicators
have no clear benchmarks (e.g. knowledge, skills and educational objectives). Measures such as the number
of community events, number of attendees, and number of groups engaged are suitable output measures,
but don’t assess community awareness, use and satisfaction, for example.
The degree of effort expended on socio-economic monitoring and assessment should reflect the priority
that Noosa Council and TNC place on the socio-economic outcomes of the project, and what can be learnt
from that. If community engagement and outcomes are considered important then I recommend the
4
11 goals are listed in Annex 1 of the MER (pg. . .69-73) but only 10 goals are shown in figure 6 of the Project Plan
(pg.32)
5
19 objectives are listed in Annex 1 of the MER (pg. 69-73) but only 13 are shown in Annex 2 (pg.74) and 15 are shown
in figure 6 of the Project Plan (pg. 32)
9
Section 15
project engage someone with appropriate skills to further develop this component of the MER Plan. If that
work proceeds it should complement the SEQ Report Card measures of satisfaction and beneficial use of
the Noosa River.
Recommendation 4. Collect, maintain and store provenance records of any species introduced to the site.
It is good practice for the provenance records of any species introduced to the site to be kept, if oyster spat
or mature oysters are translocated from elsewhere (directly or via a nursery). Refer to the National
standards for the practice of ecological restoration in Australia (Standards Reference Group SERA, 2018, p.
22).
Recommendation 5. Evaluate the partnership and share learnings
The Partnership agreement aspires to demonstrate a ‘leading example’ of a collaborative alliance that
‘achieves superior outcomes’. The Project Plan, through regular reporting cycles, includes processes for
reflecting on the implementation of the oyster restoration project.
In addition to the oyster restoration project, Objective A5 within the Project Plan includes a range of actions
to support Noosa Council, promote the project and develop conservation plans. A5 is not currently
considered within the MER Plan (which is focussed on the oyster restoration) but actions will be reported
(Table 1).
As well as learning about oyster restoration, the project should also evaluate, reflect, learn and share about
the partnership arrangements – including the activities described in Objective A5, but also more broadly, to
assess and reflect on the partnering process and outcomes. We recommend that the partners contract a
third party to facilitate a partnership evaluation near the end of the current project (July 2022). Findings
should inform future partnering arrangements between Noosa Council and TNC, and may also be helpful to
other organisations considering similar arrangements.
5.2 Recommendations for long-term monitoring of the stated Goals and
Objectives of the project, beyond the end of the current agreement term on
30 September 2022.
Monitoring of project legacy (impacts) should be addressed by incorporating recommendations for ongoing
monitoring as part of the final project report, when the performance of the current project and it’s MER are
known and future resources and capability understood (refer Recommendation 2 above). Monitoring
efforts beyond the current three-year project will depend upon:
The success of the project (whether ecological and other targets have been met);
The likely trajectory of ecological measures beyond that time (whether restored beds are still
developing, stabilised or at risk);
Whether additional oyster restoration sites or relevant estuarine activities are planned or ongoing;
Whether the partnership with TNC is expected to continue; and
The availability of scientific and community monitoring capacity and resources.
Recommendation 2 above suggests that the final report include a set of recommendations for longer-term
monitoring. Bagget et al. (2014, 2015) recommend a set of four oyster reef metrics three water quality
variables for all oyster restoration projects. Monitoring objectives will be determined by the factors above,
however. TNC, as the project manager, will be best placed to make those recommendations at that time,
using the information, models and other decision support tools from the Noosa project and other TNC
restoration projects.
10
Section 16
6 References
Australian Government. (2009). NRM MERI framework. Australian Government Natural Resource
Management Monitoring, Evaluation, Reporting and Improvement Framework. Canberra:
Commonwealth of Australia.
Baggett, L. P., Powers, S. P., Brumbaugh, R., Coen, L. D., DeAngelis, B., Greene, J., . . . Morlock, S. (2014).
Oyster habitat restoration monitoring and assessment handbook. Arlington, VA: The Nature
Conservancy.
Baggett, L. P., Powers, S. P., Brumbaugh, R. D., Coen, L. D., DeAngelis, B. M., Greene, J. K., . . . Breitburg, D.
L. (2015). Guidelines for evaluating performance of oyster habitat restoration. Restoration Ecology,
23(6), 737-745.
British Ecological Society. (2014). A guide to data management in ecology and evolution. London: British
Ecological Society.
Conservation Measures Partnership. (2013). Open standards for the practice of conservation.
Fitzsimons, J., Branigan, S., Brumbaugh, R. D., McDonald, T., & zu Ermgassen, P. S. E. (2019). Restoration
guidelines for shellfish reefs. Arlington VA: The Nature Conservancy.
Gann, G. D., McDonald, T., Walder, B., Aronson, J., Nelson, C. R., Jonson, J., . . . Liu, J. (2019). International
principles and standards for the practice of ecological restoration. Restoration Ecology, 27(S1), S1-
S46.
Noosa Shire Council and The Nature Conservancy. (2019). Alliance and Funding Agreement. Noosa: Noosa
Shire Council and The Nature Conservancy Ltd.
Roberts, A., Eberhard, R., & Dickson, M. (2019). Queensland Reef Water Quality Program 2019 Evaluation.
Melbourne: Natural Decisions.
Roberts, A., Eberhard, R., Dickson, M., Park, G., & Waterhouse, J. (2018). Queensland Reef Water Quality
Program Evaluation Framework. Melbourne: Natural Decisions.
Standards Reference Group SERA. (2018). National standards for the practice of ecological restoration in
Australia (2nd ed.): Society for Ecological Restoration Australasia.
The Nature Conservancy. (2020). Bring Back the Fish. Restoration and conservation of oyser ecosystem in
the Noosa River estuary. Project Management Plan. July 2019-September 2022. Melbourne: The
Nature Conservancy Australia,.
11
Section 17
Appendix A: MERI review checklist
Source of Assessment and
# Prompt Notes
information recommendations.
Planning and design
1 Are the project There is some confusion in terminology between the Project Plan Project Generally met.
goals/objectives SMART? and the MER Plan. Plan
Clarify and align
(Strategic, Measurable,
The Project Plan described five project objectives (A1-A5) (pg. 18). Pg. 10-15, terminology between
Appropriate, Realistic and
Table 1 (pg. 10-15) provides a series of deliverables (or outputs) 18. 32 Project Plan and MER Plan
Time bound)
that relate to each of the five objectives. This section of the (Recommendation 1).
MER Plan
project document is very operationally focussed. The project
deliverables are clear and meet SMART criteria but are at the level Pg. 68-74
of outputs.
The MER Plan provides additional detail in relation to Phase II of
the oyster restoration project, which seems to address objectives
A1-A4. Objective A5 relates to TNC providing technical and expert
support to Noosa Council’s Noosa River Plan, and is not addressed
in the MER Plan.
The MER Plan also lists goals and objectives, but used in a different
way. The goals and objectives listed in the MER are SMART.
2 Is there an explicit program There is not an explicit program logic but the use of the MER MER Plan Met
logic or similar that clearly terminology for TNC shellfish restoration projects (based on the pg.59
identifies: Open Standards for the Practice of Conservation)6 are reasonably
Outputs;
6
https://cmp-openstandards.org
12
Section 18
Source of Assessment and
# Prompt Notes
information recommendations.
Outcomes consistent with the program logic approach adopted by Australian
(biophysical, Natural Resource Management programs7
ecological,
Outputs have been identified in the Project Plan (as
community
deliverables).
engagement); and
Outcomes are identified as objectives in the MER Plan.
Impacts
(environmental, Impacts are identified in both the targets and goals of the
social and economic) MER Plan.
3 Are there any underpinning A comprehensive operational risk assessment is provided in the Project Assumed to be met.
assumptions or potential Project Plan. Plan. Table
externalities/disruptions/risks 10. Pg. 40-
The MER Plan proves an assessment of reputational risks
that should be considered 44
associated with the restoration project.
further (e.g. monitored)?
MER Plan
No specific biophysical, ecological or socio-economic risks are
Pg. 51
identified in the Project Plan or MER Plan.
It is reasonable to assume that these risks have been considered
and addressed through the following activities:
TNC Restoration Scoping Study (2015);
USC experimental trials (2018-20);
Other TNC oyster restoration experiences in Australia and
internationally; and
Oversight of the TAG and additional experts.
4 Have project legacy issues Legacy is extent to which project impacts will continue over time Project Largely met.
been identified and after funding ceases (Roberts et al., 2018). The Project Plan and Plan and
addressed (including
7
https://www.mdba.gov.au/sites/default/files/archived/proposed/NRM-MERI-Framework.pdf
13
Section 19
Source of Assessment and
# Prompt Notes
information recommendations.
monitoring where MER Plan don’t explicitly address legacy but there is evidence that MER Plan Monitoring of project legacy
appropriate) e.g. site legacy has been considered in the overall project design through: (various) (impacts) should be
management, ongoing addressed by incorporating
Partnering arrangements with Noosa Council (A5);
monitoring, knowledge and recommendations for
skills transfer post the Engagement and capacity building with community ongoing monitoring as part
current project? volunteers (A3); of the final project report,
when the performance of
Scientific support (prior projects, current project roles,
the current project and its
TAG oversight) (various); and
MER are known and future
Promotion of the project for ongoing support and funding resources and capability
(A5.7). better understood
(Recommendation 2).
Monitoring of project impacts beyond the life of the current
project has not been addressed. This is best addressed by
incorporating recommendations for ongoing monitoring as part of
the final project report, when the performance of the current
project and its MER are known and future resources and capability
better understood.
5 Have appropriate national The Project Plan commits to meeting a range of best practice Project Met
and international standards standards including: Plan and
for the project and its MER MER Plan
Restoration guidelines for shellfish reefs (Fitzsimons et al., 2019);
been identified (various)
National standards for the practice of ecological restoration in
Australia (Standards Reference Group SERA, 2018);
International principles and standards for the practice of
ecological restoration (Gann et al., 2019);
Best practice oyster restoration and the Society for Ecological
Restoration (SER) global guidelines;
Open Standards for the Practice of Conservation (Conservation
Measures Partnership, 2013); and
14
Section 20
Source of Assessment and
# Prompt Notes
information recommendations.
A guide to data management in ecology and evolution (British
Ecological Society, 2014).
In addition, project documentation references a number of TNC
standard operating protocols and methods including the following:
Methodology for habitat suitability modelling;
Oyster habitat restoration monitoring and assessment handbook
(Baggett et al., 2014); and
Shuck don’t Chuck for shell recycling.
6 Is there a formal MERI plan Yes. The MER plan is comprehensive. Refer specifically to Annex 1 MER Plan, Met
that documents measures, which describes indicators, monitoring methods, assessment Annex 1 pg.
evaluation and reporting, metrics, benchmarks, frequency, timing and responsibilities. 52-88.
including timing and
responsibility?
Monitoring
7 Are the proposed indicators Mostly yes. Refer to Annex 3. Ecological methods are well MER Plan, Met for biophysical and
and monitoring methods established and clearly described. Socio-economic methods are Annex 3, ecological indicators.
clearly described? fairly generic. pg. 75-82 Qualitative methods to
understand community
awareness, use and
satisfaction are not clearly
described but could be
developed
(Recommendation 3).
8 Will restoration and Yes. Project output measures are clearly described in Table 1 Project Met
engagement activities be (Project Plan) and are included draft reporting template (Annex 5). Plan Table
logged and reported? Quantitative measures of engagement activities (e.g. number of
15
Section 21
Source of Assessment and
# Prompt Notes
information recommendations.
presentations, number of volunteers etc) are included in Annex 1 1, pg 10-15,
(MER Plan). Annex 5
MER Plan,
Annex 1 pg.
52-88.
9 If any species are introduced It’s not entirely clear whether the project will rely on natural Project Unclear.
to the site, will provenance recruitment or whether juvenile native oysters will be pre-seeded Plan, pg.
Recommendation 4. If any
records be kept? onto recycled shells in a hatchery before deployment or adult 54, 77
species, including oyster
stock will be translocated. If hatchery stock are used it’s not clear
stock, are introduced to the
to the reviewer where that will be sourced from (it may be local).
site from elsewhere,
Regardless, it is good practice for the provenance records of any
provenance records should
species introduced to the site to be kept.
be kept.
10 Do the indicators track Yes outcome measures are being tracked. MER Plan, Met
progress to the project Annex 1 pg.
objectives outcomes and 52-88.
targets?
11 Has baseline ecosystem Yes. Previous investigations include TNC Oyster Restoration Project Met
monitoring been undertaken? Scoping Study and USC experimental project 2018-2020 that Plan
collected important ecological information.
Pg. 8
12 Has an appropriate ecological The nearest known reference rock oyster system is in Moreton Project Met
reference condition system Bay. In the absence of a suitable local reference site, the project Plan pg. 31
or conceptual framework will use an interim target reference system, based on an ecological
MER Plan
been identified? model of the rock oyster ecosystem developed using research8 and
pg. 63
local monitoring results.
8
Refer McLeod et al. (2019) and MacAfee et al. (in review).
16
Section 22
Source of Assessment and
# Prompt Notes
information recommendations.
13 Are arrangements for data The MER Plan commits to following good practice standards as MER Plan No details but commitment
storage and management outlined in the British Ecological Society’s (2014) guide. No further pg. 67 to comply with standards.
clear? detail provided. See below for longer term housing and access.
14 Will the data be accessible to Data and information will be openly shared, proactively released, MER Plan Met
the scientific community and licensed to promote re-use and housed on the Australian Ocean pg. 67
the public through an open Data Network (https://portal.aodn.org.au )
access databases such as
Atlas of Living Australia
(https://www.ala.org.au )
Evaluation
15 Are assessment methods A Before-After-Control-Impact (BACI) design with two adjacent MER Plan Met for biophysical and
identified or described? control sites for each location (one for seagrass and one for sandy ecological measures.
pg. 63-65
substrate).
Assessment methods for
The evaluation of biophysical and ecological measures is well socio-economic indicators
described and will incorporate basic statistics, time series, could be further developed
comparative analysis and restoration scores. if desired. Refer
recommendation 3.
Evaluation of qualitative socio-economic data is not described, and
many of the socio-economic indicators have no clear benchmarks
(e.g. knowledge, skills and educational objectives).
16 Have key project decision Yes. The project is split into 3 distinct phases to allow adjustment: MER Plan Met
points and their information pg. 55-56
Phase 1 external experimental project to test viability and local
needs been identified?
responses9;
Phase 2 shellfish reef restoration at 2 sites10; and
9
Completed by the University of the Sunshine Coast prior to this project.
10
The focus of the Project Plan and MERI (and this review).
17
Section 23
Source of Assessment and
# Prompt Notes
information recommendations.
Phase 3 additional restoration activities, subject to phase 2 results, Project
availability of additional sites and community licence11. Plan Table
1, pg. 10-15
The MER Plan addresses Phase 2.
Annual and six-monthly project reporting. The annual project
report is prepared by TNC, endorsed by the TAG, and presented to
NSC for approval.
17 Is an independent party Evaluation of the Queensland Reef Water Quality Program MER Plan Sufficient.
evaluating the project? recommends independent evaluation as highly desirable for large pg. 57
projects ($500K-$10M) or projects of medium risk (Roberts et al.,
2019).
An independent evaluation is not proposed for this project. A
Technical Advisory Group (TAG) has been established. The role of
the TAG includes providing expert advice to support project
implementation. The TAG provides a degree of independent
oversight of the project’s implementation and its evaluation.
Reporting
18 Are reporting commitments Yes, 6-monthly, annual and final project reports are specified in Project Met
to project partners and the Project Plan. Table 1 specifies the content of the reports and a Plan Table
investors clear (what, when, process for the TAG and then Noosa Council to review and 1, pg. 10-15
how)? endorse the report.
MER Plan
A reporting template is provided as Annex 5. pg. 83-88
19 Are reporting commitments 27 groups of project stakeholders from government, the Project Met
to community stakeholders community, private sector and research organisations have been Plan pg. 16
clear (what, when, how)? identified.
Project
A Project Communication Plan is included as Appendix 1. Plan pg. 47-
50
11
To be considered after the completion of phase 2.
18
Section 24
Source of Assessment and
# Prompt Notes
information recommendations.
Attendance at community and partner events, engagement and MER Plan
consultation and media engagement will all be tracked and pg. 71
reported.
20 Are there processes in place The Project Plan, through regular reporting cycles, includes Project Partially met.
to identify and share processes for reflecting on the implementation of the oyster Plan pg. 13-
Processes to build on TNC’s
learnings into future restoration project. 15
networks and experience,
partnership work and beyond
The Partnership agreement aspires to demonstrate a ‘leading and promote Noosa River
(including other shellfish
example’ of a collaborative alliance that ‘achieves superior Plan and the restoration
restoration projects)? Pg. 4
outcomes’. project are addressed in the
Project Plan (objective A5)
Objective A5 within the Project Plan includes a range of actions to
Pg. 10 but are not part of the
support Noosa Council, promote the project and develop
evaluation plan.
conservation plans. A5 is not currently considered within the MER
Plan (which is focussed on the oyster restoration) but actions will The partnering process
be reported on (Table 1). should also be evaluated
and learnings shared. Refer
As well as learning about oyster restoration, the project should
recommendation 5.
also evaluate the partnership to support learning and
improvement.
19