Organization: UN Children’s Fund
Closing date: 01 Jun 2018
UNICEF LAC Regional Office
Terms of Reference
Evaluating the Attribution and Contribution of UNICEF LAC in high/middle income countries
Level of experience: Junior [ ] 1-3yrs Medium [ ] 4-10yrs Senior [ X ] more than 10yrs
Location: LACRO, Panama
Language(s) required: Fluency in Spanish and English.
Closing date for applying to this consultancy: May 31, 2018
Estimated start date of this consultancy: 13 June 2018 Estimated end date: 13 December 2018
Background and Rationale
The majority of countries where UNICEF LAC functions are high-middle income countries. Country offices have strategically designed their country programmes focused on upstream work, often through a combination of policy and programme advocacy as well as modelling interventions and contributing to pilot projects to be scaled up, targeting institutional and behavior changes in societies towards better results for children. Country programmes are typically designed to create the most appropriate enabling environment, through supporting adequate child focused policies and programmes, and promoting legal frameworks laws in line with international standards, and improving learning and capacity of government institutions. Piloting and modelling are implemented in the field to create evidence and engage local partners to identify solutions for children to be replicated or scaled up. In addition, UNICEF is frequently invited to provide top-expert advice on child-related policies without a local or demonstrative project. For example, on social protection, child-focused budgeting, child protection legislation, systems strengthening, etc. This type of efforts is complex and highly demanding in terms of time, investment and capacity, considering that it usually requires an inter-sectoral approach to address key systemic barriers that impede the full realization of children’s rights. This remains a challenge to demonstrate results that can be directly attributed to UNICEF and measure the actual contribution of UNICEF to high level results.
Results for children is the main ingredient for the definition of UNICEF country programmes. For policy work, the time span of five years of a country programme is usually not enough to demonstrate results at the level that it is required. Implementing policy work is a long-term investment to achieve results at large scale and often the influence of different factors may overshadow the casual relationship between the actual achieved changes and the changes in the life of children.
A critical but frequently unanswered question remains – are these programmes actually achieving changes in children’s lives? Regular monitoring is often focused on reporting indicators at the output level, which aim at reducing bottlenecks and barriers, such as generation of evidence policy reports, or elaboration of a toolkits and guidance documents for policy makers, that are impeding institutional or behavioral changes. At the outcome level, results reflect contributions to strengthening systems, and results reports are focused on institutional changes, such as monitoring system in place, or a draft law/policy. Results also include the generation of models, which are the systematic representation of lessons learned and recommendations to scale up validated pilot projects to be used by government officials for national or sub-national programs design and implementation. Whether or not these outputs and outcomes are changing the lives of children as intended by the UNICEF strategic plan is not evident through the current monitoring and evaluation mechanisms in practice. Outputs (attribution) and outcomes (contribution) are part of a theory of change that in the long run is expected to change the lives of children which will eventually translate into children survive and thrive, all children learning, all children protected from violence and exploitation, a fair chance for all children, and all children live in secure environments.
The Role of Evaluation
In line with the UNICEF’s Evaluation Policy (2013), the evaluation function in UNICEF LAC aims at helping to improve its performance and results, by supporting organizational learning and accountability.
UNICEF LAC seeks to enhance the strategic role of Evaluations to understanding the context in which advocacy, policy advice, and models and scaled up pilots can reach results at scale. Policy advocacy is present at all levels.
In the last two years, UNICEF LAC evaluations have been mainly focused on assessing interventions at the outcome and output level. Although, some evaluations had an indication of impact results, evaluation analysis of long term and sustained results has been restricted. There is limited evidence that can validate the hypothesis (theory of change) that the activities, outputs and outcomes of a UNICEF LAC intervention is reaching (or contributing to) long term and sustained changes that can have an impact. Even when evaluations have assessed national policies and programmes (jointly with Government counterparts), the contribution of UNICEF to impact level results has been hardly assessed.
Evaluating the attribution of specific interventions and its contribution of UNICEF at the impact level is of high importance, as UNICEF is accountable to bring results at scale and contribute to high level results described in the new UNICEF Strategic Plan, and ultimately to the Sustainable Development Goals. In the context of high/middle income economies, the contribution of UNICEF may even be more “diluted” leaving the question of the value-added role of the organization in these contexts. Evaluation, through adequate evaluation methods, can help clarifying this role, making the causal inferences clear, even for advocacy strategies, and identifying the linkages all the way from activity to output to outcome to impact (which in many cases is not linear as it is portrayed) – and therefore offering a response to the question are these programmes actually achieving changes in children’s lives?, What are the conditions that caused reaching those changes?.
Considering the nature of the upstream work in the Latin America and the Caribbean region in middle/high income countries, the evaluation methods package UNICEF LAC is looking for is threefold: i) to assess the attribution of the effect of pilot / demonstrative projects; ii) to assess the contribution of those pilots / demonstrative projects to scaled up interventions; and iii) to assess policy advocacy and technical assistance at high levels.
Evaluation methods such as those based on causal inference of Theory of Change (theory-based approaches to evaluation) can show the complete spectrum of how a single advocacy intervention and/or a combination intervention that includes advocacy, technical advice, and implementation pilots is making a difference in children’s lives – where other stakeholders are also contributing. Specific evaluation methods can assess the effects of different upstream components a combination of often non-linear actions directly attributable to UNICEF and others pitched as contribution as part of a broad spectrum of a policy advocacy work, where UNICEF is one in many other stakeholders.
11. UNICEF LAC aims at positioning evaluations as a critical tool to assess the contribution /attribution of UNICEF work to results at scale, especially in the context of high/middle income transitioning economies. UNICEF LAC is looking to improve organizational learning through an evaluation framework that can assess if an intervention has produced or influenced high level results and determine the role of UNICEF in these contexts.
12. The evaluation framework intends to provide a guide for country offices and the LAC regional office (or other regional offices) to design, implement, and report results for children resulting from UNICEF LAC interventions, including one or a combination of these interventions: i) evidence generation through modelling and implementation of pilot projects, ii) policy advocacy of bring models to scale; and iii) Policy advocacy / technical assistance upstream work.
13. The evaluation framework will include a technical evaluation note that includes a proposed methodology to assess the attribution/contribution of UNICEF. The proposed methodology will then be tested in a country office – in an area of work related to the LAC Regional Priorities. Specifically, one country office evaluation will be used as case study to test the proposed methodological framework to assess the attribution/contribution of UNICEF in a specific context. The case study will help be used to fine-tune the technical note.
14. In addition, the framework will also include practical tools for applying the proposed framework. Practical tools will include sample ToRs/inception report components, checklist for a rapid evaluability assessment for applying the proposed methodology, and checklist to assess the technical soundness of the evaluation reports. Additionally, the evaluation framework will include costing the application of this methodology in the UNICEF LAC context and the identification of financial options/arrangements to fund these type of evaluations, considering financial limitations in the UNICEF LAC context
15. The objective of this assignment is to prepare a framework to de used to evaluate the attribution/contribution of UNICEF work in middle and high income countries in Latin America and the Caribbean Region. The framework will have a strong methodological focus, but at the same time realistic approach to work of UNICEF in the region.
16. The evaluation framework will include:
17. The assignment will be managed by the UNICEF LAC Regional Office, in coordination with the Office of Evaluation, and the strong involvement of one LAC country office participating in the case study. The final product will be a co-creation effort, and will be shared within the region and at the global level.
18. This framework will be built collaboratively and consultatively with organizations and institutional bodies with demonstrated expertise on this topic, and strengthening collaboration with organizations and governments with similar work. UNICEF will partner with development organizations (e.g World Bank, Inter-American Development Bank, DFID), academic institutions (e.g. University of California-Berkeley), and governments with strong evaluation national capacity (e.g. Mexico), that can share evaluation frameworks used to assess upstream work for better results on the ground.
19. A reference group will be established to act as a sounding board to the elaboration of this framework. The Reference Group is expected to provide inputs to the inception and draft reports. The reference group will be made of UNICEF LAC staff, staff at the Evaluation Office, and external Evaluation experts/advisor.
20. This work is expected to be implemented in six months.
21. The anticipated budget will be distributed as follows:
Evaluation Technical Note (draft and final versions)
Case study- Country Office evaluation (evaluation and lessons learned)
Approval of Products
22. The steering committee, led by the Regional M&E Specialist and Monitoring Specialist will review products delivered by the Consultant and will approve their final version. The consultant or team of consultants will inform the M&E Specialist and Monitoring Specialist of any remarks on the product, within five (5) working days after the product has been delivered.
23. Products will be reviewed by the Reference Group. The Reference Group will provide feedback within two (2) weeks of receiving the deliverable. The consultant will then edit the deliverable as needed and will provide a revised version within one (1) week after receiving the feedback.
24. The required consultant or team of consultants (a team of two three people can be considered) will be a qualified professional with the following characteristics:
26. UNICEF reserves the right to withhold all or a portion of payment if performance is unsatisfactory, if outputs are incomplete, not delivered of for failure to meet deadlines
26. All materials developed will remain the copyright of UNICEF and UNICEF will be free to adapt and modify them in the future.
27. Individual consultant or a small team of consultants (two or three) can apply for this work.
28. Applicants will submit: i) a technical proposal, and ii) an economic proposal, as follows:
A technical proposal should include:
A CV and/or P11 for he individual or each of the team members
Profile of the consultant or the team
A brief technical proposal (5 pages max.) describing methods that will be considered for the work, methodology to complete each of the components of the assignment, timeline, and workplan.
b. An economic proposal including a total fee to conduct this consultancy.
29. Incomplete proposals (technical and economic) will not be considered.
How to apply:
UNICEF is committed to diversity and inclusion within its workforce, and encourages qualified female and male candidates from all national, religious and ethnic backgrounds, including persons living with disabilities, to apply to become a part of our organization. To apply, click on the following link http://www.unicef.org/about/employ/?job=512977
Read more here:: RELIEFWEB CAREERS INT