Consultancy assignment Ritu midline survey in Netrokona district, Bangladesh
Terms of Reference | November 2018 | Simavi & Maastricht University
BACKGROUND OF RITU IMPACT EVALUATION
The Ritu program is an integrated program of numerous alliance partners (Simavi, DORP, BNPS, Red Orange and TNO) and is funded by the Embassy of the Kingdom of the Netherlands. The alliance partners have joined forces to improve menstrual health (MH) among girls and increase their educational achievement. The Ritu program targets 10-15 year old girls in grades 6 to 8 at schools in Netrokona district, using a multi-level approach. This approach includes the establishment of MH-friendly WASH facilities in schools, training teachers and training parents. Through this approach, the program recognizes the complexity that underlies safe Menstrual Health Management, e.g. knowledge, attitudes, gender equality, taboos, etc. Existing evidence about the impact of poor MH on attendance, drop out and academic performance, as well as psychosocial outcomes is scarce, and hardly any studies are large and rigorous enough to show statistically significant effects of programs. Together with the Ritu alliance partners, a research alliance (hereinafter referred to as ‘UM’) of Maastricht University, The United Nations University Maastricht Economic and Social Research Institute on Innovation and Technology the Netherlands, are currently conducting a large impact study to evaluate the extent to which the program improves the academic achievements and well-being of girls.
In order to evaluate the impact of the Ritu program, UM designed an impact evaluation named the Ritu study. It is a clustered randomized controlled trial on the impact of the Ritu program. The baseline survey data collection took place between 20 January and 20 February 2017. A survey has been designed that captures, amongst others; general wellbeing, school attendance during menstruation, MH-related knowledge, attitudes and practices. The baseline survey has been administered to approximately 3881 girls.
PURPOSE OF THIS CONSULTANCY
The purpose of this consultancy is to manage all aspects of data collection, data entry and data management for the midline of this rigorous impact evaluation. The sample is the exact same as during the baseline, 3881 girls (age 12-15) in Netrokona District spread over 149 schools (with an average of 26 girls per school). Similar to the baseline, the midline survey will be administered at school. Only for girls who are no longer frequenting school, the survey will be administered in their homes. The midline survey modules are similar to the baseline survey modules, and the duration of the survey is approximately 60 minutes.
UM will provide the survey firm team with the exact survey in English. The midline survey is very similar to the baseline survey (+/- 70%) and therefore the Bengali version of most questions is already available. The survey firm team is expected to translate the remaining parts of the English survey in Bengali.
The midline survey is planned to start the first week of March 2019 and is estimated to be completed at the end of April 2019 (+/- 7 weeks). Enumerator training will be held at the end of February.
It is important to note that a large team of experienced female enumerators is needed, as the surveying consists of a large group of young girls on sensitive issues, in a very short period of time. The budget of this survey for 3881 girls in the aforementioned time span is maximally 45,000 Euros incl. VAT. and inclusive of all other costs e.g. transportation, accommodation, subsistence and insurance.
Please note that the midline survey is conducted by the survey firm team as an independent research, meaning that participating schools and respondents are not to be reminded of the Ritu program in any way through the midline survey. Please keep this in mind when designing the introduction text to schools. Hence, there will be limited collaboration with Ritu implementing partners to get their support in organizing the survey activities.
The survey firm team will report to the UM and the Ritu program team, most importantly to Marianne van Diggele (PMEL Advisor Simavi, Amsterdam, The Netherlands). The survey firm will be under the direct supervision of Khabir Ahmed (Monitoring and Evaluation officer Ritu program, Simavi, Dhaka Bangladesh) and Lidwien Sol (Principal Investigator, Maastricht University, The Netherlands).
BACKGROUND OF THE PROJECT AND THE EVALUATION
This midline survey is part of a larger impact evaluation, a clustered randomized controlled trial. 149 schools are part of the study and they are all located in the Netrokona district of Bangladesh. Each school has been randomly assigned to treatment arm 1, treatment arm 2, or a control group. Hence, we have 3 groups of girls that receive different (combinations of) interventions:
- Treatment 1: a set of school level interventions
- Treatment 2: both the set of school level interventions and community level interventions
- Control: no interventions at all
This random assignment allows us to compare outcomes such as attendance, drop-out and academic performance, as well as psychosocial outcomes (e.g. psychosocial wellbeing, self-efficacy, stigma), and MH-related knowledge, attitudes and practices.
The entire survey firm team (incl. enumerators) will be blind to the treatment condition of the schools.
DELIVERABLES FOR MIDLINE DATA COLLECTION FEBRUARY – APRIL 2019
The survey firm team is expected to manage the following aspects of the survey collection, entry and management.
1. Ethical clearance documentation
The survey firm team is responsible for acquiring all permissions necessary for conducting the survey in due time. This study has received ethical approval from the Erasmus University Rotterdam IRB and we have permission of all participating schools to survey girls during classes. The granting of ethical approval of the survey of the Directorate of secondary education (on time to start the survey process on February 9th, 2019) is the responsibility of the survey firm. The survey firm team is also responsible for adhering to local formalities and obtaining any required permits related to the survey implementation (contacting the local education offices to get written approval to access schools) as well as survey team health and accident insurance, salary, taxes, and others as necessary. Through the course of obtaining ethical clearance, the survey firm team should also identify and secure respondent compensation packages/gifts according to local custom if necessary.
- Adapted research protocol and informed consent forms to be signed by head of school level and the girls that are surveyed and identification of respondent compensation as per local customs
- For when surveying at home instead of at school: informed consent forms to be signed by parents and the girls
- Evidence of ethical clearance, insurance and permits needed to implement the survey.
2. Translation of questionnaire
The survey firm team will be provided with the midline survey in excel format, developed for the use of (preferably) ODK software, in English. The survey firm team will translate new or revised sections of the survey, adapt any components from the baseline survey that are necessary to accurately capture the intended information of the 13- year old girls and deliver an initial translation of the entire survey in Bengali. The UM team will be involved in this process, and notified of any proposed changes.
The final translated survey will be sent to the Ritu team and the UM team at least 7 days before the commencement of enumeration training. Allowing for enough time to check the translation.
- Initial translation of full questionnaires into local language
- Adapted survey to fit local context
- Final local language questionnaires
- Final corresponding English questionnaires
3. Pre-testing of the electronic version of the questionnaires
The survey firm team is responsible for scripting the questionnaires, for use on the tablets using (preferably) ODK software. The scripted version should extensively be checked for consistency, especially the skip patterns.
DELIVERABLE (3): A written plan of scripting and pre-testing of the electronic version of the questionnaire.
4. Preparation of Field Sampling Plan
The Ritu team will provide the survey firm team with a detailed list of the sample; a list of all 3881 girls to be surveyed, including their names, schools, and if necessary (if they are not in school) also their contact details upon request. The UM team will provide a replacement list. Based on this, the survey firm team will develop the field sampling plan, which must detail:
- Coding strategy in order to ensure each individual has a unique code for data entry that merges seamlessly across data sets, and matches the unique code used in the baseline survey
- Define geographical survey clusters of schools based on the provided details of the girls in the sample
- Detailed plan for the census in the treatment and control area
- Strict guidelines for selecting replacement individuals
DELIVERABLES (4): A written Field Sampling Plan approved by the UM team.
5. Final Fieldwork Plan
The field procedure plan should outline in detail all aspects of the fieldwork to be conducted by the survey firm team via electronic (eg. tablet) data collection. The Fieldwork Plan includes:
- Composition of a field team
- Number of female enumerators
- Number of field-supervisors
- Number of field data entry agents
- Qualifications, training of each member of the team
- Expected tasks, responsibilities and schedule of delivery of each member of the team
- The expected time each team will spend in a geographical cluster (Interviews have to be conducted with school girls in or around the school (or at home) creating a location where interviewees feel their privacy is sufficiently protected)
- Transportation and lodging logistics
- Sample Control File for data collection in each geographical cluster
- Protocols and procedures for addressing data inconsistencies/miss-reporting when identified.
- If applicable: Protocols for Computer Assisted Field Entry (CAFE), whereby questionnaires are captured and validated immediately following tablet surveys, and the results transmitted back to the field teams to conduct quality checks as needed (after each day), i.e. daily backchecks.
This Fieldwork Plan should be presented to the UM team for comments, and revised as necessary prior to commencing fieldwork. The survey firm team must then implement the survey, adhering as closely to the plan as conditions allow. As field conditions dictate significant changes to these plans, the survey firm team’s Field Supervisors are obliged to inform the UM team via the survey firm team’s management, in the form of a written report or progress report.
DELIVERABLES (5): A written Fieldwork Plan approved by UM team.
6. Final Data Entry Program
The survey firm team must develop (or adapt) a robust data entry program in CS-Pro or another suitable program approved by the UM team.
The adapted program must be robust:
- Adapt data entry range and consistency checks to values appropriate for the country context.
- To the greatest extent possible, the data entry program should conduct range and consistency checks.
- Violations of these checks should lead to an immediate and transparent message sent to the enumerator, along with a practical method for correcting keypunch errors, or overriding and documenting any answers that violate the range and consistency check rules.
- The program should allow valid open-ended and “other” textual responses outside of the response options provided in the questionnaire
- Variable names generated by the program should correspond clearly and logically to the question labels used in the questionnaire.
- Coding strategy in order to maintain consistent, unique identifiers for respondents for matching longitudinal data
- Written data entry protocol for data entry agents detailing program
- Final data entry program adapted for the local questionnaires
- The dataset dictionary with all variables labelled and defined
7. Recruitment of Qualified Field Staff
The personnel requirements for this project include:
Field Team: The number of field teams is to be determined in consultation with the UM team. Exact personnel requirements (i.e. how many supervisors, enumerators and data entry agents) is flexible, but all enumerators must be female. There should be an experienced ICT expert on the team.
DELIVERABLES (7): A roster of recruited personnel with their corresponding qualifications
8. Procurement of Materials and Training of Field Staff
A comprehensive general training should be given by the survey firm team to the supervisors, enumerators and data entry agents in order to create a team environment and to allow for substitution between roles should any team member take a leave of absence due to illness or other emergency. The supervisors should receive supplemental training as needed.
The enumerator training and pilot testing should be scheduled for the last half of February 2019, it will take place in Netrokona Sadar at the Ritu local office. The enumerator training will be led by members of the Ritu team and the UM team, and assisted (and translated if necessary) by the survey team who will provide all necessary equipment and materials. The pilot testing will be led by the survey firm team. All members of the field teams, including the experienced ICT expert, are expected to be present in Netrokona Sadar during the training.
The Training program should include:
- Theoretical: Training should include a review of the theory of the questionnaire and each question and answers in order to fully understand the objective of each question. Standard quantitative interviewing techniques and field protocols should also be covered. This process will be led by the Ritu and UM team.
- Classroom practice: Training should include individual and group exercises to become familiar with the practice of asking and filling in questionnaires. Including one-on-one role plays.
- Pilot testing: at least 1 day of pilot testing in an area outside the scope of the study, each enumerator will conduct one survey each. Followed by one day of debriefing.
- Evaluation: Following the training, enumerators, supervisors and data entry agents should be evaluated based on their understanding of the questionnaire and their ability to correctly record data using the same test scenarios as used in the classroom practice. Any enumerators, supervisors or data entry agents not adhering to the standards, will be released. The training period should conclude only once the field teams have demonstrated mastery of the designated tasks.
- Procured materials
- Training materials and field manuals
- Report with the results of the evaluation process and pilot test
9. Pilot Test
Indicators of success include:
- Interview teams correctly list, and interview girls in the enumeration area
- Interview teams create a location where interviewees feel their privacy is sufficiently protected
- Interview team members understand their roles
- Interview team members understand, and correctly follow interviewing protocols
- Survey administered within 60 minutes
- Data is correctly administered and stored
- Tablets run smoothly, no data or entry errors
DELIVERABLES (9): A documented process of the pilot test and data successfully transferred to the UM team.
10. Fieldwork Management and Supervision
The project managers should report on real-time validity checks upon receipt of each cluster’s data and provide the UM team with daily extensive field reports on the first two days, after which the data collection can be adapted if necessary. After day two the project manager delivers 3-day progress reports to the UM team. The progress report include:
- Dates of arrival and completion of each cluster
- Any notable difficulties or deviations from the standard field plan
- Record of each omission of girls that may have been required, including the reasons for omision
- The numbers and IDs successfully completed
- Any other notable occurrences
DELIVERABLES (10): The Project Manager’s timely submission of written progress reports of the midline data collection, including the information detailed above.
11. Final Database & Report
Conduct final cleaning of data, including:
- Identify incomplete interviews and redundant observations
- Ensure all components are correctly linked – datasets can be merged cleanly
- Final completion numbers
- Completion inventory
- All variables are clearly named and labelled in coherence with the survey modules in STATA
- All variables are clearly named in coherence with the survey modules in EXCEL
- The completed databases, including the listing data, school data, with data correctly organized, variables named and labelled and appropriate identifiers in line with the identifiers of the baseline survey.
- Final Data Collection Completion report to Simavi with description of and reflection on process, final questionnaire, enumerators & supervisor specifications, training report and number of respondents surveyed per school/community.
|15-12-2018||28-12-2018||2 weeks||contract signing survey firm|
|29-12-2018||01-02-2019||5 weeks||Request ethical approval at Directorates of education, allowing more time due to general election turbulence|
|02-02-2019||15-02-2019||2 weeks||Translate survey + programming + ethical clearance documents|
|16-02-2019||23-02-2019||1 weeks (parallel)||Checking translation with Ritu team, and alter accordingly|
|16-02-2019||23-02-2019||1 weeks (parallel)||translate sampling plan into geographic travelling plan enumerators|
|24-02-2019||01-03-2019||1 week||enumerator training + pilot testing in schools outside of study area|
|02-03-2019||20-04-2019||7 weeks||Surveying 3881 girls|
|20-04-2019||11-05-2019||3 weeks||Final data entry and cleaning database|
|18-05-2019||Send final database and final report to Simavi|
ESTIMATED LEVEL OF EFFORT
The deadline for the final deliverable is May 18th 2019, 4 weeks after completion of the data collection.
|Signature of Contract||30%|
|Deliverables 1: |
1.1. Adapted research protocol and informed consent forms to be signed by head of school level and the girls that are surveyed and identification of respondent compensation as per local customs
1.2. Informed consent forms to be signed by parents and the girls when survey at home
1.3. Evidence of ethical clearance, insurance and permits needed to implement the survey
2.1 Initial translation of questionnaires into local language
2.2 Adapted survey to fit local context
|Deliverables 2: |
2.3. Final local language questionnaires
2.4. Final corresponding English questionnaires
|Deliverable 3: |
A written plan of scripting and pre-testing electronic version of the questionnaire
A written Field Sampling Plan approved by UM
A written Fieldwork Plan approved by UM
|Deliverables 6: |
6.1 Written Data Entry Protocol for data entry agents detailing program
6.2 Final data entry program adapted for the local questionnaires
6.3 The dataset dictionary with all variables labelled and defined
A roster of recruited personnel with their corresponding qualifications
|Deliverable 8: |
8.1 Procured materials
8.2 Training materials and field manuals
8.3 Report with the results of the evaluation process and pilot test
A documented process of the pilot test and data successfully transferred to UM
|Deliverable 10: |
The Project Manager’s timely written progress data collection reports
11.1. The completed Databases, including the listing data, school data, with data correctly organized, variables named and labelled and appropriate identifiers in line with the identifiers of the baseline survey in EXCEL & STATA
11.2. Data Collection Completion Report approved by Simavi
REQUIRED SKILLS/ EXPERIENCE
The Survey Firm must meet the following minimum requirements:
- Legal status recognized by the Government of Bangladesh, enabling the organization to perform the above-mentioned tasks under the laws of Bangladesh.
- Minimum 5 years experience managing surveys of similar scale (2000+ respondents); experience working on health-related surveys preferred.
- Strong capacity and experience in planning and organizing survey logistics.
- Good network of experienced enumerators, supervisors and data entry clerks.
- Proven experience with electronic tablet surveying (eg. provide an example of data format previously used)
- Strong capacity in data management and statistics.
- Strong knowledge in the following software: ODK, CS-Pro, and STATA.
- Strong interpersonal skills and a team oriented spirit.
In the technical proposal, the firm must also indicate the proposed staff and qualifications for each of the three key managerial positions based on the following minimum requirements:
- Project Manager: plans, supervises and manages the entire survey with the assistance of the field and data managers. The Project Manager must be based in-country for the entire duration of the survey and must have experience in managing large-scale surveys of over 1500 respondents; experience managing surveys of at least 100 communities and/or schools; Degree in management, or other relevant field. Specific training in survey management.
- Field Manager: responsible for training of field staff; plans, supervises and manages the fieldwork. The Field Manager must be based in-country for the entire duration of the survey and have experience in managing fieldwork of large-scale surveys of over 1500 respondents; experience managing fieldwork for surveys of at least 100 communities and/or schools; Degree in management, or other relevant field. Specific training in survey management.
- Data + ICT Manager: plans, supervises and manages data entry, error checking, processing and consolidation of data. The Data Manager must be based in-country for the entire duration of the survey and must have experience in managing data entry of large–scale surveys of over 1500 respondents; experience in managing data entry for surveys of at least 100 schools and communities. Experience with digital survey entry. Degree in statistics or other relevant discipline; Specific training in data entry for survey management, Specific training and experience in using and programming CS-PRO.
- The survey firm must list 3 references of previous international employers
- The Data + ICT manager must list 3 references of previous international employers
This assignment is subject to the confidentiality requirement of the Ritu and UM. All data and information received for the purpose of this assignment are to be treated confidentially and are only to be used in connection with the execution of these Terms of Reference. All intellectual property rights arising from the execution of these Terms of Reference are assigned to the UM. The contents of written materials obtained and used in this assignment may not be disclosed to any third parties without the expressed advance written authorization of the UM.
How to apply?
Interested survey firms should submit a detailed proposal, addressing all deliverables and requirements
(including the references requested) by 6th of December 2018. Please upload your application via this link: http://simavi.onlinevacatures.nl/vacancy/details/116150.
Only applications which are uploaded via this link will be considered.
Do you have questions about this consultancy assignment? Please sent an e-mail to email@example.com and mention ‘Ritu midline survey’ in the subject line.
Or visit our website www.simavi.org.
Is this not your challenge? Support Simavi by sharing this vacancy with your network!