Research has shown that parental involvement in children’s learning can positively affect their child’s academic performance and social-emotional development. However, in low-income communities, parental involvement in children’s learning tends to be low because parents typically lack understanding of what learning in early classes should look like and are likely to focus on ‘visible’ outcomes when deducing school quality (e.g. school infrastructure).
Involving parents in organizing assessments in their respective communities could help make learning visible to them and enable them to play a role in their child’s education as ‘assessors’. It would also provide an avenue for structured engagement with school stakeholders and local officials around learning outcomes. A low stakes community-led assessment model could also be a more reliable way to generate data on student learning, as unlike other stakeholders within education systems, parents do not have incentives to misreport their child’s performance; or at least their incentives would cancel out against the incentives of other parents.
To this end, the ‘Gram Panchayat Math Contests’ by Akshara Foundation inspired us to design an intervention which involves parents and local community members in the assessment process through community-based events.
Ideation Phase
We undertook ideation on the basis of the following guiding principles:
- Design by doing: We implemented small interventions on the field to test our assumptions, learn what works, and uncover contextual challenges that need to be factored in to strengthen our design.
- Test and iterate: We conducted a series of mini prototypes with parents and communities to iterate on several aspects of design before developing an intervention model that was ready to be tested at a large scale.
- Design for scale: We embedded technological efficiencies in the intervention design. We collaborated with a startup, Smart Paper, led by Nirmal Patel. They developed an AI tool that automated post-assessment evaluations by scanning answer sheets. Recently, the Government of Rajasthan, used their tool to scan 13 million answer sheets in five days, bagging them a spot in the World Book of Records!
This article describes our experience with designing and iterating on the field, highlighting learnings from three prototypes conducted in Delhi and Uttar Pradesh.
Prototyping Phase: Designing and Testing
We aimed at figuring out answers to the following questions through a series of prototypes:
- Convening authority: Who will convene children, parents and other community members for the event?
- Event management: Who will proctor the assessment and manage the event?
- Engagement: How can we make the event fun and engaging for everyone participating?
- Assessment results: Is the technology tool easy to use after a short training and is the tool giving accurate results?
Convening authority: who will convene children, parents, and other community members for the event?
In the first two prototypes, we relied on NGOs as the convening authority, since our primary focus was to test whether parents would even show up for such an event. In the first prototype, we partnered with Saarthi to conduct the prototype in Tughlaqabad region of Delhi. In the second prototype, we partnered with SPID Society to conduct the prototype in Najafgarh region of Delhi.
Moving ahead in the process, we tried to look for a scalable model to implement community-led assessments. Leveraging NGOs as convening authority was not an optimum solution, since NGO penetration was not uniform across all districts of the country. Keeping this in mind, we wanted to check whether Panchayats could function as convening authority for such an event; we partnered with Piramal Foundation in Balrampur, Uttar Pradesh. Piramal Foundation connected us to the Pradhan of Gondipur to organize a prototype in the village.
The Pradhan had a strong local network that he leveraged to invite parents and children for the event. ~30 parents showed up for the event. Based on our experience, Panchayats could be visualized as a feasible option to scale the idea of community-led assessments. However, we need to test this further in other geographies to be reasonably sure.
Event management: who will proctor the assessment and manage the event?
In the three prototypes, our team co-managed the event along with representatives from partner organizations and local volunteers.
Proctoring of assessment was done by team members from Saarthi during the first prototype and local volunteers recruited through NGOs during the second and third prototype. Volunteers were trained through a short session on how to facilitate the event and use the Smart Paper technology tool to auto generate results of the assessment.
Training was an important part of the overall success of the prototype. Over the course of the prototypes, we realized that the training needed to include proctoring related demonstrations and roleplays to ensure volunteers were set up for success and felt confident to facilitate the event.
Engagement: how can we make the event fun and engaging for all the actors participating?
Over the course of the first two prototypes, we saw parents leaving the venue halfway. We were not able to fully engage them in the process and hence realized the need for performative testing to complement the student assessment in order to break monotony. Moreover, we needed to relook parent engagement from two angles: before the start of assessment and during the assessment process.
Course correcting in the third prototype, we introduced two activities to engage parents. Before the assessment started, we played a modified version of Dumb Charades. During the assessment, we facilitated a drawing activity in parallel to engage the parents.
To drive overall engagement, we realized that children were the most important actors for us. We, therefore, ensured that we recognized the efforts of all the children who participated in the process. We gave participation gifts to all children, in addition to the prizes for the top three scorers.
Assessment results: is the technology tool easy to use after a short training and is the tool giving accurate results?
The Smart Paper technology promised instant results post assessment. During the preparation phase, we designed scan-friendly assessment tools with Educational Initiatives providing item-bank support.
On the accessibility front, over the course of the prototypes, our intent was to make the tool self explanatory for users. We tried out different things- instructions in vernacular medium, easy-to-comprehend icons, use of the tool by low RAM phones keeping rural settings in mind, etc.
On the accuracy front, we constantly iterated with Smart Paper to maximize accuracy over the course of the prototypes. Initially, tick-mark samples were collected from a village in Gujarat to improve the accuracy. We also tried out different things such as modifying the question paper design, automating student results across assessments, counting in newer versions of tick marks etc.
Next Steps
We are yet to finalize a model that could work at scale with minimal intervention by our team. We are planning for the next set of prototypes with the following open threads:
- On the political front, will the panchayats be motivated to take ownership of the process?
- On the administrative front, which officer will oversee the entire process end-to-end?
- On the financial front, who will fund such events at a hyper-local level?
- On the social front, how to sustain the interest of community members throughout the process?
- On the technological front, can the Smart Paper tool be used in areas without active internet connectivity?
If you have innovative solutions to the open questions or any feedback on the approach adopted, please write to us: satyam@centralsqaurefoundation.org, niharika@centralsquarefoundation.org.