The latest National Achievement Survey (NAS), a large-scale assessment survey of student learning levels, was held on 12 November, 2021. Led by the National Council of Educational Research and Training (NCERT), under the aegis of the Ministry of Education, NAS has been administered periodically since 2001 at the elementary level and from 2015 at the secondary level.
The primary objective of the assessment is to examine the effectiveness of the country’s education system. As a system level assessment, NAS provides a snapshot of student achievement at National, State/UT and district levels. This is achieved by conducting standardized assessments to students at the end of grades 3,5,8 and 10 in key curricular areas, as well as collecting contextual data on school, teacher and student related background factors associated with student achievement. The findings from NAS inform stakeholders at different levels in education policy and planning, teaching practices to improve quality and ensure equity in learning. Over the years, NAS has become a national measure of student achievement, providing periodic, system-level feedback on student learning, assessing relative performance of sub-groups, and determining effectiveness of policies designed to improve learning outcomes.
The previous cycle of NAS was administered in 2017 and the next was scheduled to be conducted in 2020. However, because of the Covid-19 pandemic, it was decided to postpone the survey to 2021. Apart from providing robust data to inform and support teaching-learning practices, the results from the 2021 survey will also help diagnose the extent of learning loss caused due to prolonged closure of schools during the pandemic. The findings will enable policy makers and practitioners to take important steps to improve overall learning levels at district, state and national level.
The findings from NAS inform stakeholders at different levels in education policy and planning, teaching practices to improve quality and ensure equity in learning. Over the years, NAS has become a national measure of student achievement, providing periodic, system-level feedback on student learning, assessing relative performance of sub-groups, and determining effectiveness of policies designed to improve.
Large scale assessments of student learning can provide valuable data on the education system with respect to issues of access, quality, efficiency and equity. For stakeholders to trust and use results from these assessments, they must have confidence that the assessments were designed and implemented in a technically robust way that reflects, to the extent possible, best practices in educational measurement and evaluation (AERA, APA, and NCME 2014). Adherence to technical standards is one recognized way of contributing to the quality of an assessment and ensuring it meets standards for reliability, validity, and fairness.
The following section talks about the processes and procedures followed by NCERT for design and development of NAS-2021 assessment framework and instruments in alignment with technical standards for quality.
Assessment Design for NAS-2021
The fundamental purpose of assessment in education is to establish and understand the learning levels of students at the time of assessment. The following decisions were taken by NCERT to ensure a robust assessment design for NAS 2021:
- Increase the number of questions (items) testing the specific learning outcomes across grades and subjects to improve the validity of assessments.
To accommodate the increased number of items without increasing the test length, time, and therefore cognitive load on students- the number of test booklets were increased, thereby limiting the number of items per booklet, attempted by a student. This also ensured an increase in test reliability through enough observations from which students’ scores can be derived to make inferences about performance. - All subjects were integrated into a single test booklet for grades 3 and 5 to give teachers a holistic view of student’s learning levels
- A modular matrix design was used — a characteristic of many large-scale assessments such as PISA, TIMSS and others — where each student takes a combination of only two subjects
- To enable comparison of NAS-2021 results with those from 2017, common items from the NAS-2017 were included
Development of NAS-21 test items and booklets
The purpose of NAS is to provide information about student’s learning levels across key curricular areas. The usefulness of that information depends both on the quality of the items or the test questions – and on the way the items are put together to make up the assessment instruments.
Procedures followed for conducting Field Tryouts for NAS-2021
- For each grade, the number of test booklets tested was almost 3 times the number of operational or final booklets for the main survey. This allowed for “weeding out” poorly performing items after the analysis and for the assessment framework specifications still to be represented after such items have been removed.
- The Tryouts were conducted across 7 States – including a mix of Government and Private schools covering approx. 9000 students
- One NCERT representative was appointed for each State to lead the field tryout operations
- The testing was supervised by the DIET faculty trained by the NAS cell
- Strict measures were undertaken to ensure security of test booklets and avoid exposure. The NCERT representatives carried sealed copies of test booklets to the test location and printing of booklets was done in presence of the representative to avoid leakage of test booklets.
NAS-2021 Background Questionnaires
Background questionnaires are critical to large-scale student assessments and provide complimentary data to the achievement tests. They help gain a better understanding of the other factors related to a student’s learning – such as their background or home-related factors, teacher and school factors. Each of the questionnaires included a section on teaching-learning during school closures due to the Covid-19 outbreak. Similar to test and item design, the background questionnaires were designed by NCERT through a robust development process- in consultations with experts from UNICEF, CBSE, Central Square Foundation, and the empanelled technical agency.
Research on assessment reveals that the right kinds of assessment activities, and the right uses of data generated by those activities, can contribute to better outcomes— such as better learning and better informed policy decisions (Clarke, 2011). For assessments to provide reliable and valid information on student learning levels, it is imperative to ensure that the assessment framework and instruments meet the standards of technical rigour. NCERT has been making systematic efforts to put in structures, resources, practices and tools, to build effective assessments to provide high quality data on the health of the country’s education system.