<<  US-Taiwan Relations Since 1979 Use of Microsoft Office 2003 Excel Pivot Tables  >>
Use of assessment to monitor and attain learning goals in South Africa
Use of assessment to monitor and attain learning goals in South Africa
Outline of presentation
Outline of presentation
Background to developing learning outcomes
Background to developing learning outcomes
Brief Description
Brief Description
12 Key National Outcomes
12 Key National Outcomes
Key Documents
Key Documents
Learning Goals in South Africa
Learning Goals in South Africa
Developments in Education Sector
Developments in Education Sector
Large Scale Assessments in SA 1994 - 2014
Large Scale Assessments in SA 1994 - 2014
Overview of Learning Outcomes
Overview of Learning Outcomes
Minsters Delivery Agreement
Minsters Delivery Agreement
Delivery Agreement LO Indicators
Delivery Agreement LO Indicators
Indicator
Indicator
Key Lessons
Key Lessons
Development of learning goals
Development of learning goals
Planning
Planning
Strategic focus of Goals
Strategic focus of Goals
Clarity - Goals, targets & indicators
Clarity - Goals, targets & indicators
Consensus & Communication
Consensus & Communication
Ownership & Responsibility
Ownership & Responsibility
Long term targets
Long term targets
Accountability
Accountability
Assessment surveys helping or hindering
Assessment surveys helping or hindering
Clear Purpose of Assessments
Clear Purpose of Assessments
National Assessments: DUAL Purposes
National Assessments: DUAL Purposes
Use of Results
Use of Results
Use of Results
Use of Results
Example  Comparing Trends
Example Comparing Trends
EXAMPLE ANA reporting - Challenge
EXAMPLE ANA reporting - Challenge
DBE Reporting vs SS - Maths
DBE Reporting vs SS - Maths
DBE Reporting vs SS - Languages
DBE Reporting vs SS - Languages
Similar inaccuracies by province & district
Similar inaccuracies by province & district
District Performance  Grade 3
District Performance Grade 3
Summary
Summary
TENSIONS - local, national & global results
TENSIONS - local, national & global results
EXAMPLE Grade 9 TIMSS vs ANA results
EXAMPLE Grade 9 TIMSS vs ANA results
Grade 9 2012 ANA results
Grade 9 2012 ANA results
Grade 9 TIMSS vs ANA results
Grade 9 TIMSS vs ANA results
Obtaining evidence of learning  Key issues
Obtaining evidence of learning Key issues
Are learning goals being attained
Are learning goals being attained
Summary Key Lessons  revise
Summary Key Lessons revise
Way forward and Future Challenges
Way forward and Future Challenges
Way forward  Improve system
Way forward Improve system
Way forward  Teachers
Way forward Teachers
Future challenges
Future challenges
Conclusion
Conclusion
Questions Comments kanjeeA@tut
Questions Comments kanjeeA@tut
Consultative Process  by Office of President
Consultative Process by Office of President
Education Policy & Assessment: 1994-2014
Education Policy & Assessment: 1994-2014
Large Scale Assessment Surveys: 1994-2014
Large Scale Assessment Surveys: 1994-2014
Conclusion
Conclusion

: Use of assessment to monitor and attain learning goals in South Africa Anil Kanjee. : user. : Use of assessment to monitor and attain learning goals in South Africa Anil Kanjee.ppt. zip-: 2096 .

Use of assessment to monitor and attain learning goals in South Africa Anil Kanjee

Use of assessment to monitor and attain learning goals in South Africa Anil Kanjee.ppt
1 Use of assessment to monitor and attain learning goals in South Africa

Use of assessment to monitor and attain learning goals in South Africa

Anil Kanjee

2 Outline of presentation

Outline of presentation

Background - developing learning outcomes Policy Context Systems and structures: development and monitoring Learning Goals in South Africa Kinds of learning goals and assessments Reporting and communication of goals Key lessons for policy makers / countries Development, reporting and communication Are learning goals being attained? Tensions between local versus national versus global goals Is assessment helping or hindering attainment of goals? Way forward and Future challenges

2

3 Background to developing learning outcomes

Background to developing learning outcomes

3

4 Brief Description

Brief Description

Long term consultation process Began in 2004 2009 3rd term of new government 12 Key outcomes Government wide policy framework For monitoring and evaluation of goals and targets Spearheaded by office of the President Dept. of Performance Monitoring & Evaluation (2009) National Development Plan 2030 (2012) Delivery agreements (2009 2014) Between Minister and President Systems for data collection and reporting Annual National Assessments

4

5 12 Key National Outcomes

12 Key National Outcomes

BASIC EDUCATION: Quality basic education HEALTH: A long and healthy life for all South Africans SAFETY: All people in South Africa are and feel safe EMPLOYMENT: Decent employment through inclusive economic growth SKILLS: Skilled and capable workforce to support an inclusive growth path ECONOMIC INFRASTRUCTURE: An efficient, competitive and responsive economic infrastructure network RURAL DEVELOPMENT: Vibrant, equitable, sustainable rural communities contributing towards food security for all INTEGRATED HUMAN SETTLEMENTS: Sustainable human settlements and improved quality of household life LOCAL GOVERNMENT: Responsive, accountable, effective and efficient Local Government system ENVIRONMENT: Protect and enhance our environmental assets and natural resources INTERNAL AND EXTERNAL RELATIONS: Create a better South Africa, a better Africa and a better world PUBLIC SERVICE: An efficient, effective and development oriented public service and an empowered, fair and inclusive citizenship

5

6 Key Documents

Key Documents

6

7 Learning Goals in South Africa

Learning Goals in South Africa

Overview of learning goals Overview of assessments How goals and indicator specified?

7

8 Developments in Education Sector

Developments in Education Sector

Well established and strong examination system Over 120 years experience Improvement in capacity to implement NAS Within Ministry External agencies DRAMATIC increase in National Assessment Surveys Increase in Participation in Regional and International Surveys Key Policy review and revisions

8

9 Large Scale Assessments in SA 1994 - 2014

Large Scale Assessments in SA 1994 - 2014

Annual National Assessments 6 million students Grades 1, 2, 3, 4, 5, 6 9 2015 Grades 7 & 8

2010 2014

10 Overview of Learning Outcomes

Overview of Learning Outcomes

Link to specific goals and targets Listed in the Minister's Delivery Agreement 4 Key outputs 27 indicators 12 Indicators related to learning outcomes

10

11 Minsters Delivery Agreement

Minsters Delivery Agreement

Output 1:Improve the quality of teaching and learning Output 2: Undertake regular assessment to track progress Sub output 1: Establish a world class system of standardised national assessments Sub output 2: Extract key lessons from ongoing participation in international assessments Output 3: Improve early childhood development Output 4 Ensure a credible, outcomes-focused planning and accountability system Sub output 1: Strengthen school management and promote functional schools Sub output 2: Strengthen the capacity of district offices

11

12 Delivery Agreement LO Indicators

Delivery Agreement LO Indicators

Indicator

1.1 Percentage of Grade 3 learners performing at the required literacy level according to the Annual National Assessments.

1.2 Percentage of Grade 3 learners performing at the required numeracy level according to the Annual National Assessments.

2.1 Percentage of Grade 6 learners performing at the required language level according to the Annual National Assessments.

Annual National Assessments Grade 1 to 6 & 9

2.2 Percentage of Grade 6 learners performing at the required mathematics level according to the Annual National Assessments.

3.1 Percentage of Grade 9 learners performing at the required language level according to the Annual National Assessments.

3.2 Percentage of Grade 9 learners performing at the required mathematics level according to the Annual National Assessments.

12

13 Indicator

Indicator

Delivery Agreement LO Indicators

4 Number of Grade 12 learners who become eligible for a Bachelors programme in the public national examinations.

5 Number of Grade 12 learners passing mathematics.

6 Number of Grade 12 learners passing physical science.

7 Average score obtained in Grade 6 in language in the SACMEQ assessment.

8 Average score obtained in Grade 6 in mathematics in the SACMEQ assessment.

9 Average Grade 8 mathematics score obtained in TIMSS.

Grade 12 End of Schooling - Examinations

Regional Survey - SACMEQ

International Survey - TIMSS

13

14 Key Lessons

Key Lessons

Development of learning goals Obtaining evidence of learning Are learning goals being attained? Tensions between local, national & global goals Are assessment surveys helping or hindering attainment of goals?

14

15 Development of learning goals

Development of learning goals

Requires time and strategic consultation Start with what you have - Catch 22 in beginning stages Require reliable data to set realistic targets No reliable data available No single agency responsible for data No repository / database within Ministry QUESTION If you want data / results NOW from studies conducted a few years ago, Do you know where to get from? NO? Unsure You / System in trouble ??? Accountability systems critical To take responsibility To ensure focus on goals Unintended consequences teaching to the test SA Context good thing for NOW

15

16 Planning

Planning

Planning process accounts for the budgeting process, government reporting requirements, annual planning cycles, etc. -

16

17 Strategic focus of Goals

Strategic focus of Goals

Goals focus on key stages of the schooling system Grade 3 - end of Foundation Phase Grade 6 end of Intermediate Phase (Primary) Grade 9 End of Senior Phase (Junior Secondary) Grade 12 end of Schooling

17

18 Clarity - Goals, targets & indicators

Clarity - Goals, targets & indicators

Goal 1 Increase the number of learners in Grade 3 who, by the end of the year, have mastered the minimum language and numeracy competencies Indicator Percentage of Grade 3 learners performing at the required literacy levels according to the Annual National Assessments Target 2014 60%

Minimum NOT specified

18

19 Consensus & Communication

Consensus & Communication

Goals & targets specified in: Ministers delivery agreement Action Plan 2014 National Development Plan Specifies the required competency levels be defined as 50 percent and above Ministry document - Percentage of learners performing at the required .. Levels

19

20 Ownership & Responsibility

Ownership & Responsibility

National & Regional (SACMEQ) Assessments Ministry initiatives managed and funded Provincially - managed & reporting Developed capacity within Ministry Used in planning, reported to Parliament, etc TIMSS & PIRLS Prior to 2010 Managed & funded - University & Research Council No national or provincial ministry involvement Besides press outcry, limited (OFFICIAL) uptake and response from Ministry CURRENTLY Ministry funded, Included as indicators and targets CHALLENGE - Districts, schools & some provinces See National assessments as another task to complete

20

21 Long term targets

Long term targets

Specified in Action Plan 2025 Linked to Ministers delivery agreement

21

22 Accountability

Accountability

Minister accountable to deliver on goals Currently accountability mechanism for schools and teachers under discussion Action Plan 2014 Unclear Once-off incentives to ALL teachers in school demonstrating improvement National Development Plan Hold schools and school principals accountable for performance ? Argues for testing of principals as well

22

23 Assessment surveys helping or hindering

Assessment surveys helping or hindering

Assessment Surveys only useful if CONSENSUS and OWNERSHIP regarding: Purpose How results will be used Key agency/unit for managing survey SA Experience National Assessments Regional & International Assessments

23

24 Clear Purpose of Assessments

Clear Purpose of Assessments

1999 2010 To monitor attainment of key policy goals regarding Access, Quality, Equity, Efficiency 2010 2014 . Expose teachers to better assessment practices. Provide districts with information to target schools in need of assistance. Encourage schools to celebrate outstanding performance. Empower parents by providing them information about the education of their children.

24

25 National Assessments: DUAL Purposes

National Assessments: DUAL Purposes

Dual Purpose System Monitoring & Evaluation Purposes Focus on National and Province Local Intervention Purposes Focus: Districts, Schools, Teachers, Learners & Parents

PROBLEM Single Assessment No specification for monitoring and evaluation of system

25

26 Use of Results

Use of Results

Clear Relevant Doable

Purpose

Reporting

Province & District Identify schools in need of support and effect relevant interventions

National Level Monitor & evaluate performance of system (Implicitly stated)

Trends in performance by province & districts Identify key factor affecting performance

Trends in performance by district and school Identify areas of intervention

School, Teacher & Parent Provide information on learner strengths & weaknesses Provide support to learners in need Inform parent on performance of their children

Performance by each teachers Report to each learner Identify needs of each learner Report by performance levels

26

27 Use of Results

Use of Results

Purpose

Challenges

Trend data must be comparable SA all test items released and thus new tests not comparable

Province & District Identify schools in need of support and effect relevant interventions

Unclear why scores improving or declining? Annual administration limited time for interventions

Annual data good as data for new students Results based on arbitrary cut-points which provide limited information for intervention

National Level Monitor & evaluate performance of system (Implicitly stated)

School, Teacher & Parent Identify learner strengths & weaknesses Provide support to learners in need Inform parent -performance of their children

27

28 Example  Comparing Trends

Example Comparing Trends

How to explain these results? Improvement / Decrease in performance? Change in test difficulty? Implications for interventions?

Percentage students obtaining 50%

28

29 EXAMPLE ANA reporting - Challenge

EXAMPLE ANA reporting - Challenge

0-34%

36-49%

51-69%

71-100%

Scores reported by four levels of achievement to provide more information to teacher on which learners need support No empirical basis of selecting these cut-scores why 35% and not 30% or 25% Why 50% and not 60%?

Not Achieved

Partially Achieved

Achieved

Outstanding

ANA Report, 2011, p. 30

30 DBE Reporting vs SS - Maths

DBE Reporting vs SS - Maths

30

31 DBE Reporting vs SS - Languages

DBE Reporting vs SS - Languages

31

32 Similar inaccuracies by province & district

Similar inaccuracies by province & district

Grade 6 Language

Grade 6 Mathematics

33 District Performance  Grade 3

District Performance Grade 3

Incorrect identification of learner performance levels - Significant implications for interventions cost, time, planning, etc,

A

38

B

52

C

18

A

20

B

41

C

3

% of learners in each achievement category

% of learners in each achievement category

% of learners in each achievement category

% of learners in each achievement category

42

20

31

7

25

23

28

24

57

25

12

6

50

30

10

10

31

28

26

15

64

33

2

1

Grade 3 Subject

Grade 3 Subject

District No.

District No.

NA

PA

Ach

Out

Literacy

Literacy

Literacy

Numeracy

Numeracy

Numeracy

Sum: Ach + Out

Sum: Ach + Out

34 Summary

Summary

ANA scores inaccurately reported DBE levels over-estimate % learners at NA level (lowest level) DBE reporting under-estimate % learners at PA level No clear trend (or problems) noted at the highest levels. IMPLICATIONS Negative impact on interventions incorrectly identifying schools and learners for intervention Greater costs ito funds and HR

35 TENSIONS - local, national & global results

TENSIONS - local, national & global results

International survey results mainly impact on policy makers rarely teachers and public besides the period when results are published Creates confusion in relation to National Ass Different scales, Different time frames Different instruments Different results Grade 9 TIMSS results vs ANA results

35

36 EXAMPLE Grade 9 TIMSS vs ANA results

EXAMPLE Grade 9 TIMSS vs ANA results

TIMSS 2011 Grade 9 Maths results Overall 63 point improvement

36

37 Grade 9 2012 ANA results

Grade 9 2012 ANA results

National Average 13% Provincial averages 9 to 17%

37

38 Grade 9 TIMSS vs ANA results

Grade 9 TIMSS vs ANA results

Key point we need to able explain these different sets of results to our policy makers And be clear on how we will use the results

Question how is it that we show improvement in international surveys but find EXTREMELY low performance in our National Survey ? What do these two sets of results tell us? What does this mean for interventions? Which set of results do policy makers use?

38

39 Obtaining evidence of learning  Key issues

Obtaining evidence of learning Key issues

Require long term, integrated strategy National, regional and international data Realistic time frames to collect data Annually or Regular Cycles? Census or Sample ? Capacity to use assessment evidence District officials? Teachers School leadership Principal & Deputy, HoD Assessments to measure change No interventions No use to measure again

39

40 Are learning goals being attained

Are learning goals being attained

Difficult to tell at the moment No valid data to monitor trends (i.e. data not comparable) Indicator unclear required level ?? Results NOT reported using valid sound standard setting measures NB - mechanism to monitor & evaluate Assessment System

40

41 Summary Key Lessons  revise

Summary Key Lessons revise

Government wide framework and support Integrated national framework Clear goals and targets Long and Short term Different levels of the system Specific accountability systems Key persons held accountability Regular, valid & reliable evidence Effective systems for reporting, dissemination and use of evidence

41

42 Way forward and Future Challenges

Way forward and Future Challenges

42

43 Way forward  Improve system

Way forward Improve system

International Advisory Committee established Review assessment system to establish an integrated & effective assessment system Improving capacity at province and district levels to enhance use of assessment data National Reporting Framework Standard setting process Implement effective accountability system

43

44 Way forward  Teachers

Way forward Teachers

Trainees Teachers Comprehensive 3 year programme on testing AND classroom assessment; FOCUS use of assessment evidence & provision of feedback to improve learning and teaching In-service Similar programmes but Large scale (District wide)

44

45 Future challenges

Future challenges

Improving administration, data management and reporting of ANA Training district and provincial officials Scaling up in-service programme Extending pre-service programme to all teacher training institutes Implementing effective accountability mechanisms at lower levels of system

45

46 Conclusion

Conclusion

Significant progress made over the last 15 years in improving the assessment system in South African education sector. However, we still have a long way to go before we have an integrated, and effective assessment system for use in improving learning and teaching practices in all schools.

46

47 Questions Comments kanjeeA@tut

Questions Comments kanjeeA@tut

ac.za

47

48 Consultative Process  by Office of President

Consultative Process by Office of President

48

49 Education Policy & Assessment: 1994-2014

Education Policy & Assessment: 1994-2014

1995 - South African Qualifications Authority bill 1997 - National Curriculum Statement 1998 - Assessment Policy

2000 Report Min Review Comm Curri 2005 2002 - Revised National Curriculum Statements

2005 National protocol for Ass in schools 2007 National policy on Ass for schools 2008 Foundations of Learning Campaign

2009 - Education split Basic & Higher 2009 Min Task Team Report Review of NCS 2010 Action plan to 2014 draft 2011 Curriculum and Policy Statements 2012 National protocol for Ass : Grades R - 12 2012 Action plan to 2014 Final

1994 1999

1999 - 2004

2004 2009

2009 - 2014

2014 - 2019

Revised Assessment Policy Long term National Assessment Plan Reporting Framework

Period

Minister

Transformation focus

Key assessment related policies

Policy implementation; Development of systems for delivery

Improving of quality Understanding impact

??

Enhance Accountability

S. Bengu

Development of policy and regulatory framework

K. Asmal

N. G. Pandor

A. Motshekga (Minister of Basic Education)

Improvement of quality, introduction of accountability systems

49

50 Large Scale Assessment Surveys: 1994-2014

Large Scale Assessment Surveys: 1994-2014

Term

Policy Focus

National Assessment Surveys

1994 - 1999

2000 - 2004

Provincial Assessment Surveys

2005 2009

2010 - 2014

frameworks & documents

1995 TIMSS - MLA 1999 - TIMSS

implementation, systems development & delivery

2001 Grade 3 NA 2001 SACMEQ 2002 TIMSS 2002 Grade 9 NA HSRC 2002 2005 - CTA Grade 9 2004 Grade 6 NA

Improving quality and understanding impact

2006 PIRLS 2007 SACMEQ 2008 - FLC

Improving quality -accountability systems

2010 ANA 2010 TIMSS, PIRLS 2011 ANA 2011 - SACMEQ 2012 ANA 2013 - ANA

51 Conclusion

Conclusion

South Africa needs to develop and strengthen collaboration with other education systems in the English-speaking world and the BRIC countries (Brazil, Russia, India and China) to ensure that students from the South African education system are equipped to be admitted into those systems. International collaboration in education will help South Africa to successfully tackle its education challenges. Stronger educational links with these countries will also help students to become increasingly broadminded in terms of how they look at both South Africa and the wider world.

51

Use of assessment to monitor and attain learning goals in South Africa Anil Kanjee
http://900igr.net/prezentacija/anglijskij-jazyk/use-of-assessment-to-monitor-and-attain-learning-goals-in-south-africa-anil-kanjee-77536.html
c

29
900igr.net > > > Use of assessment to monitor and attain learning goals in South Africa Anil Kanjee