<<  Ride Control Europe Technology Update Media Drive Robert Louis Balfour Stevenson  >>
ROAD SAFETY ETP EVALUATION TRAINING
ROAD SAFETY ETP EVALUATION TRAINING
AIM OF THE DAY
AIM OF THE DAY
Intended Learning Outcomes
Intended Learning Outcomes
Glossary Treasure Hunt
Glossary Treasure Hunt
The Evaluation Process
The Evaluation Process
7 Key Steps in Planning and Evaluating an Intervention
7 Key Steps in Planning and Evaluating an Intervention
7 Key Steps in Planning and Evaluating an Intervention
7 Key Steps in Planning and Evaluating an Intervention
The Intervention-Evaluation Cycle
The Intervention-Evaluation Cycle
What is the Difference
What is the Difference
Definitions: a refresher
Definitions: a refresher
Example: Older drivers project
Example: Older drivers project
Older drivers project
Older drivers project
Logic model Components
Logic model Components
EXAMPLE LOGIC MODEL
EXAMPLE LOGIC MODEL
Evaluation Designs
Evaluation Designs
Evaluation Designs
Evaluation Designs
Evaluation Designs
Evaluation Designs
Methods in our madness
Methods in our madness
Question Writing Basics
Question Writing Basics
Question Writing Basics
Question Writing Basics
Question Writing Basics
Question Writing Basics
Training Day Survey
Training Day Survey
Were here to help
Were here to help

: ROAD SAFETY ETP EVALUATION TRAINING. : jmcwhirter. : ROAD SAFETY ETP EVALUATION TRAINING.ppt. zip-: 295 .

ROAD SAFETY ETP EVALUATION TRAINING

ROAD SAFETY ETP EVALUATION TRAINING.ppt
1 ROAD SAFETY ETP EVALUATION TRAINING

ROAD SAFETY ETP EVALUATION TRAINING

www.roadsafetyevaluation.com

2 AIM OF THE DAY

AIM OF THE DAY

To develop participants ability to carry out ETP evaluation

www.roadsafetyevaluation.com

3 Intended Learning Outcomes

Intended Learning Outcomes

Be familiar with the meaning of specific terms used in evaluation Be able to distinguish between goals, aims and objectives Be able to set appropriate objectives Be able to construct and use a logic model Understand the pros and cons of different evaluation designs Be able to write closed and open questions

www.roadsafetyevaluation.com

4 Glossary Treasure Hunt

Glossary Treasure Hunt

Remember that pre-seminar task? Check your answers with the definitions displayed around the room Make sure you introduce yourself to anyone you dont already know!

www.roadsafetyevaluation.com

5 The Evaluation Process

The Evaluation Process

Planning and Evaluating an intervention key steps What comes to mind?

www.roadsafetyevaluation.com

6 7 Key Steps in Planning and Evaluating an Intervention

7 Key Steps in Planning and Evaluating an Intervention

Needs Analysis 1st data collection stage: What exactly is the problem and What is the best way to solve it? Intervention Planning What are we going to do and how? (aims and objectives) Monitoring and Evaluation Framework What, When and How to measure? 2nd data collection stage: base-line data

www.roadsafetyevaluation.com

7 7 Key Steps in Planning and Evaluating an Intervention

7 Key Steps in Planning and Evaluating an Intervention

4) Implement Intervention 3rd data collection stage: collect monitoring data 5) Main data collection - Collect post-intervention data. Review the data collected have what you need? Analyse data 6) Report Write report and disseminate findings 7) USE Apply findings: amend intervention as necessary. Use in needs analysis stage (Step 1) of future interventions

www.roadsafetyevaluation.com

8 The Intervention-Evaluation Cycle

The Intervention-Evaluation Cycle

www.roadsafetyevaluation.com

Identify need, target group & likely best intervention (local data, strategy, MAST, previous evaluations)

Feed results and lessons learned into future projects

9 What is the Difference

What is the Difference

..

Between an Aim and an Objective? Between a Goal and an Aim?

www.roadsafetyevaluation.com

10 Definitions: a refresher

Definitions: a refresher

www.roadsafetyevaluation.com

Goal (Very broad): The overall reason behind the intervention e.g. To reduce road casualties Aim(s) (Specific): What and who the intervention will change e.g. To reduce the incidence of drink-driving amongst 17-25 year olds Objective(s) (Very specific): Specific outcome the intervention intends to achieve - e.g. To see an increase of 20% in the knowledge of 17-25 year olds about the legal penalties for drink-driving, by September 2011 To reduce mean scores in a self-report survey of drink- driving behaviour completed by 17-25 year olds, by November 2011 (SMART)

11 Example: Older drivers project

Example: Older drivers project

Aims: To increase self awareness of older drivers To improve driving skills of drivers over the age of 55

www.roadsafetyevaluation.com

12 Older drivers project

Older drivers project

Objectives (By March 2012): To increase older drivers knowledge of the effects of medication on driving ability To improve older drivers journey planning To increase self identification of new aspects of their driving which could be improved To increase older drivers skills scores in a repeat test drive

www.roadsafetyevaluation.com

13 Logic model Components

Logic model Components

Inputs Outputs Outcomes Assumptions External factors Aims and objectives

www.roadsafetyevaluation.com

14 EXAMPLE LOGIC MODEL

EXAMPLE LOGIC MODEL

SKILLS PUT INTO PRACTICE DRIVERS SELF IDENTIFY NEW AREAS FOR IMPROVEMENT

INPUTS

OUTPUTS

S.T.OUTCOMES

INTERMED.

LONG

NUMBER OF EVENTS HELD NUMBER OF OLDER DRIVERS ATTENDING TEST DRIVES

INCREASE KNOWLEDGE ABOUT EFFECTS OF MEDICATION INCREASE JOURNEY PLANNING SKILLS

OLDER DRIVERS CONTINUE TO REVIEW AND REFLECT ON THE AREAS FOR IMPROVEMENT IMPROVED SKILLS OBSERVED

ASSUMPTIONS: Drivers accept the ADI assessments of their skills. Drivers identify themselves as an older driver and see a need to attend the sessions/assessments. That the sessions and assessments will have a positive influence, i.e. Not make drivers unduly cautious or over-confident. EXTERNAL FACTORS: Adverse weather, local news stories involving older drivers, increased insurance premiums, family members, poor health.

STAFF MONEY ADIS RESEARCH VENUE FOOD/DRINK

www.roadsafetyevaluation.com

15 Evaluation Designs

Evaluation Designs

Post only with no comparison group Post only with comparison group Post then pre, no comparison group Pre and post with no comparison group (non experiment) Pre and post with comparison group (quasi experiment) Randomised controlled trial (RCT)

www.roadsafetyevaluation.com

16 Evaluation Designs

Evaluation Designs

Points to consider: How many groups of participants need to be surveyed and how many times? (e.g. A post only design with no comparison group only surveys one group once) What are the implications for your time and resources, participants responses, and drop-out rates? Any base-line measurement taken? You need a base- line to be able to measure change

www.roadsafetyevaluation.com

17 Evaluation Designs

Evaluation Designs

More points to consider: Can you know if it was the intervention that caused the change, or factors external to the intervention? e.g. age/maturation, other interventions in the area, or random chance (such as someone close to the participant being involved in a road traffic incident).

www.roadsafetyevaluation.com

18 Methods in our madness

Methods in our madness

You want to show effectiveness and have the opportunity to do a quasi experiment (e.g. Pre- and post with comparison group) Choose a method suitable for the intervention you have been working on, e.g. Telephone interview or self-complete questionnaire Write 5 questions. At least one question should be open ended.

www.roadsafetyevaluation.com

19 Question Writing Basics

Question Writing Basics

Keep questions short Do not ask two questions in one, e.g. How enjoyable and informative did you find this workshop? Be sure that everyone will understand the question in the same way pre-test questions Avoid jargon and abbreviations do not assume people will know what something means just because you do, e.g. serious injury, or ADI

www.roadsafetyevaluation.com

20 Question Writing Basics

Question Writing Basics

Avoid leading questions, e.g. Do you agree this workshop was enjoyable? as opposed to: Please rate how enjoyable you found this workshop Avoid using two negatives in one question, e.g. How much do you agree with the following statement: I never not wear a seat belt Avoid surplus questions. Do you really need to ask that question? How is it different to your other questions? How will you use the data?

www.roadsafetyevaluation.com

21 Question Writing Basics

Question Writing Basics

Be Specific about what you are asking: Please rate this course on a scale of 1-5 with 1 meaning Poor and 5 meaning Excellent Please rate the following aspects of this course on a scale of 1-5 with 1 meaning Poor and 5 meaning Excellent (Opportunity to ask questions, Knowledge of presenters, Use of Examples...)

www.roadsafetyevaluation.com

22 Training Day Survey

Training Day Survey

Take a look at our own training day evaluation survey. What are the advantages and disadvantages of this design and method? Please complete the survey before you leave if possible.

www.roadsafetyevaluation.com

23 Were here to help

Were here to help

lsimkins@rospa.com jmcwhirter@rospa.com Thank you!

www.roadsafetyevaluation.com

ROAD SAFETY ETP EVALUATION TRAINING
http://900igr.net/prezentacija/anglijskij-jazyk/road-safety-etp-evaluation-training-97726.html
c

29
900igr.net > > > ROAD SAFETY ETP EVALUATION TRAINING