Monitoring and Evaluation

Information to help you think about your work and its effects more clearly, and to demonstrate to the outside world that what you are doing is useful.

Most people involved in running a group, and certainly all of those who have applied for a grant, will have heard the phrase ‘monitoring and evaluation’. It isn’t always clear, though, exactly what it means. Seen on an application form, or when compiling a report on how money has been spent, it can sometimes seem like a bureaucratic hoop that you need to go through.Used properly, though, monitoring and evaluation can be a very useful tool and, stripped of the jargon, it isn’t too complicated:

  • Monitoring is collecting and recording information about what your group is doing
  • Evaluating is using the information you have collected, together with other information and your overall experience, to get a good clear picture of your group and its work.

Looked at like this almost every group does monitor and evaluate what they do, if only informally: counting how many people turn up at a meeting is monitoring, and chatting about how an event went for a few minutes at the end is evaluation.

This information sheet aims to help when you need to do something more systematic than that. It aims to help you think about your work and its effects more clearly, and to do so in a more formal way that lets you demonstrate to the outside world, such as funders, that what you are doing is useful. It looks at

  • Why you should monitor and evaluate your work
  • How you should go about planning your monitoring and evaluation
  • Some techniques for monitoring
  • Some things to think about when evaluating

We also give a couple of case studies, which will hopefully make some of the issues clearer. This information sheet is intended as a simple introduction. Monitoring and evaluation, especially for larger groups, can become quite complicated and so we also give some other sources of information which you might find useful.

Why do monitoring and evaluation?

There are two good reasons why you might want to make your monitoring and evaluation more structured. The first, and most important, reason is that it helps you judge how effective your efforts are and what you could do to make them more effective. The second reason, and the one which prompts most groups to look at monitoring seriously, is that people who give out funds increasingly expect it.

In particular, monitoring and evaluating can help you

  • check that what you are doing is actually having the effect you want
  • improve things by identifying things that aren’t working
  • change what you are doing if the situation changes
  • provide evidence for funders or potential funders that what you are doing is working

All of these are useful but only if the monitoring doesn’t take so much time that you can’t get on with your other work. This is why the first step in any monitoring and evaluation should be planning it.

Planning your monitoring and evaluation

Every organisation will have a different way of monitoring and evaluating their work. What it is useful to do depends on what that work is. However there are two principles that are worth all groups bearing in mind when planning monitoring and evaluation.

Two principles…..

  1. Involve several people in deciding what you are going to do, and working out how you are going to do it. The aim is to provide useful information to your group, not to build up reams of paperwork. You will only design something useful if you have ideas from a number of people. Also, you are going to be reliant on people to collect the information, and this will only happen effectively if they understand why they are doing it.
  2. Keep it as simple as possible. You don’t want to take on too much, especially if you are only just starting to monitor your work. You can always get more detailed later, if you need to.

…..and some questions

The aim of monitoring is to provide the basis for your evaluation, and so when planning it is best to start with thinking about the evaluation. You will need to ask yourself

  • What are the main things we want to evaluate?  These should be the central things your group or project is trying to achieve.
  • What information will help us to evaluate these?
  • Who is going to record this information, and how?

The first of these is the key question, and you need to be as clear as possible about answering it. For many groups there is a problem in writing down clearly what they are trying to achieve. They often have, on one hand, a very general aim in their constitution and, on the other, a series of quite specific things they are doing. What they are trying to achieve lies somewhere in between.

For example, a pre-school’s constitution may give its aim as being ‘to enhance the development and education of children primarily under statutory school age’. Their activity may be to run an pre-school with 20 places between 9.30 and 12.15 every weekday. Providing places is obviously pointless if no one takes them up, so part of their aim will clearly be to fill places. They may also have decided that they want to ensure that some of the places are available for children of families who are unable to pay the full fee, and so offer reduced rates for children where the family is on tax credit.

All of this may translate into an aim of ‘providing 20 places between 9.30 and 12.15 every weekday, with a take up of at least 90% over the course of a year. A minimum of 25% of all children to come from families in receipt of tax credit.’ It is possible to monitor this – by counting the children, and keeping figures on those paying reduced rate – and to evaluate it by checking whether the targets set have been achieved.

Being clear about exactly what you hope to achieve from what you are doing is the most important part of planning your monitoring and evaluation. If you don’t get this right then everything else will inevitably be muddled. Ask yourself

  • Do we have a clear statement of what we are trying to do?
  • Does it contain clear targets which are measurable?
  • Does it spell out what we will regard as a success?

Allow time at the beginning to get this right. Involve other people so that everybody agrees that this is what you are trying to do, and how much of it you need to do to be successful. Write it down so that you have a clear and detailed set of targets against which to judge what you are doing.

The last part of your planning is to decide how to let people know the results of your monitoring. Funders are likely to have their own forms you will need to fill in, but you may also want to get news of how well you are doing out more widely. Perhaps an article in your newsletter, or some details in your annual report? You might even want to press release the results if they reflect particularly well on your work.

Tips on how to write a news release is available in our News Release*** information sheet. If you need contacts for local newspapers, TV, radio stations etc, you can find these in our Media Contacts sheet.

How to do it – Monitoring

There are various methods for collecting the information you need. Which you choose will depend on what you are trying to monitor. Again, there are two underlying principles: be consistent about what you are doing and keep everything as simple as possible. Never collect information when you don’t know why you are doing it.

You want to be collecting information about what you are doing, but you also need to try and get information on the effect your work is having. (Funders increasingly call the things you are doing ‘outputs’ and their effect ‘outcomes’.)

The Big Lottery Fund produces a useful booklet called ‘Explaining the difference your project makes’. This clarifies the kind of records they expect groups they fund to keep, and is also a good introduction to ‘outcome monitoring’. Increasingly funders are using a special vocabulary for what they want monitored and if you are receiving significant amounts of money you will need to understand it. The Big Lottery Fund booklet is a clear introduction.

Collecting information about what you are doing will mainly consist of counting things. What you need to record will depend on what you are doing. A playscheme will probably only need to record the number of sessions held, and the number of children who attended. Longer term projects may need to record more details about their users: were the people who came mainly women or men? From a particular ethnic community? From the area or outside? From a certain age group? Employed or unemployed? In this case you need to be clear with people why you want to collect information on them, and that they feel comfortable about it.

Again the most important thing here is to collect the information every time, and write it down in a consistent form so you can compare over time.

Collecting information about the effect you are having (the ‘outcomes’ of your work) can be more complicated. You need to try and find out what people are thinking and the effect that you are having on their lives. There is a range of things you could do to collect this information:

Feedback from users: Getting the ideas of users about how things are going is a first step. You might want to use a comments book or suggestions box; give users evaluation forms at the end of sessions; produce a questionnaire for everybody who comes in a particular week. You might also find it useful to record informal comments from users, and especially their comments about how things could be improved.

Feedback from workers: Whether they are volunteers or paid staff it is useful to get the ideas of people who are actually doing the work. If it is a one off event then try and get people together at the end, if only for a few minutes, to talk about what went right and what went wrong. If you have staff then make it a regular item in staff meetings to talk about how things are going.

If you are a larger project there are some more complicated things you might do.

Following up users: If you are providing advice or training sessions then it might be useful to follow up users. For example, if you gave a piece of advice on fundraising to a group did they actually get any money? If you trained someone who was unemployed did they then go on to get a job? This kind of follow up gives a clear picture of what effect you are having, but is clearly quite a lot of work. You should also be prepared for the fact that following up people may itself generate more work with them.

Focussed questionnaires: A health education project running a course for parents might want to produce an exit questionnaire at the end of the course. They could then ask participants whether they felt better informed about the various things the course had covered. This would produce information about which sessions needed more work to make them effective.

To get the best picture, of course, you would always do exit questionnaires and follow up. In practice if you spend too much time monitoring you never get any work done (and your users get sick of answering your questions!) It is important to get a balance. One way of doing this, if doing things that are more major, is to do them with only a sample of users. A 10% sample should give you a good enough picture of your users as a whole.

The Community University Partnership Project can help with more complicated research needs. They provide training, a help-line, one-to-one guidance and matching to research specialists in the University.

How to do it – evaluating

Once you’ve collected your information you need to work out what it is telling you. Again how long this takes and who should be involved varies a lot from group to group. However there are some basic points worth making:

  • Accept that this can be quite a long process and needs a reasonable amount of time.
  • Include more rather than less people. Your aim is to get a clear picture of what you are doing well and what you are doing badly. The more viewpoints the better.
  • Make sure you draw on all the sources of information you have. Just because you have some new statistics don’t ignore the experience of people in the organisation. Put what you have learnt into the context of what you already know.
  • Recognise that people in the group have invested time and effort into making things happen. If your monitoring is showing that an event didn’t have the effect you hoped then be aware that this is the responsibility of the whole group, not just the people who were most closely involved.

Finally

Having put this effort into collecting information, and working out what it means, get the most benefit you can from it.

Where you find you have done a good job then let people know. Tell your users and your funders that you are doing a good job, and have the facts to prove it.

Where you find things haven’t gone so well then think about changing what you are doing. Everybody makes mistakes the trick is to recognise them and to learn from them.

Updated October 2014