The Bath Audit Course
the most simple way of understanding audit
14 Assess How Good is Your Audit
Here is the marking crib sheet that used to be used in years gone by in GP Training. It was called The Summative Assessment Audit Marking Sheet and was developed by COGPED. It still has relevance today. The marking schedule for Summative Assessment Audit is given below in italics. My tips follow each point. And in those days… your audit report had to satisfactorily cover all eight points to pass!
AUDIT & SUMMATIVE ASSESSMENT MARKING SCHEDULE
1. Reason for choice of audit
- Potential for change
- Relevant to the practice
- Try making the audit title a question. For example, if your criterion is “All patients on Viagra should have had a serum rhubarb done in the last year”, a good audit title would be: “Have all patients on Viagra had a serum rhubarb in the last year?”
- If your audit results from a problem you’ve noticed in the practice, document it in your write-up; it suggests that there is a potential for change.
- Think about whether you will be able to make any changes that are needed as a result of the audit. If you know you won’t be able to make any changes, there’s not much point in doing the audit.
- If you are doing the audit while in a hospital post, you need to choose a topic that looks at the GP-hospital interface. Referrals or discharge letters are possible areas for audit. Again, you need to demonstrate that you’ve found a problem that needs to be investigated. I suggest that you discuss your proposed audit with your GP Trainer or Educational Supervisor before you go ahead – your hospital colleagues may not know what’s needed for GP Training. And it has to be GP related!
2. Criterion/Criteria chosen
- Relevant to audit subject and justifiable, e.g. current literature
- Don’t make your audit complicated or overambitious! Often the best audits are the simplest; ones with too many/too complex criteria tend to run into trouble. I suggest you limit it to one or two criteria.
- Make sure that your criteria are clearly stated and unambiguous – try using the one on the Setting audit criteria page as a model.
- You must justify your criteria with references. You will need to do a literature search. I suggest that you quote at least four relevant references, preferably more.
- Make sure that your references are properly quoted (authors, year, journal/book, volume, pages etc).
3. Standards set
- Targets towards a standard with a suitable timescale
I’m afraid that the marking statement above doesn’t make a lot of sense! I take it to mean that:
- You need to set a standard for each criterion.
- Make sure that the standard is directly related to the criterion; see the examples on the Setting audit standards page.
- Make sure that you can justify your standard. If you can’t find any literature evidence to back it up, explain why you chose the percentage that you did.
4. Preparation and planning
- Evidence of teamwork and adequate discussion where appropriate
- Audit will only result in change if you involve all the relevant team-members from the start. This may include the GPs and Nurses who will have to implement any changes, as well as office staff who can help you doing a computer search. Again, you will need to document this.
5. Data collection (1)
- Results compared against standard
- See the Looking at your results page for tips on this. You need to explain why you think the practice didn’t meet the standard that was set.
6. Change(s) to be evaluated
- Actual example described
- Just telling people to do things better won’t result in change. You need to write up in some detail how the changes will take place.
- Example: “The GPs agreed to do a serum rhubarb on any patient that they see who is on Viagra” – fail – this wouldn’t be likely to pass, as there is no system to help them remember.
- “(a) The GPs were given a prompt card that they could stick on their computer screen as a reminder to do a serum rhubarb on any patient that they see who is on Viagra; (b) the secretary will search every three months for patients who are overdue for their serum rhubarb, and flag it as an active problem on the computer system” – pass – as it should result in change.
7. Data collection (2)
Comparison with Data collection (1) and standard
- Summary of main issues learned
Comment on any improvements that have resulted.
- How well did your proposals for change work?
- If you again didn’t reach the standard that you set, why not?
- If you did, should you be aiming higher next time, or look at something else e.g. whether abnormal serum rhubarbs have actually been acted on?
- Where should the practice go from here?
The Bath Audit Course, Designed by Dr Michael Harris (Bath)