Quality Improvement Activity

Audit

DOWNLOADS

path:

The Bath Online Audit Course

the most simple way of learning about clinical audit

Is Audit Mandatory for MRCGP?

Not any more.  The requirement is for you as a trainee to show that you can successfully engage in Quality Improvement Activity (QIA) and Audit is just ONE of many types of quality improvement activity.    By the end of ST3, you need to have done one type of QIA.     It’s best done in the 1st GP post that you do (i.e. ST1), where there is more time and space for it.    Don’t leave it until ST3!  Click this page to see all the types of QUALITY IMPROVEMENT ACTIVITY that you can do: 

What is Clinical Audit?

Clinical audit is a process that has been defined as “a quality improvement process that seeks to improve patient care and outcomes through systematic review of care against explicit criteria and the implementation of change”.[NICE, 2002]  It is a way to find out if healthcare is being provided in line with standards and lets care providers and patients know where their service is doing well, and where there could be improvements.   The key component of clinical audit is that performance is reviewed (or audited) to ensure that what you should be doing is being done, and if not it provides a framework to enable improvements to be made.

Hopefully, you will have heard about Clinical Audit before and whilst you might not be able to recall the full audit cycle, in it’s basic form, audit is simply doing what is depicted on the right.  

 

How does Clinical Audit differ from Clinical Research?

'Research is concerned with discovering the right thing to do; audit with ensuring that the right thing is done'. 

Smith R.  Audit and Research. BMJ 1992, 305: 905-6. Tweet

 

RESEARCHCLINICAL AUDITSERVICE EVALUATION
Designed and conducted to generate new knowledgeDesigned and conducted to provide new knowledge to provide best careDesigned and conducted to define current care

Quantitative research – hypothesis based
Qualitative research – explores themes following established methodology

Designed to answer the question:
“Does this service reach a predetermined standard?”

Designed to answer the question:
“What standard does this service achieve?”

 Measures against a standardMeasures current service without reference to a standard
May involve a new treatmentDoesn’t involve a new treatmentDoesn’t involve a new treatment
May involve additional therapies, samples or investigationsInvolves no more than administration of questionnaire or record analysisInvolves no more than administration of simple interview, questionnaire or record analysis
May involve allocation to treatment groups NOT chosen by HCP or patientDoes not involve allocation to treatment groups: the HCP and patients choose treatmentDoes not involve allocation to treatment groups: the HCP and patients choose treatment
May involve randomisationDoes NOT involve randomisationDoes NOT involve randomisation

Is clinical audit old fashioned and should I be doing PDSAs instead?

If QI is the slick, new, sexy end of service development, then clinical audit can seem old fashioned, a fuddy duddy.  It’s perceived relevance can suffer through endless cycles of audits where change doesn’t occur. Annual audits can generate significant activity for clinicians with actions planned, only to discover that when re-audit happens again the next year, little has changed. It is easy to see how monumental effort for little gain can be demotivating. People can become disconnected.

We need to re-assert why clinical audit is a good thing, shout about the value that it adds for patients, and show that change is possible. Clinicians need to be helped to understand that Quality Improvement and Clinical Audit aren’t necessarily separate things. They can be two sides of the same coin. High quality clinical audit can lead to high quality improvement if it is done right.

Both rely on measurement to assess change, and both broadly use a Plan, Do, Study, Act cycle to get their work done. For me, the key is in the action planning. After a large scale piece of data collection for audit, I’ve often found people feel chuffed to get there, relieved that their hard work has paid off. And they can think they’re done—action planning can be a bit of an afterthought. In my experience, when a piece of work is focused on action planning and improvement, people place more emphasis on making change. Action planning becomes the journey, and data collection serves a purpose. QI can help with action planning as it promotes “rapid experimentation,” or PDSA cycles in quick succession. This can encourage people to stay engaged rather than wait for the annual audit wheel to turn around until next year.

In both Clinical Audit and QI we need to celebrate having a consistent and methodical approach to looking at problems in clinical practice and committing to making positive contributions. QI and clinical audit can support each other and both work towards the same goal: improving patient care. What could be sexier than that?

Billy Boland, Nov 2017, BMJ Opinion, https://blogs.bmj.com/bmj/2017/11/23/billy-boland-clinical-audit-is-not-dead/ Tweet

The 6 stages of audit

There are 6 easy steps to audit. 

Stage 1: Identify the problem or issue

This stage involves the selection of a topic or issue to be audited.   You basically want to pick an area in your practice that you think may or may not be at variance with healthcare processes that HAVE BEEN SHOWN to produce best outcomes for patients.  

Try and pick a topic for audit…

  • where national standards and guidelines exist; 
  • where there is conclusive evidence about effective clinical practice (i.e. evidence-based medicine).
  • areas where problems have been encountered in practice.
  • what patients and public have recommended that be looked at.
  • where there is a clear potential for improving service delivery.
  • areas of high volume, high risk or high cost, in which improvements can be made.

 

Stage 2: Define criteria and standards

Define the overall purpose of your audit, either as 

  1. what should happen as a result of the audit, or 
  2. what question you want the audit to answer, 

These will form the criteria for your audit – basically a series of statements or tasks that the audit will focus on. These criteria are explicit statements that define what is being measured and represent elements of care that can be measured objectively. A criterion is a measurable outcome of care, aspect of practice or capacity.  For example, 

  • ‘patients with type 2 diabetes should have a HBA1C at or below 48’

A standard is the threshold of the expected compliance for each criterion (these are usually expressed as a percentage). For the above example an appropriate standard would be: 

  • 80% of our patients with type 2 diabetes should have a HBA1C at or below 48’
  • Note: it is unwise to pick 100% or an non-achievable figure like 95% if you think you’re actual performance results might be in the region of 60%.  In the latter case, might be wise to go for 75% and then another audit later on to try and achieve 85% and so on.  
  • Remember – small steps for improvement – Rome was not built in a day!

Stage 3: Data collection

This is where the fun starts.   Use your computer system to its fullest to make light work of the data collection.  Get someone who is well versed in doing searches on your clinical system to help you.   In GP Practices, there is always a senior admin person who knows their stuff!

To ensure a reliable search though, you must define…

  • The user group to be included
  • Any exceptions?
  • The period over which the criteria apply.
  • The formula: in our example, patients with HBA1C >48

You can of course delegate the collection process to a person well-versed in this area (and we suggest you do so – so that you have more brain power left to do the analysing!).   Getting someone else to do this work makes Audit fun for you rather than something cumbersome and tiring – which it can be if you start doing EVERYTHING!!!

Ethical issues must also be considered; the data collected must relate only to the objectives of the audit, and staff and patient confidentiality must be respected – identifiable information must not be used. Any potentially sensitive topics should be discussed with the local Research Ethics Committee.

Stage 4: Compare performance with criteria and standards

This is the analysis stage, where you compare the results of the data collection  with the criteria and standards. In this section you will conclude…

  1. How well the standards were met by actual performance.  
  2. Figure out reasons why the standards weren’t met in all cases. These reasons might be agreed to be acceptable, i.e. could be added to the exception criteria for the standard in future, or will suggest a focus for improvement measures.

Where the performance was quite close to but not quite at the standard, it might be agreed that any further improvement will be difficult to obtain and you may then wish to focus on another clinical area for audit.  

In other cases, it might be really important to meet the standard or go beyond it (for example, some ‘life or death’ type cases you might need to set the standard as 100%).    This decision will depend on the topic area and best discussed with your colleagues in a group setting.

Stage 5: Implementing change

Once the results of the audit have been published and discussed, an agreement must be reached about the recommendations for change. 

  • Using an action plan to record these recommendations is good practice
  • Present this to your wider team in a dedicated protected setting (e.g. a meeting)
  • Then discuss and achieve a group agreement on what should be done and who has agreed to what and by when.
  • Each point needs to be well defined, with an individual named as responsible for it, and an agreed timescale for its completion.

Action plan development may involve refinement of the audit tool particularly if measures used are found to be inappropriate or incorrectly assessed. In other instances new process or outcome measures may be needed. 

Too often audit results in criticism of other organisations, departments or individuals without their knowledge or involvement. Joint audit is far more profitable in this situation and should be encouraged by the Clinical Audit lead.  This is not about blame – it’s about Quality Improvement.

Stage 6: Re-audit: Sustaining Improvements

After an agreed period, the audit should be repeated. The same strategies for identifying the sample, methods and data analysis should be used to ensure comparability with the original audit. The re-audit should demonstrate that the changes have been implemented and that improvements have been made. Further changes may then be required, leading to additional re-audits.

This stage is critical to the successful outcome of an audit process – as it verifies whether the changes implemented have had an effect and to see if further improvements are required to achieve the standards of healthcare delivery identified in stage 2.

  • Doing stages 1-5: is often referred to as a “5-point audit cycle”
  • Including stage 6 is often referred to as an “8 point audit cycle” and is the best way of doing audit because it shows whether or not your suggestions from the first round of audit made any difference.   
  • Try and always do an 8 point audit cycle rather than a 5 point one.

Any top tips?

Remember, clinical audit can be exciting and fun.  But far too often, those leading an audit start of all excited but then in the middle and end, become overwhelmed and exhausted and end up disliking it.   Why?   Let’s look at some of the reasons so that you don’t fall into the same trap.

THE DON’TS

Those doing the audit make it too complicated.  

  1. They pick a complicated topic which is affected by all sorts of complex variables.  So, choose something that is clinically straight forward.
  2. They set too many criteria when actually 1 or 2 will do.   So aim for 1-2.   3 max.
  3. They do all the work themselves.    The data collection process can be exhausting – especially if you don’t know how to use the clinical system for searches.    So DELEGATE it!   Use someone who does these things often.  There always is somebody!  And they can do it in a quarter of the time it would take you and they will help you refine your search so that there is not a lot of manual work like  looking into each patient’s records etc.
  4. Don’t set a standard that is stupidly high!   Try and set a standard which is reasonable – remember, we live in the real world.   NICE, QoF, LES, DES, CCGs etc might actually have already defined standards for various clinical things in General Practice – so use what is already out there.
  5. They do a rubbish write up!   Use the “Simple structure for doing and assessing audit” document in the QUICKLINKS section above.   It will help you with the write up too.

THE DO’S

So, here are some helpful tips

  • Pick a topic that genuinely interests you
  • Try to do something original.
  • Choose a worthwhile topic – one that you feel could make a real difference to many patients and/or members of staff.
  • Working on a topic where you can generate lots of data will provide more robust results.
  • If you choose a topic that is quantifiable, figures can be presented and used as a standard for the future.
  • Remember that it can take time to get permission from governing bodies (like the ethical approval committee) to access the information you may need.
  • Bear in mind that if you’re on a short placement, you may not be able to finish the audit.

 

How to write up your audit project...

As mentioned earlier, use the “Simple structure for doing and assessing audit” document in the QUICKLINKS section above.   In summary, your write up should cover the following sections. 

  1. A statement of the aims of the project: the background and reason for doing the project should be stated.
  2. Reference to the literature relevant to the project: the references used should be listed in a consistent style at the end of the project. Normally there would be 6 or more relevant references. They should be referred to clearly in the text.
  3. A clear description of the method used to collect evidence: the reader should have enough information to be able to repeat what was done. For audit projects, the criteria and standards should be stated.
  4. Presentation of the evidence: this should be clearly related to the aims of the project, and the method. Consider different visual ways of presenting numerical data.
  5. Discussion of the evidence presented: this should consider how well the chosen method addressed the aims of the project, the relevance of the findings to practice, and consideration of change in practice as a result of the findings. Evaluate your work.
  6. A conclusion to sum up the points raised in the discussion: make some practical suggestions to improve a situation. Make constant reference to the marking schedule throughout the planning and writing stages and aim for high marks for each criterion.

Please leave a comment if you have a tip, spot an error, spot something missing or have a suggestion for a web resource.
And of course, if you have developed a resource of your own, please email it to me to share with others.

'Make GP Training Better Together'

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.