Educational Supervision

Finding the evidence for the Competency Rating Scales

Before an ES review, both the trainee and the Ed Supervisor are required to rate the trainee against the 13 professional competencies – and you have to justify your rating with ‘good enough’ evidence.  It’s one of the hardest bits of Educational Supervision and a lot of you seem to have difficulty with the write up.   

The rating scale section is perhaps the most important area of Educational Supervision (and ARCPs like to look at it too). And yet so many (trainees and supervisors) find it difficult. So, I’ve written this guide to make your lives easier.     It is hard work - providing evidence for every one of those 13 competences - but a) this guide will make it easier and b) your ES and the ARCP panel will be impressed with your write up. How would that make you feel?

What evidence do I use for the various competences?

This guide makes it incredibly simple… but only if you SLOWWWW down and follow what is written.  Click on each competency below and the box will open up and tell you which bits of your ePortfolio and WPBA provides good evidence of that particular competency.    Then, you can go about your write up by simply picking things listed there.   Try and go for the items in blue because they provide the stronger evidence.   However, if you have evidence from the remaining items, include that too; pack in as much evidence as you have – but be concise.

 In terms of what is good and reliable evidence, performance that is witnessed or observed generally carries more weight than performance that is inferred. For example, for the competency ‘Working With Colleagues’, evidence from an MSF (= comments from others on observed behaviour) has greater weight than say a CBD (= where you say what you think you did). The items in blue generally link to evidence that is observed.

This competence is about communication with patients, and the use of recognised consultation techniques.

  1. COT – in particular, items 1, 2, 3, 4, 8, 9 and 11.
  2. PSQ – all PSQ items are about this
  3. CEX – item 3: Communication skills
  4. CSR –under ‘Relationship’ items 1: Explores pt’s ICE, 2: Negotiates a plan, 3: Impact of problem on pt’s life
  5. MSF –under professional behaviour and/or clinical performance
  6. Log entries – esp Clinical encounters and OOH entries

The ability of the doctor to operate in physical, psychological, socioeconomic and cultural dimensions, taking into account feelings as well as thoughts.

  1. COT – item3: psychosocial context and 5: Pt’s health understanding
  2. CBD – item 1:Practising holistically
  3. PSQ – items 4: Interested in you as a whole person, 5:Fully understanding your concerns
  4. CSR – under ‘Relationship’ items 1: Explores pt’s ICE and 3: Impact of problem on pt’s life
  5. Log entries – esp Clin. encounters, Prof. conv., OOH entries,  some NOE

Working effectively with other professionals to ensure patient care, including the sharing of information with colleagues.  The ‘performance’ of working with colleagues is best reported by colleagues themselves i.e. through MSF.

  1. MSF – under professional behaviour
  2. Log entries  esp NOE – Clin. Encounters, Prof Conv, SEAs, Audit
  3. CSR – under ‘Relationship’ item 4. Works cooperatively with team members and ‘Professionalism’ item 2: Shows respect for others
  4. CBD – under  item 7: Working with colleagues in teams

The gathering and use of data for clinical judgement, the choice of physical examination and investigations, and their interpretation.

  1. CEX – Items 1: history and 2: examination
  2. COT – items 1: Encourages pt’s contribution 2: Responds cues 3:Psychosocial context 4: Pt’s health understanding 5:Includes/excludes significant condition, 6: Examination
  3. CBD – item 2: data gathering & interpretation
  4. CSR – under ‘Diagnostics’ items 1:Hx, Ex, Ix systematically & appropriately  2: Elicits signs and interprets information, 3: Appropriate differential diagnosis
  5. MSF – under clinical performance
  6. Log entries – esp. Clin. Encounters, OOH entries and SEA

Any reference to the doctors’s ability to perform clinical examination and clinical procedures/investigations.

  1. DOPS/CEPs
  2. COT –  item 6: apprpriate physical or mental examination
  3. Mini-CEX – item 2: physical examination skills
  4. CSR – under Diagnostics domain – 1. takes a history, examines and investigates appropriately and 2. elicits important clinical signs and interprets information appropriately.
  5. MSF – any clinical comments about clinical examination?
  6. Log entries detailing particular CEPs

This competence is about a conscious, structured approach to decision making. taken to protect patients.   The trainee must show not only his or her decision, but the thinking behind that decision.

  1. CBD  item 3: Making diagnosis/decisions2. 
  2. COT – items 6: Appropriate examination and 7: Appropriate working diagnosis
  3. CEX – item 4: clinical judgement
  4. CSR – under ‘Diagnostics’ items 3: Appropriate differential diagnosis and 5. Refers appropriately
  5. MSF – under clinical performance
  6. Log entries – esp Clin Encounters, Prof conv, OOH entries, SEA

The recognition and management of common medical conditions in primary care.

  1. CEX – item 7:overall clinical care
  2. COT – item 10:Appropriate management plan & FU
  3. CBD – item 4:Clinical management
  4. CSR – under ‘Diagnostics’ items 4: Management plan and 5: Refers appropriately
  5. MSF – under clinical performance
  6. Log entries – any of them

Aspects of care beyond managing straightforward problems, including the management of co-morbidity, uncertainty, risk and the approach to health rather than just illness.  Don’t forget, it also includes PROMOTING HEALTH.

  1. CBD – item 5:Managing medical complexity
  2. Log entries – esp Clin Encounters, Prof Conv, and NOE
  3. CSR – ????????
  4. CEX ?item 4: clinical judgement, 7: overall care and look at the ‘complexity of the case’ drop down box

Taking responsibility for organising oneself, developing systems to manage one’s workload, taking leadership in managing others or particular areas, bettering systems of care, engaging with IT systems beyond what is routinely expected.

  1. CBD – item 6: Organisation, Management, Leadership
  2. Log entries – esp QIA – quality improvement activity

Community orientation is the management of the health and social care of the practice population and local community.  Community orientation only comes to life when we look at the impact of disease/provision of health care in the wider patient context. This is where the NOE of audit work or engaging in a project that looks at the patient population becomes invaluable. Incidentally, these are often the best ways of learning about this element of the curriculum.

  1. NOE – Like audits, projects, significant events
  2. CBD – item 8: Community orientation
  3. CSR – under ‘Management’ item 2: uses resources cost effectively???
  4. Log entries – esp. Clin. Encounters, Prof. Conv. about this area

Maintaining the performance and effective continuing professional development of oneself and others.

  1. NOE – Audit, SEA, referrals analysis, presc. Analysis & complaints
  2. Log entries – teaching others
  3. CSR – under ‘Management’ item 3:Keeps up to date
  4. MSF – under ‘professional behaviour’ and/or ‘clinical performance’

Practising ethically with integrity and a respect for diversity.

  1. CBD – item 9: Maintaining an ethical approach
  2. NOE – significant events, complaints
  3. CSR – under ‘Professionalism’ items 1: Identifies & discusses ethical conflicts and 2: Shows respect for others
  4. MSF – under ‘professional behaviour’
  5. Log entries – esp. In Clin Encounters, Prof. Conv,

Fitness to practise is the doctor’s awareness of when his/her own performance, conduct or health, or that of others, might put patients at risk and the action taken to protect patients.  The Fitness To Practice competency is rarely witnessed, but can be probed through CbD.

  1. CBD – under item 10: Fitness to practise
  2. NOE – significant events, complaints
  3. CSR – under ‘Professionalism’ items 3: Is organised, efficient and takes appropriate responsibility and 4: Deals appropriately with stress
  4. MSF – under ‘clinical performance’
  5. Log entries – Clin. Encounters, Prof. Conv.
Competence AreaMSFPSQCOTCbDCEXCSR
Communication and consultation skills      
Practising holistically

 

     
Data gathering and interpretation      
Making a diagnosis/decisions      
Clinical management      
Managing medical complexity 

 

    
Primary care admin and IMT 

 

   

 

Working with colleagues and in teams      
Community orientation  

 

   
Maintaining performance, learning and teaching      
Maintaining an ethical approach      
Fitness to practise      

 

How to write it up

Remember the purpose of this rating scale: for you to provide evidence of performance in the 13 professional competences.  Nothing more, nothing less.  Now, if we focus on the term evidence, there are two types of evidence that you can provide – quantitative or qualitative.   Both are different ways of providing a measure of things.  One is not better than the other (see the infographic below) – both have their pros and cons.  

Both  are like two windows on different walls looking into the same room.  Looking through one window will give one “picture” of that room.  Looking through the other will also give a similar picture, but with differences that you will not have been able to see with first.  But if you look at both, a new picture emerges that is far more comprehensive, inclusive, complete and reliable.   In a similar way, you have to provide both quantitative and qualitative evidence of your performance for each of the professional competency areas – because this provides a more complete “picture” about you – one that is richer, deeper and more meaningful.

Some Do’s and Don’ts

  • First – start with the quantitative evidence to show that you’ve had a good enough exposure, rather than just a one-off or odd experience here and there.  In practical terms, this means writing about the numbers e.g. 6/8 CBDs were competent or above for Practising Ethically.
  • Then write about the qualitative, to add a bit of depth and meaning to your evidence.  But when you write this up, use individual cases or pieces of evidence to back up what you say rather than formulating an UNJUSTIFIED opinion.  For example, saying “log entry xxx demonstrates ….. which shows that i was thinking in ethical terms” rather than “i am always mindful of the patient’s autonomy in all my consultations hence practising ethically”.  The latter is an opinion, but the former provides evidence to justify the comment.   Another way of writing it: “i am always mindful of the patient’s autonomy in all my consultations hence practising ethically – as demonstrated in log entry dated 12.3.18”

The numbers don't tell you the whole story. For that, you need the narrative as well.

An analogy for you....

Let’s say you’ve got a birthday coming up and you are considering employing my services to bake you a beautifully decorated, delicious and moist cake.  How could I give you a bit of faith that I can bake a cake well and thus the right person for the job? 

If I presented you with a taster – and the cake I baked you was these three things – beautifully decorated, delicious taste, moist texture – surely that would provide the qualitative evidence for proving that I am the right person for the job.   But is this QUALITATIVE evidence enough?   

Does this ONE cake provide enough faith that the next bake will be good, and the next one, and the next one and indeed the one for your birthday?     Clearly, not.  For that – you need numbers (i.e. QUANTITATIVE evidence).   If I presented to you 10 perfectly baked, beautifully decorated, moist cakes – would you now be happy that I am a pretty good consistent cake baker?   

For those of you who said yes, are you sure?  What if I told you I baked 50 cakes and 40 turned out bad and I only presented to you the 10 that were good?  How consistent do you think am I now at baking cakes?   Not so happy?   What if my friend Ambar (who also bakes cakes) presented you 10 perfectly baked cakes but that Ambar only made 11 in total and only one turned out wrong?  Who is the better baker, me or Ambar?  (I hope you will say Ambar).  

So, what do we learn from all of this that is transferable to providing evidence for the competency rating scales?

  1. To provide a complete picture – you need both QUANTITATIVE and QUALITATIVE information.
  2. When providing QUANTITATIVE information, you need to provide the DENOMINATOR.  You cannot derive meaning from the numbers without their denominator.  Quantitative evidence needs to presented within the context of its denominator.  
  3. Quantitative evidence can be found in things like the number of CBDs, COTs, miniCEXs, MSF, PSQ and CSR (essentially the elements of WPBA.   
  4. Qualitative evidence can be found in individual log entries.

 

PROVIDE QUANTITATIVE EVIDENCE FIRST

  • In the free text boxes for each competency, provide quantitative evidence (i.e. focus on the numbers).  In other words, focus on the number of pieces of WPBA evidence which demonstrate the competence  rather than picking one or two learning log entries and talking about them in detail.   Use the COT, CBD, CEX mapping sheets.
  • But remember what I said – numbers mean nothing without their denominator.
    • For example, for Practising Holistically, you might write…
      • 7 out of 8 CBDs marked competent for Practising Holistically.    
      • 9 out of 11 COTs marked competent for Psychosocial Context. 
      • CSR writes meet expectations for exploring ICE and impact on patient’s life. 
      • MSF comments on ‘explores the impact of patient’s problems’
    •  AS OPPOSED TO    ‘Log Entry dated 11.2.14 shows a case where I did this, that and the other.’

ONLY THEN PROVIDE QUALITATIVE EVIDENCE

  • Tag individual Learning Log Entries as qualitative evidence.   But be sure to only pick those learning log entries which will clearly demonstrate competence to any third partly reading that entry.
  • Once you have tagged it, you shouldn’t really need to explain or write anything further about that lag entry – because it should be apparent from the entry itself.
  • Do not pick entries that are a bit wishy washy or weak.   It doesn’t look good if you link a weak entry that an independent third person cannot (at a glance) see the relevance to that competency.
  • You can use individual elements of WPBA as qualitative evidence – for instance, you could tag a particular CBD where you thought you demonstrated (say) ‘practising holistically’ well.   But as you will probably have already used this evidence as quantitative evidence, try and go for evidence that you haven’t used (e.g. log entries) instead.

Write-up Examples - please read

Here are two Professional Competency write ups from one of our trainee’s ePortfolio which I was incredibly impressed with.  Have a read and compare it with what you currently do.  Is there anything you can learn from it to make yours even better?   Remember, when you are providing evidence for the Competency Self-Rating Scales, you are meant to be “showing yourself off” to panel members who do not know you!   A bit like in a job interview – where you are selling and proving your worth.   This is the same – so why would you want to do it quickly and undersell yourself?  Spend time and write it up with care, thought and consideration.   Impress the many people who will be reading it (some of whom you will not know – but will be making a decision about you as to whether you move up an ST grade or not in your GP training).

COMMUNICATION & CONSULTATION SKILLS

Evidence

  • COTS  (9 done in total)
    item 1 patient contribution:  2/9 E, 7 C, 0 NFD, 0 IE
    item 2 cues:  1/9 E, 7 C, 1 x NFD, 0 IE
    item 3 PSO context:    6/9 C, 3 NFD, 0 IE
    item 4 patient’s ICE:  5/9 C, 2 NFD, 2 IE
    item 8 explanations:  7/9 C, 2 NFD, 0 IE
    item 9 confirms patient understanding:    4/9 C, 3 NFD, 2 IE
    item 11 patient involvement in Mx:  6/9 C, 1 NFD, 2 IE
  • My PSQ was very good in all areas.  Mean score generally 5 for most items and Median score generally 6. 
  • CSR – says meets expectations for exploring ICE, excellent for recognising impact of problem on patient’s life and meets expectation for negotiates a plan in partnership.  
    MSF themes – ‘good communicator’, ‘explains things well’, ‘good at communicating treatment plan’.   

Tagged Evidence

  • 06/05/2019 Learning log: Taught medical students consultation models
  • 30/01/2019 Learning log: Antibiotics wrongly prescribed by another GP
  • 10/05/2019 Learning log: Gentleman with unexplained back pain and lots of worries

Suggested Action Before Next Review

  • Maybe now try and look at difficult consultations and advanced consultation techniques.   Start by finding a course on handling difficult consultations – and then put that into practise?

MAKING A DIAGNOSIS/DECISIONS

Evidence

  • CBDs: for item Making Diagnosis/Decisions = 9/11 C, 2/11 NFD, 1/11 IE
  • COTs: (9 done in total)
    item 6 appropriate examination = 6/9 C,  2 E, 1 NFD. 
    item 7 Appropriate working diagnosis: 9/9 C
  • Previous CSR suggested to try and work more independently and to balance when to seek reassurance from others. Current CSR comments are that I meet expectations for diagnostics items, appropriate differential diagnosis and refers appropriately with specific comments stating that clinically very good, and does not over or under investigate. 
  • MSF themes around having good clinical knowledge, explores differential diagnosis very well, good at knowing when to ask for help, thinks laterally when needed

 

Tagged Evidence

  • 30/04/2019 Learning log: The girl who was taking the pill incorrectly
  • 20/03/2019 Learning log: Home visit man with haematuria – what next?
  • 25/04/2019 Learning log: Training in telephone triage

Suggested Action Before Next Review

  • I feel I have made incredible progress in terms of coping with uncertainty over the last 12 months. I would like to build on this further – perhaps read Tim Crossley’s book on ‘I Don’t Know What It Is But I Don’t Think It’s Serious’ (confidence and decisiveness in primary care).

Conclusions
Can you see that…

  1. It need not be particularly lengthy
  2. It is concise and specific
  3. The brown bits provide quantitative evidence in reference to the WPBA components
  4. The blue bits use Learning Log Entries as additional ‘tagged’ qualitative evidence.
  5. The green bit is quite specific about how the trainee plans to build on each competency in the near future.

Can you do something similar?  I’m sure you can.   Speak to your Educational Supervisor or GP Trainer if there is anything here that confuses you or you want more help with.

The last bit... Action Points

All trainees need to write up reasonable action points for each competency.  They must not be woolly or vague like ‘continue to build on this skill’.  This is another area which both trainees and supervisors struggle with – why? – because the points generated are too woolly and vague.    So, reading this section will help you loads!

And even if you are an ST3 who is completing to CCT, YOU STILL HAVE TO write action points for each of the Professional Competencies – because some of these will form the basis of your PDP for your very first GP appraisal.   In the case of ST3s who are completing, the ‘suggested action before next review’ box should be interpreted as ‘suggested action before first appraisal’.

  • In summary, you need to write objectives which are SMART.  The easiest way to do this is to simply think what next practical step the trainee needs to take which will help progress him or her in this competency.
  • Feeling stuck?   Click on the button below to some top ideas…

Got any suggestions or advice?

Got any advice?  Anything we’ve missed?  Then leave a message below.   Got a resource to share: contact [email protected]

Make GP Training Better Together’

1 thought on “Finding the evidence for the rating scales”

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Scroll to Top