Finding the evidence for the rating scales

Before an ES review, both the trainee and the Ed Supervisor are required to rate the trainee against the 13 professional competencies – and you have to justify your rating with ‘good enough’ evidence.  A lot of you seem to have difficulty with this write up.    So, I’ve written this guide to make your lives easier.  Please can you pay quite a bit of attention to your self-rating scale in your ePortfolio by reading this page carefully.

Where to find the evidence (click to open me)

Where to find the evidence

It’s simple… scroll down and look at the list of competency headings below.  Pick evidence from the things listed there (try and go for the items in blue because they provide the stronger evidence). However, if you have evidence from the remaining items, include that too; pack in as much evidence as you have – but be concise. Performance that is witnessed or observed generally carries more weight than performance that is inferred. For example, for the competency ‘Working With Colleagues’, evidence from an MSF (= comments from others on observed behaviour) has greater weight than say a CBD (= where you say what you think you did). The items in blue generally link to evidence that is observed.

How to write it up

How to write it up

I’m going to present to you a method where you can provide evidence for each competency in a quantitative AND qualitative way.  Quantitative means measuring something through numbers (or quantity).  Qualitative means measuring something through its quality rather than numbers.   Both are measures of how good something is.

For example, if I was to provide you with a beautifully decorated, delicious and moist cake, that would in some way (I hope) give you a bit of faith that I can bake a cake well.  These three things – beautifully decorated, delicious taste, moist texture – is the qualitative evidence that proves this cake (that I made) is good!   So, this qualitative evidence gives you some faith I can bake a cake and bake it well (i.e. some expression of my level of competency at baking a cake).  But does this one cake tell you how consistent I am at making good cakes (which is another angle of competency)?  No it doesn’t.  For that – you need numbers (i.e. quantitative evidence).   If I presented to you 10 perfectly baked, beautifully decorated, moist cakes – would you now be happy that I am a pretty good consistent cake baker?  For those of you who said yes, are you sure?  What if I told you I baked 50 cakes and 40 turned out bad and I only presented to you the 10 that were good?  How consistent do you think am I now at baking cakes?   Not so happy?   What if my friend Ambar (who also bakes cakes) presented you 10 perfectly baked cakes but that Ambar only made 12 in total and only two turned out wrong?  Whose the better baker, me or Ambar?  (I hope you will say Ambar).  Can you see that you cannot derive meaning from numbers without their denominator.  Quantitative evidence needs to presented within the context of its denominator.  

In summary, there are two aspects to how good I am at baking cakes – can I bake a cake that is of good quality (qualitative evidence) and can I repeatedly bake good quality cakes (quantitative evidence).  In a similar way, you need to present your evidence for the competency rating scales in both a quantitative and qualitative way.  And if you read on, I will show you how.  Quantitative evidence can be found in things like the number of CBDs, COTs, miniCEXs, MSF, PSQ and CSR (essentially the elements of WPBA.   Qualitative evidence can be found in individual log entries.

PROVIDE QUANTITATIVE EVIDENCE FIRST

  • In the free text boxes for each competency, provide quantitative evidence (i.e. focus on the numbers).  In other words, focus on the number of pieces of WPBA evidence which demonstrate the competence  rather than picking one or two learning log entries and talking about them in detail.   Use the COT, CBD, CEX mapping sheets.
  • But remember what I said – numbers mean nothing without their denominator.
    • For example, for Practising Holistically, you might write…
      • 7 out of 8 CBDs marked competent for Practising Holistically.    
      • 9 out of 11 COTs marked competent for Psychosocial Context. 
      • CSR writes meet expectations for exploring ICE and impact on patient’s life. 
      • MSF comments on ‘explores the impact of patient’s problems’
    •  AS OPPOSED TO    ‘Log Entry dated 11.2.14 shows a case where I did this, that and the other.’

ONLY THEN PROVIDE QUALITATIVE EVIDENCE

  • Tag individual Learning Log Entries as qualitative evidence.   But be sure to only pick those learning log entries which will clearly demonstrate competence to any third partly reading that entry.
  • Once you have tagged it, you shouldn’t really need to explain or write anything further about that lag entry – because it should be apparent from the entry itself.
  • Do not pick entries that are a bit wishy washy or weak.   It doesn’t look good if you link a weak entry that an independent third person cannot (at a glance) see the relevance to that competency.
  • You can use individual elements of WPBA as qualitative evidence – for instance, you could tag a particular CBD where you thought you demonstrated (say) ‘practising holistically’ well.   But as you will probably have already used this evidence as quantitative evidence, try and go for evidence that you haven’t used (e.g. log entries) instead.

Example of a good write up

Example of a good write up

Below are two Professional Competency write ups from one of our trainee’s ePortfolio which I was incredibly impressed with.  Can you see that…

  1. It need not be particularly lengthy
  2. It is concise and specific
  3. The brown bits provide quantitative evidence in reference to the WPBA components
  4. The blue bits use Learning Log Entries as additional ‘tagged’ qualitative evidence.
  5. The green bit is quite specific about how the trainee plans to build on each competency in the near future.

Can you do something similar?  I’m sure you can.

COMMUNICATION & CONSULTATION SKILLS

Evidence

  • In all 9 COTS in this review items 1.2,3,4,8,9 and 11 are competent or above. In fact items 1 (patient contribution), 8 (appropriate language) and 11 (patient involvement) have been marked as Excellent in 2, 3 and 2 COTs respectively. 
  • My PSQ was very good in all areas.  Mean score generally 5 for most items and Median score generally 6. 
  • CSR – says meets expectations for exploring ICE, excellent for recognising impact of problem on patient’s life and meets expectation for negotiates a plan in partnership. 
    MSF themes – ‘good communicator’, ‘explains things well’, ‘good at communicating treatment plan’.   

Tagged Evidence

  • 06/05/2015 Learning log: Taught medical students consultation models
  • 30/01/2015 Learning log: Antibiotics wrongly prescribed by another GP
  • 10/05/2015 Learning log: Gentleman with unexplained back pain and lots of worries

Suggested Action Before Next Review

  • Maybe now try and look at difficult consultations and advanced consultation techniques.   Start by finding a course on handling difficult consultations – and then put that into practise?

MAKING A DIAGNOSIS/DECISIONS

Evidence

  • CBDs 9/11 competent for making a diagnosis/decision
    COTs item 6 appropriate examination 6/9 Competent, 2/9 excellent, 1 NFD. Item 7 Appropriate working diagnosis 9/9 competent
    Previous CSR suggested to try and work more independently and to balance when to seek reassurance from others. Current CSR comments are that I meet expectations for diagnostics items appropriate differential diagnosis and refers appropriately with specific comments stating that clinically very good, and does not over or under investigate.
    MSF themes around having good clinical knowledge, explores differential diagnosis very well, good at knowing when to ask for help, thinks laterally when needed

Tagged Evidence

  • 30/04/2015 Learning log: The girl who was taking the pill incorrectly
  • 20/03/2015 Learning log: Home visit man with haematuria – what next?
  • 25/04/2015 Learning log: Training in telephone triage

Suggested Action Before Next Review

  • I feel I have made incredible progress in terms of coping with uncertainty over the last 12 months. I would like to build on this further – perhaps read Tim Crossley’s book on ‘I Don’t Know What It Is But I Don’t Think It’s Serious’ (confidence and decisiveness in primary care).

How to write up the Suggested Action Point for each competency

How to write up the Suggested Action Point for each competency

  • All trainees and Educational Supervisors need to write up reasonable action points for each competency.  They must not be woolly or vague like ‘continue to build on this skill’.
  • Even if you are an ST3 who is completing to CCT, you STILL have to  write action points for each of the Professional Competencies – because some of these will form the basis of your PDP for your very first GP appraisal.   In the case of ST3s who are completing, the ‘suggested action before next review’ box should be interpreted as ‘suggested action before first appraisal’.
  • In summary, you need to write objectives which are SMART.  It might make it easier just to think of what next practical step the doctor needs to take which is achievable within the first year post CCT.
  • This webpage may help you formulate an appropriate next action step if you are struggling.  www.bradfordvts.co.uk/educational-supervision/action-points-for-competency-rating-scales .

Evidence in blue bold type are particularly strong indicators of the domain.

RELATIONSHIP

1. Communication and consultation skills

  1. COTin particular, items 1, 2, 3, 4, 8, 9 and 11.
  2. PSQ – all PSQ items are about this
  3. CEX – item 3: Communication skills
  4. CSR –under ‘Relationship’ items 1: Explores pt’s ICE, 2: Negotiates a plan, 3: Impact of problem on pt’s life
  5. MSF –under professional behaviour and/or clinical performance
  6. Log entries – esp Clinical encounters and OOH entries

2. Practising holistically

  1. COT – item3: psychosocial context and 5: Pt’s health understanding
  2. CBD – item 1:Practising holistically
  3. PSQ – items 4: Interested in you as a whole person, 5:Fully understanding your concerns
  4. CSR – under ‘Relationship’ items 1: Explores pt’s ICE and 3: Impact of problem on pt’s life
  5. Log entries – esp Clin. encounters, Prof. conv., OOH entries,  some NOE

3. Working with colleagues and in teams

The ‘performance’ of working with colleagues is best reported by colleagues themselves i.e. through MSF.

  1. MSF – under professional behaviour
  2. Log entries esp NOE – Clin. Encounters, Prof Conv, SEAs, Audit
  3. CSR – under ‘Relationship’ item 4. Works cooperatively with team members and ‘Professionalism’ item 2: Shows respect for others
  4. CBD – under  item 7: Working with colleagues in teams

DIAGNOSTICS

1. Data gathering and interpretation

  1. CEX – Items 1: history and 2: examination
  2. COT – items 1: Encourages pt’s contribution 2: Responds cues 3:Psychosocial context 4: Pt’s health understanding 5:Includes/excludes significant condition, 6: Examination
  3. CBD – item 2: data gathering & interpretation
  4. CSR – under ‘Diagnostics’ items 1:Hx, Ex, Ix systematically & appropriately  2: Elicits signs and interprets information, 3: Appropriate differential diagnosis
  5. MSF – under clinical performance
  6. Log entries – esp. Clin. Encounters, OOH entries and SEA

2. Clinical Examination and Procedural Skills (CEPs)

  1. DOPS/CEPs
  2. COT –  item 6: apprpriate physical or mental examination
  3. Mini-CEX – item 2: physical examination skills
  4. CSRunder Diagnostics domain – 1. takes a history, examines and investigates appropriately and 2. elicits important clinical signs and interprets information appropriately.
  5. MSF – any clinical comments about clinical examination?
  6. Log entries detailing particular CEPs

3. Making a diagnosis/decisions

  1. CBD item 3: Making diagnosis/decisions2. COT – items 6: Appropriate examination and 7: Appropriate working diagnosis
  2. CEX item 4: clinical judgement
  3. CSR under ‘Diagnostics’ items 3: Appropriate differential diagnosis and 5. Refers appropriately
  4. MSF – under clinical performance
  5. Log entries – esp Clin Encounters, Prof conv, OOH entries, SEA

4. Clinical management

  1. CEX item 7:overall clinical care
  2. COTitem 10:Appropriate management plan & FU
  3. CBDitem 4:Clinical management
  4. CSR under ‘Diagnostics’ items 4: Management plan and 5: Refers appropriately
  5. MSF – under clinical performance
  6. Log entries – any of them

 5. Managing medical complexity

  1. CBD item 5:Managing medical complexity
  2. Log entriesesp Clin Encounters, Prof Conv, and NOE
  3. CSR – ????????
  4. CEX ?item 4: clinical judgement, 7: overall care and look at the ‘complexity of the case’ drop down box

MANAGEMENT

1. Organisation, Management, Leadership

  1. CBDitem 6: Organisation, Management, Leadership
  2. Log entriesesp NOE – audit

2. Community orientation

Community orientation only comes to life when we look at the impact of disease/provision of health care in the wider patient context. This is where the NOE of audit work or engaging in a project that looks at the patient population becomes invaluable. Incidentally, these are often the best ways of learning about this element of the curriculum.

  1. NOE – Like audits, projects, significant events2. CBD – item 8: Community orientation
  2. CSR – under ‘Management’ item 2: uses resources cost effectively???
  3. Log entries – esp. Clin. Encounters, Prof. Conv.

3. Maintaining performance, learning and teaching 

  1. NOE Audit, SEA, referrals analysis, presc. Analysis & complaints
  2. Log entries – teaching others
  3. CSR – under ‘Management’ item 3:Keeps up to date
  4. MSF – under ‘professional behaviour’ and/or ‘clinical performance’

PROFESSIONALISM

1. Maintaining an ethical approach

  1. CBD – item 9: Maintaining an ethical approach
  2. NOE – significant events, complaints
  3. CSR – under ‘Professionalism’ items 1: Identifies & discusses ethical conflicts and 2: Shows respect for others
  4. MSF – under ‘professional behaviour’
  5. Log entries – esp. In Clin Encounters, Prof. Conv,

2. Fitness to practise

The fitness to practice competency is rarely witnessed, but can be probed through CbD.

  1. CBD – under item 10: Fitness to practise
  2. NOE – significant events, complaints
  3. CSR – under ‘Professionalism’ items 3: Is organised, efficient and takes appropriate responsibility and 4: Deals appropriately with stress
  4. MSF – under ‘clinical performance’
  5. Log entries – Clin. Encounters, Prof. Conv.

Another way of looking at where the evidence is

Another way of looking at where the evidence is: (but log entries not included)

Competence Area MSF PSQ COT CbD CEX CSR
Communication and consultation skills            
Practising holistically

 

         
Data gathering and interpretation            
Making a diagnosis/decisions            
Clinical management            
Managing medical complexity  

 

       
Primary care admin and IMT  

 

     

 

Working with colleagues and in teams            
Community orientation    

 

     
Maintaining performance, learning and teaching            
Maintaining an ethical approach            
Fitness to practise            


Remind me - what are the competences which form the framework for WPBA?

Remind me – what are the competences which form the framework for WPBA?

  • Communication and consultation skills. This competence is about communication with patients, and the use of recognised consultation techniques.
  • Practising holistically:  the ability of the doctor to operate in physical, psychological, socioeconomic and cultural dimensions, taking into account feelings as well as thoughts.
  • Data gathering and interpretation: the gathering and use of data for clinical judgement, the choice of physical examination and investigations, and their interpretation.
  • Making a diagnosis / making decisions. This competence is about a conscious, structured approach to decision making.
  • Clinical management: the recognition and management of common medical conditions in primary care.
  • Managing medical complexity and promoting health: aspects of care beyond managing straightforward problems, including the management of co-morbidity, uncertainty, risk and the approach to health rather than just illness.
  • Primary care administration and IMT: the appropriate use of primary care administration systems, effective recordkeeping and information technology for the benefit of patient care.
  • Working with colleagues and in teams: working effectively with other professionals to ensure patient care, including the sharing of information with colleagues.
  • Community orientation: the management of the health and social care of the practice population and local community.
  • Maintaining performance, learning and teaching: maintaining the performance and effective continuing professional development of oneself and others.
  • Maintaining an ethical approach to practice: practising ethically with integrity and a respect for diversity.
  • Fitness to practise: the doctor’s awareness of when his/her own performance, conduct or health, or that of others, might put patients at risk and the action taken to protect patients