country-office-performance-audits.md

Country Office Performance Audits

Data Owner	Narayanan Chettiar ([email protected]), OAI
Availability in Data Warehouse	Available
Data Refresh Rate	Daily
Accountability Weighted Scoring	30%

Introduction

Country Office Audits are audits performed by OAI (Office of Audit and Investigations). They measure compliance with corporate rules and regulations. The Performance App takes data from 2018 onwards for all the Country Office audits captured in CARDS and published on the Audit Public Disclosure website. We showcase the rating from the latest country office audit, as some country offices may have had multiple audits during that period.

There are four main possible audit ratings:

• Fully Satisfactory
• Minor Improvements
• Major Improvements
• Unsatisfactory

The underlying data has more categories, but we have mapped it as follows:

Old Categories	New Categories
Fully Satisfactory	Fully Satisfactory
Good	Fully Satisfactory
Satisfactory	Fully Satisfactory
Partially Satisfactory/SI	Satisfactory/Some Improvement Needed
Partially Satisfactory	Satisfactory/Some Improvement Needed
Satisfactory/Some Improvement Needed	Satisfactory/Some Improvement Needed
Marginally Deficient	Partially Satisfactory/Major Improvement Needed
Partially Satisfactory/Major Improvement Needed	Partially Satisfactory/Major Improvement Needed
Partially Satisfactory/MI	Partially Satisfactory/Major Improvement Needed
Seriously Deficient	Unsatisfactory
Unsatisfactory	Unsatisfactory
Deficient	Unsatisfactory

An audit will result in a set of recommendations that are tracked in a system called CARDS, managed by OAI. The Country Office is responsible for implementing these recommendations and providing status updates via CARDS within specific timeframes. Once the updates are provided, OAI evaluates them to confirm whether the recommendations have been implemented.

The recommendations are flagged according to the following traffic-light methodology based on OAI's assessment of the implementation status:

Time Frame	Status	Indicator Color
0-12 months	Great	Green
12-18 months	Acceptable	Yellow
18+ months	Unacceptable	Red

Organisational Objective

The organisational objective is the best audit performance possible, with no unsatisfactory audits and maximising entirely satisfactory audits. Additionally, the fastest and comprehensive implementation of the audit recommendations.

Data

{% hint style="warning" %} Check that we are not missing the field here for the actual main result of the audit {% endhint %}

Field	Description
as_of_date	Date of data extraction or last update.
audit_categ	Category or type of audit (e.g., OAI - Office of Audit and Investigations).
audited_bureau	Bureau or regional office audited (e.g., RBAS - Regional Bureau for Arab States).
audited_org_id_c	Unique identifier for the audited organization.
audited_cty_code	ISO 3166-1 alpha-3 country code (e.g., MAR - Morocco).
audited_unit_desc	Description of the audited unit.
action_bureau	Bureau responsible for audit action.
action_org_id_c	Identifier for the action organization.
action_cty_code	Country code for the action location.
action_unit_desc	Description of the action unit.
action_responsible_person	Name or title of the responsible person.
audit_id	Unique audit identifier.
audit_title	Title of the audit.
audit_period_start	Start date of the audit period.
audit_period_end	End date of the audit period.
audit_issue_date	Date when the audit report was issued.
audit_type	Type of audit conducted (e.g., COA - Country Office Audit).
reco_no	Number of the recommendation.
recom_id	Unique recommendation identifier.
reco_title	Title or summary of the recommendation.
reco_descr	Detailed recommendation description.
reco_planned_imnpl_d	Planned implementation date for the recommendation.
reco_priority	Priority level (e.g., High, Medium, Low).
reco_oai_status	Status of the recommendation (e.g., Implemented, In Progress, Not Implemented, Withdrawn).
reco_open	Binary indicator of recommendation status (0 = closed, 1 = open).
sub_domain	Specific organizational domain (e.g., SRF - Strategic Results Framework).

Data Export

{% hint style="danger" %} Requires definition {% endhint %}

Calculation of Scoring

We score only using each country's latest CO audit reports. Previous historical audit ratings are shown in the indicator drill down but are not considered in the scoring methodology.

For each audit result, we assign the following scores:

Audit Result	Score
Fully Satisfactory	100
Satisfactory / Some Improvement Needed	80
Partially Satisfactory / Major Improvement Needed	60
Unsatisfactory	0

Unsatisfactory is rated as zero. We want to significantly reduce the aggregate scores if even a small number of country offices in a region have Unsatisfactory audit ratings because these are so important to the organisation.

For Regional Bureau aggregation, we take an average of all the scoring, and the global average is calculated as an average of all country scores, not as an average of the Regional Bureau aggregation. This is to avoid skewed results, as different Regional Bureaus have different numbers of country offices. No weight is attached to country population size, amount of programming, or office size/budget.

Traffic Light System

Traffic Light	Score
Green	80+
Yellow	70+
Red	<70

Limitations & Future Improvements

• Not incorporating country size, budget, programming amount, and other key details into weightings could fail to account for larger country offices' naturally higher complexity and risk exposure. Weightings may better reflect performance. An office with many projects and budgets receiving an Unsatisfactory audit rating is far more important to the organization than a smaller office receiving an Unsatisfactory audit rating — but this is currently not reflected in the methodology.
• Looking only at the latest audit per country loses historical context. Incorporating a rolling 3-5-year average could smooth out anomalies and show improvement/regression over time.
• No qualitative context is provided on why audits scored the way they did or what actions are being taken. Incorporating audit insights could make the data more meaningful.
• We do not take into account the management responses in the scoring methodology, but we do show it in the drill-down

Resources

• Public Audit Disclosure Website
• CARDS (Requires invitation from OAI)