Tricky question as neither ISO9001 or AS9100 define KPI’s, or place a requirement on an organisation to adopt KPI’s. KPI’s only get a mention in AS9101, which is the standard defining requirements for audits against the aerospace standards AS91XX, but is enhanced through the requirement for an auditor to report on a companies KPI’s, and grade their process performance using KPI results, as such the requirement for KPI’s is inferred on an organisation due to the requirements for aerospace audit reporting.

As a result it appears that often any KPI or metric will be taken by an auditor and reported within the context of aerospace audits, with subsequent process performance grading based on this result, whatever it is, and when challenged auditors often struggle to be able to find a clause against which to raise a nonconformance on KPI’s, the result can sometimes be some bizarre KPI’s seen reported.

So lets dig little through the requirements and definitions to see what we can piece together to enable a meaningful audit of KPI’s, and therefore meaningful process performance grading – or consideration of process effectiveness for pure ISO9001 audits.

AS9101 first gives us a definition of key performance indicator (KPI) as follows:-

“measures associated with goals or targets showing how well an organization is achieving its objectives or critical success factors for a particular project. KPIs are used to objectively define a quantifiable and measurable indication of the organization's progress towards achieving its goals

Note 1 to entry: KPIs relating to an organization’s financial performance are not in the scope of the EN 9101 standard; however, economic measures (e. g., sales quotas, scrap value reduction) can be considered acceptable measures for process improvement.”

In reading the above we see the introduction of other terms within the aerospace auditing standard that are not in the requirements for certification themselves, such as ‘critical success factors’ and ‘goals’, however these terms are used alongside ‘objectives’, which is a term within AS9100 and ISO9001, although slightly more broadly defined as ‘quality objectives’.

Now within the requirements of clause 6.2 of ISO9001/AS9100, which are exactly the same for both standards, so no extra requirements for aerospace certification here, there are many requirements relating to quality objectives which we can use to critique and audit KPI’s presented during the audit. Some key requirements relating to quality objectives are highlighted as follows:-

· The quality objectives shall be consistent with the quality policy. So here is the first place to take an audit trail and answer the question as to whether or not the presented KPI’s are consistent with the quality policy, noting also the requirements within the standard for content of a quality policy. Assuming the quality policy itself is compliant and reasonable, KPI’s should support achievement of this, if they bear no relationship to the policy then they are non-compliant quality objectives.

· The quality objectives shall be measurable. So are the KPI’s presented expressed in quantifiable metrics, e.g. counts, values, indices, etc.

· The quality objectives shall take into account applicable requirements. So what do customers want – take an audit trail to contracts, service level agreements, or other general processes of determining what this key external interested party wants from the organisation. A KPI for on time delivery set at 85% takes no account of customer requirements if contracts or other information shows a requirement for 95% on time delivery. Regulatory requirements may also be a factor, although generally these do not specify performance levels directly, but take a look and follow audit trail to regulatory requirements. Broaden this search also to requirements of any external or internal interested parties, how are they served by the KPI ?

· Quality objectives shall be relevant to conformity of products and services and enhancement of customer satisfaction. This really is the main one on which to judge the objectives in a quality management framework – do they relate to product quality and customer satisfaction, if not then they are noncompliant, or if you like not a ‘quality objective’.

Now this may become more tricky where an organisation has a myriad of KPI’s set at various levels of the organisation, however the organisation should be able to present at least what it considers to be its top level quality objectives, which should be scrutinised as above and nonconformities raised against section 6.2 of the standard where the requirements are not being met. This is sometimes the problem I feel as auditors dive straight into asking personnel throughout the organisation what their KPI’s are without having first established the top level objectives established by the organisation within the QMS. In larger organisations the top level objectives may well be cascaded down through lower level departmental or process objectives, or even personal objectives – these linkages should however be clear and lower level measures seen to support the higher level objectives, this should be the focus. Other KPI’s presented should be considered by the auditor but largely to confirm that they would not counteract the top level objectives if there is no clear link to support the top level objectives.