Evidence: Sufficiency and Reliability

Evidence, sufficiency and reliability. This lesson is about whether the evidence you have gathered actually supports your conclusion, which turns on two words the standards use together: sufficient, and appropriate.

Start with a request. You ask for evidence that privileged access was appropriate during the year, and you receive a spreadsheet of current privileged accounts, exported yesterday, emailed to you by the platform team. There are three separate problems with that evidence, and only one of them is obvious. By the end of this lesson all three will be.

The standards phrase is sufficient and appropriate, and the two words do different jobs. Sufficiency is quantity: is there enough evidence to support the conclusion? A sample of three changes is unlikely to be sufficient to conclude on four thousand. Appropriateness is quality, and it has two parts: relevance, does the evidence bear on the objective you are testing, and reliability, can the evidence be trusted? More evidence of the wrong kind does not make up for poor quality.

Underneath the reliability hierarchy is one simple principle. Reliability rises with the auditor independence from the person who could benefit from the evidence saying something particular. The further the evidence sits from that person, the more you can trust it. Hold that idea, and the ranking almost writes itself.

So here is the hierarchy, most reliable to least, and it is tested heavily. It runs from evidence you obtain yourself, at the top, all the way down to what management simply tells you, at the bottom.

At the top, evidence obtained directly by the auditor from the source system. A configuration you extract yourself, a recalculation you perform, a report you pull straight from the device. There is no opportunity for anyone to select or alter it.

Below that, evidence from independent external sources, such as a third-party confirmation. Then evidence produced by a system whose general controls you have tested and found effective. Each step down adds a party between you and the fact.

Lower still, evidence the auditee provides, and it is weaker again when the auditee chose what to hand over. And at the bottom, oral or written representations from management. This is why a screenshot from the administrator sits so low: it shows a moment somebody chose, in a format that is trivial to alter.

Which brings us back to the spreadsheet, and its most important flaw. The audit objective covers the whole period, but a current listing is point-in-time evidence. It shows the position on one day, so any account granted and removed during the year is invisible. Point-in-time evidence cannot support a period conclusion on its own. You need evidence spanning the period, such as a log of grants and revocations, or several points in time plus what happened between them. This is one of the most reliable traps in Domain 1, because point-in-time evidence is exactly what auditees naturally produce.

When your primary evidence is weak, corroboration can rescue it. Corroborative evidence comes from a different and independent source and agrees with what you already have. Its value is that independent sources are unlikely to be wrong in the same way. But note the limit: two reports from the same system are not corroboration, because they share a common source, and a fault in that source affects both identically.

Finally, documentation, because evidence you cannot stand behind is not much use. Your working papers must let a competent reviewer with no prior involvement understand what you tested, how you selected the items, what you found and what you concluded. The exam framing is whether another auditor could re-perform the work and reach the same conclusion. If they could not, the documentation is insufficient, however good the underlying testing was.

So carry this away. Quantity does not cure quality. If a question describes weak evidence and one option offers more of it while another offers a more reliable source, the more reliable source wins. And wherever direct extraction is available, prefer it to a file someone handed you.