Sample Size and Evaluating Exceptions

Sample size and evaluating exceptions. This lesson is about what actually determines how many items you test, and then what to do with the exception once you have found one, which is a harder question than most auditors treat it as.

Start with the instinct. Your population is four thousand changes instead of four hundred, and something in you says take a much larger sample. That instinct is close to wrong, and the exam likes testing it, so let us see why.

Three parameters do most of the work in setting sample size, and population size is barely one of them. The three that matter are the confidence level, the tolerable deviation rate, and the expected deviation rate.

Confidence level first. This is how certain you want to be that the sample supports your conclusion. Higher confidence means accepting less sampling risk, and less risk needs more evidence. So confidence level and sample size move together.

Tolerable deviation rate next. This is the maximum rate of deviation you could accept and still conclude the control is effective. A lower tolerable rate means less room for error, so the sample grows. Tolerable rate and sample size move in opposite directions.

And expected deviation rate. This is the rate you anticipate finding, based on prior audits or the control environment. A higher expected rate means you need more evidence to distinguish an acceptable control from an unacceptable one, so the sample grows. Expected rate and sample size move together.

Here is the relationship candidates miss most. Sample size depends on the gap between the tolerable and expected rates. As those two converge, sample size rises sharply. If you expect a four per cent deviation rate and can tolerate five, you need a great deal of evidence to tell the difference. And population size? Once the population is reasonably large, doubling it barely changes the sample at all. That is counter-intuitive, and it appears as a distractor again and again.

The exam also separates two sampling risks. The risk of over-reliance is concluding the control is effective when it is not. That is the dangerous one, because it leads to unwarranted reliance and too little substantive work. The risk of under-reliance is concluding the control is ineffective when it is. That one is merely costly, it produces unnecessary extra testing. If asked which matters more to the effectiveness of the audit, it is over-reliance.

Now, you have found an exception. This is the beginning of the analysis, not the end of it, and what you do next is exactly where the marks are.

Work it in order. First, establish what the control actually required, because an item that looks like a deviation may have followed a legitimate alternative path, such as a documented emergency change route. Second, establish the cause, because a deviation caused by one person being unaware of the procedure is a different problem from one caused by a system that permits an unapproved deployment. Third, do not classify the exception as isolated without evidence, isolated is a conclusion you have to support. And fourth, where the sample was statistical, project the rate and compare it, plus the allowance for sampling risk, against the tolerable rate.

The dominant real-world failure, and a heavily tested one, is accepting a plausible explanation and moving on. Management explains the exception, it sounds reasonable, and the auditor marks it resolved. The exam treats that as substituting representation for evidence. An explanation you have not tested against the documented process is not audit evidence, it is an assertion.

So carry this away. One exception is not automatically immaterial just because the rate looks low. One deviation in forty is two and a half per cent, and whether that is acceptable depends entirely on the tolerable rate you set during planning, not on the number feeling small in the moment.