Test Methods and CAATs

Test methods and computer assisted audit techniques. This lesson is about the four ways you gather evidence, which carry very different weight, and then about the tools that let you stop sampling altogether.

Here is the situation. You need to conclude that the nightly interface reconciliation between the payment system and the ledger actually works. So you ask the control owner to explain it, and they explain it clearly and correctly. You now understand how the control is meant to work. But notice what you have: you have no evidence that it did work. Understanding is not assurance.

There are four test methods, and the exam expects you to rank them by the strength of evidence they produce. In ascending order: inquiry, observation, inspection, and reperformance. The order is not trivia, it is the point of most questions that list all four.

Inquiry is asking. It is essential for understanding design, for finding out who does what, and for spotting where the real process differs from the documented one. But it is the weakest form of evidence, and on its own it cannot support a conclusion that a control operated. Every question that offers inquiry as the way to conclude something is offering a wrong answer.

Observation is watching the control being performed. It is stronger than inquiry because you see the activity rather than hearing about it, but it has two well-known limits: people behave differently when they know they are being watched, and it evidences only the moment you observed. Useful for physical and manual controls, weak for a conclusion about the whole period.

Inspection is examining evidence that the control was performed: signed reports, approval records, system logs, configuration files. This is the workhorse of controls testing. Its limitation is that it evidences performance, not correctness. A signed reconciliation shows that somebody signed it, not that the reconciliation was right.

Reperformance is you independently executing the control and comparing results. Recalculating the reconciliation, re-running the access report against the approved matrix, recomputing a sample of payroll. It is the strongest method, because it establishes that the control produces the correct outcome, not merely that it ran.

So carry this rule. Inquiry is never the answer to whether a control operated. In real audit a great deal runs on conversations, and an experienced auditor can tell when someone knows their process. The exam does not accept that at all. Inquiry corroborated by inspection is fine. Inquiry alone is never enough.

Now the tools, and the one thing about them that matters most for the exam. Computer assisted audit techniques let you examine data directly, and where the condition is precisely definable, a CAAT can test every record rather than a sample. That eliminates sampling risk entirely. When a scenario describes a precise, machine-testable rule over a large population, and one option offers a CAAT over the full population while another offers a sample, the CAAT is the better answer, because a definitive answer beats a projected one.

The complete-population cases are the obvious ones: segregation of duties conflicts, duplicate payments, entries posted outside business hours, dormant accounts that still hold access. And the techniques have names the exam uses. Test data submits prepared transactions to see how the system processes them. An integrated test facility runs fictitious audit transactions through the live system alongside real ones. Parallel simulation processes real production data through your own program and compares the output to the system. And continuous auditing runs these tests on an ongoing basis rather than at a single point in the cycle.

That last one carries a distinction the exam draws sharply. Continuous auditing is performed by audit. Continuous monitoring is performed by management. They sound alike and they are not the same thing, and confusing them is an independence error as much as a terminology one, because it puts audit and management in each other roles.

So carry away two things. Inquiry alone never concludes that a control worked. And sampling exists because testing everything used to be impossible, so where the data is available and the rule is precise, sampling a population a CAAT could test completely is the weaker audit, and the exam rewards recognising that.