Dr. Tommie Singleton and I served as instructors for the recently completed AICPA Advanced IT Audit School. The school was well attended and, as always, was interactive with loads of great ideas and comments. The SOC reporting process is addressed during the school and brings insight and information from all over the U.S.
One of the areas that was a topic of discussion was IT testing for SOC 1 (SSAE16) and SOC 2. Often IT is not tested properly and leaves the users of the SOC report (auditors, regulators, and user organizations) without a clear understanding of what was tested and the results. Basically the term "best practices" is used quite often to describe when a control does not meet a criteria. The important thing to remember is IT testing should be applicable to the user of the report and the services offered. Testing a policy DOES NOT complete a test of operating effectiveness of things like password parameters, firewall effectiveness, proper user removal, and many, many more. I often see referral to a policy as a test of effectiveness of logical access. While an adequate policy is important, it is only effective when the controls are set up to meet the policy requirements. The only way to know a control is effective is to TEST.
Another important point in SOC reporting is for service organizations to maintain evidence a control is working properly over the testing period (usually 12 months). This means things like log files should be maintained for more than 12 months to ensure evidence to support a conclusion the control operated effectively over the period. If you have any questions about adequate testing or audit evidence contact one of our team at Auditwerx.