1 Kogan, Sudit & Vasarhelyi / JIS
Continuous Online Auditing: An Evolution
Alexander Kogan, Ephraim F. Sudit, and Miklos A. Vasarhelyi
Faculty of Management, Rutgers University
180 University Ave., Newark, NJ 07102
1Article submitted to the Journal of Information Systems
The advent of computers has affected numerous aspects of accounting and auditing. Computerization of accounting operations induced the development of electronic data processing (EDP) auditing as a new auditing field (see e.g. Hansen & Hill, 1989). Computer-assisted auditing has become common place, leading to a significant increase in the efficiency of auditing. Developments in information technology enabled management and reporting (internal and external) of finer information sets at progressively narrower time frames. Internal corporate management and many processes are increasingly dependent on daily closing balances and even online real-time reporting.
The proliferation of corporate-wide networks is enabling progressive integration of worldwide manufacturing, inventory keeping, and financial management. In turn, these developments have substantially reduced the incremental costs and complexity of consolidated reporting and its disclosure to related parties. Widespread availability of computer networking makes it possible to dramatically increase the frequency of periodic audits by redesigning the auditing architecture around online auditing.
The spectacular growth of the Internet in general, and the World Wide Web (WWW) in particular, has created a new set of opportunities and challenges confronting corporate management and reporting. These developments set the stage for the possibility of continuous online reporting. In parallel, WWW has spawned the development of the area of electronic commerce. The exponential growth of online retailing, online securities trading, and online procurement systems emphasizes the need for continuous online monitoring of transactions. This article focuses on the evolving field of continuous online auditing attempting to create a framework for and identify research issues related to its reasons, methods, implications, and available experiences.
1 We appreciate the comments given to us by the reviewers and editors of the Journal. 01/27/11 on-aud11.doc
2 Kogan, Sudit & Vasarhelyi / JIS
1.1 What Is Continuous Online Auditing?
Continuous auditing is a type of auditing which produces audit results simultaneously with, or a short period of time after, the occurrence of relevant events.While this definition reflects the commonly accepted meaning of continuous auditing, it would be more accurate to call this type of auditing instant rather than continuous. The confusion arises
because in many cases instant auditing leads to producing audit results at very high frequency, approaching a continuous stream of results. However, a continuous audit, in the sense of being instant according to our definition, can produce results infrequently if relevant events occur only sporadically. Practically speaking, if the scope of relevancy is wide and the audited entity is dynamic, it is highly likely that continuous auditing will indeed produce audit results very frequently.
Continuous auditing can only be feasible if implemented as (a) a fully automated process, and (b) a process with instant access to relevant events and their outcomes. The only known way to satisfy these requirements is to implement continuous auditing on an online
computer system. In this context, an online system refers to a system that is permanently connected through computer networking to both auditees and auditorsTherefore, this article discusses auditing which is both continuous and online, i.e. continuous online
1.2 Feasibility of Continuous Online Auditing
1.2.1 Technological Feasibility
In theory, the technological feasibility of COA rests on two important technological advances. First, accounting information is now almost always recorded and stored in electronic form. Second, ubiquitous computer networking allows continuous remote access to this information. This access is further facilitated by the apparent marketplace success of open Internet standards. Not only is the networking infrastructure widely available, but the protocols and tools have also become prevalent and affordable.
In practice, however, the development of COA has to surmount numerous technological and organizational challenges. The great variety of software systems used in enterprises makes it very difficult for auditors to develop integrated online auditing systems. A significant portion of these enterprise systems was designed as stand-alone systems having only rudimentary, if any, networking capabilities. Such legacy systems are being slowly replaced by new ones. The current developments in enterprise information systems clearly exhibit the trend toward more standardization and better integration of related subsystems. This trend towards enterprise systems suggests that many of the hurdles in the way of continuous online auditing should be overcome in the near future.
1.2.2 Economic Feasibility
Presently, continuous online auditing is technologically feasible only in certain industry sectors and for certain limited purposes. The acceptance of COA, however, depends on 01/27/11 on-aud11.doc
3 Kogan, Sudit & Vasarhelyi / JIS
whether it is economically feasible, i.e., whether the costs of COA can be lowered to levels that make its applicationcost-effective. A COA system can save auditors substantial costs (e.g. costs of travel, physical presence, manual collection of evidence). Furthermore, the costs of the technology required to implement online auditing (software, hardware, network connectivity) have been declining exponentially. These savings are likely to make it possible to develop and deploy online auditing systems without incurring prohibitive costs. Note however that the actual system development costs of COA will remain substantial, as the cost of software development has not benefited significantly from technological developments.
2 History and Institutional Background
Prompted by developments in information technology, COA research started over a decade ago (see Groomer and Murthy, 1989; Halper, Snively & Vasarhelyi, 1992; Koch, 1981; Vasarhelyi & Halper, 1991; Vasarhelyi, Halper & Ezawa, 1991). Following early developments in EDP auditing (see Cash, Bailey Jr. & Whinston, 1977), Groomer and Murthy (1989) described a prototype system for continuous audit of database applications. Subsequently, the accounting profession, as represented by the AICPA - —American
Institute of Certified Public Accountants and CICA - Canandian Institute of Chartered Accountants) came to realize that practice needs to expand beyond traditional annual audit of financial statements to the provision of broader types of assurance services. These developments are addressed in detail by the AICPA’s Special Committee on Assurance Services chaired by Robert K. Elliott.
2.1 Continuous Audit of Database Applications
Groomer and Murthy (1989) proposed an approach to address the unique control and security concerns in database environment. Their approach used embedded audit modules that capture information on a continuous basis. This approach is consistent with an evolutionary view of continuous auditing as the next natural step after traditional legacy system based EDP auditing. This early development in COA is especially important since it presents an implementation of COA based on relational database technology, which is the cornerstone of modern enterprise information systems. Embedded audit modules continue to be an essential part of COA architecture.
A further important theoretical development in the use of embedded audit modules for independent continuous online monitoring was described in Minsky (1996), where a law-governed architecture was proposed as the means of resolving the conflict between being independent and being embedded, i.e., part of the system. Early research efforts (Bailey, Duke, Gerlach, Ko, Meservy & Whinston, 1985; Gal & McCarthty, 1985) in the formalization of the representation of internal controls can potentially be linked to the concepts around embedded audit modules. Formal representation this would allow for adaptive analysis of transactions based on some normative progressive review of the perceived risk of existing internal control structures.
4 Kogan, Sudit & Vasarhelyi / JIS
2.2 The Elliott Committee
The Elliot Committee argued that dramatic societal, economic and technological developments were generating substantive changes in the accounting profession, and that major opportunities existed for accountants. The Committee has guided a new plan of action, initially developing six new types of assurance services (Risk Assessment, Business Performance Measurement, Information Systems Reliability, Electronic Commerce, Health Care Performance Measurement, ElderCare). Further developments in Electronic Commerce assurance have lead to the announcement of CPA WebTrust, (see http://www.aicpa.org/webtrust/index.htm; Greenstein, 1998.) The committee gave the
following description of the Information Systems Reliability service:
―The CPA monitors the functioning of the organization's systems to ensure
that they provide reliable data. This service involves either regular or,
ultimately, continuous oversight. It presumes some level of direct
involvement in computer operations by the CPA. He or she would either (1)
embed some level of monitoring or control in the client's system or (2)
direct regular inquiries into client processing systems/databases. This
service, while initially aimed at internal users, would have its greatest
appeal to external users who want to rely on entity data delivered at interim
dates and, ultimately, continuously….
Evaluating controls over real-time systems must be computer-based. …
Data flowing through the system will be monitored and analyzed using
CPA-defined rules. Exceptions to these rules trigger real-time warnings to
call the CPA's attention to potential problem areas and issues that need 2immediate resolution.‖
One can easily notice in this description several important features characteristic of COA. This changes relate to the timing, process, and tooling of the audit process.
2.3 The Systems Reliability Committee
The Systems Reliability Committee was established to respond to anticipated demand for new assurance services related to systems reliability. These services are deemed necessary in the evolution of systems towards online audit and assurance. The steps evolving towards a continuous audit encompass a new product under the umbrella of the SYSTRUST denomination – system reliability assurance (including software reliability, infrastructure reliability, process reliability and data reliability.) This proposed service is still at a conceptual stage, with numerous barriers to overcome. It illustrates however, a
2 Adapted from the ASEC report (see AICPA, 1997, page no.?), emphasis added.
5 Kogan, Sudit & Vasarhelyi / JIS
growing tendency on the part of the profession to provide services that bridge the route towards COA.
2.4 CICA/AICPA Committee on Continuous Auditing
The CICA and later the AICPA established a committee chaired by Richard Wood to examine ―continuous auditing‖. The committee submitted its report in December 1998 Continuous Auditing, Research Report, 1999. This report discusses the nature, purpose, scope and fundamentals of a continuous audit. Subsequently, the report deals with more complex continuous audits and draws a set of conclusions. It concludes with the following statement:
―This study has discussed a conceptual framework for continuous audits in general,
and described some significant issues that would need to be addressed in
performing such services. If some of the significant hurdles associated with
continuous audits can be overcome, there are likely many types of subject matter
regarding which an auditor could add significant value to an entity by performing a 3continuous audit.‖
The report provides examples of potential continuous auditing services as summarized below.
3 Chapter 5, draft 3 of CICA / AICPA, 1999.
6 Kogan, Sudit & Vasarhelyi / JIS
； Continuance assurance regarding the authenticity, integrity, non-repudiation of
electronic commerce transactions in connection with the AICPA/CICA WebTrust Seal
； Continuous assurance on controls over electronic commerce systems. ； Continuous assurance regarding compliance with debt covenants.
； Continuous assurance regarding security over Web sites containing reports on
significant decision-making information.
； Continuous assurance regarding the effectiveness of controls over publicly accessible
databases for electronic commerce and other purposes.
； Continuous assurance regarding on time delivery and quality of products being sold. ； Continuous assurance on the entity’s going concern status.
Traditional financial information
； Continuous assurance on specific financial information such as inventory levels,
receivables balances, amounts and age of accounts payable and other debts. ； Continuous assurance on mutual fund unit values, including assurance on effective
controls over the unit-holder system
； Continuous audits of financial statements.
； Continuous assurance on estimates and reserves.
； Continuous assurance regarding marketing information such as sales of a new product
by a software vendor.
； Continuous assurance regarding media ratings, hits to the Web site, and banner
Other types of information
； Continuous assurance on rates of pollution emission.
； Continuous assurance on any key performance indicators for an entity, possibly using
3 Some Experiences in Continuous Online Auditing
In 1991, Vasarhelyi and Halper (Vasarhelyi & Halper, 1991) focused on the “Continuous
Process Auditing System” (CPAS) designed to deal with the problems of auditing large paperless database systems. It developed a methodology for continuous auditing and described its implementation at AT&T.
The CPAS methodology was designed to measure and monitor large systems, drawing key metrics and analytics into a workstation environment. The data were displayed interactively, providing auditors with a work platform to examine extracted data and prepare auditing reports. CPAS monitored key operational analytics, compared these with standards, and rang alarms when necessary. Data collection, performed in the shadow of the corporate legacy system, was based on scanning patterns of reporting data, and on 01/27/11 on-aud11.doc
7 Kogan, Sudit & Vasarhelyi / JIS
inserting those patterns in a relational database which supported its ―advanced audit
decision support tool.‖ To the best of our knowledge, CPAS (see also Halper, Snively & Vasarhelyi, 1992; Vasarhelyi, Halper & Ezawa, 1991) is the only operational COA system in actual use whose architecture is described in detail in scholarly publications.
The CPAS effort entailed the continuous audit and monitoring of AT&T billers that were processed at four large data centers in different parts of the nation. The CPAS process used a ―measurement‖ methodology to capture data and to feed its ―Advanced Decision Support System.‖ The ―measurement‖ method of data provisioning can be contrasted with the ―monitoring‖ data provisioning that actually draws information from direct computer processes while they are being performed.
8 Kogan, Sudit & Vasarhelyi / JIS
System Operational Reports
OperationalDF-level 1DF-level 1DF-level 1Report
AlarmDF-level 0Data Flow DiagramsDatabase
Figure 1: CPAS Architecture
The CPAS architecture is described in Figure 1. Systems reports, regularly distributed to process management, are also mailed to the CPAS workstation. Upon arrival, the appropriate data is filtered out, extracted, and placed in a relational database. This relational database is then utilized to perform the analytic functions, which define the Continuous Audit Process in CPAS. The system relates actual data to many standards through analytics and issues alarms where substantive discrepancies are found.
3.2 Other Experiences
3.2.1 Fund Radar
Fund Radar is an actual system used at KPMG to audit mutual funds. The principles of operation are similar to the ones in CPAS with industry averages drawn from an online source and serving as benchmarks. The mutual funds industry is particularly suitable for COA as three vendors supply software to most funds in the industry. Consequently, three software implementations of Fund Radar with similar analytics and different data provisioning could conceivably be sufficient for the majority of the firms of the industry. 01/27/11 on-aud11.doc
9 Kogan, Sudit & Vasarhelyi / JIS
3.2.2 Ernst & Young
The accounting firm Ernst & Young (E&Y) is using online auditing and monitoring in several applications. In particular, they use online monitoring of a client's network for network monitoring and security purposes and are developing a CPAS-like application using HMOs as the application domain. HMOs, as in the mutual fund industry example above, have one software package with substantial market share. Consequently, it makes it easier for E&Y to capitalize on COA investment and deploy it in other HMO clients that use the same software.
3.2.3 Bank case
A local bank in Spain has developed a suite of applications programmed in COBOL within legacy systems, which create analytics relating products, customer care, marketing and risk management. Although the applications are not real time, many monitoring functions are performed during system operations.
4 Implications and Research Issues
Since COA is in its embryonic stage, numerous problems and research issues are bound to arise. We outline below some of the more important issues as we see them now. We start by discussing research issues related to methods of performing COA, then proceed with
the exploration of factors affecting COA, and finally conclude with discussion of major
consequences of COA. This same schema is used in the research summary table presented later (table 1).
4.1 Methods of Performing Continuous Online Auditing
Continuous online auditing requires an elaborate architecture anchored in a more intricate and formal structure as compared with traditional auditing. Knowledge ware provides some system understanding for assurance purposes. A COA system should be designed very carefully to take full advantage of its benefits, and to minimize the residual inherent problems.
4.1.1 Audit Risk Evaluation
Audit risk measurements and estimates of traditional nature can be arbitrary and judgmental. The advent of COA and its extensive data collection and monitoring features brings new meaning to actual risk measurement. Quantitative techniques that will substantially anchor audit risk measurements and estimates with real empirical values must be developed. This opens new venues for this well-established area of auditing research. Moreover, automation of internal control representation and embedded audit modules can further improve COA.
Research Issue: The use of COA calls for the development of new or refinement of the existing audit risk model which assumes periodic rather than continuous auditing. 01/27/11 on-aud11.doc
10 Kogan, Sudit & Vasarhelyi / JIS
4.1.2 Data Capture
COA systems need elaborate data capture mechanisms that supply the enterprise data for auditing. Today, these data capture mechanisms have to be custom-made for individual audit clients. Enterprise resource planning systems (e.g., SAP, PeopleSoft, Oracle) and generic industry software (e.g., Funds Radar) may allow for actual generation of specific records/reports designed to support audit analytics for the COA process. These are clearly superior to the measurement approach and provide a degree of standardization that improves the economics of COA. Standard formats for enterprise data will greatly simplify data capture problems of COA.
Research Issue: Explore and design standard formats of enterprise data to facilitate data capture for COA. The possibility of using XML for defining such standard formats for presentation of accounting information on the Web should be investigated. Scope of Auditing
Research Issue: COA systems are potentially capable of reprocessing or parallel
processing the whole population of business transactions. Investigate whether and when the complete reprocessing of the entire population of business transactions is feasible and desirable. COA allows real-time decisions concerning the level of review desirable for a particular transaction. The desirability of auditing larger samples, and the level of rule-based scrutiny of transactions of diverse populations, will impose constraints on the design of COA systems.
Research Issue: Investigate the tradeoffs between the frequency of auditing and the scope and diversity of its tasks (e.g. expanding audit to cover non-financial variables like intangibles or quality). An extension of the above investigation would entail developing theoretical models of COA that relate formal specifications of a COA system with various audit objectives.
4.1.3 Systems Audit
The field of COA inherits a long-standing debate from EDP auditing about whether to audit the information system or to audit the data flowing through that system. There seems to be a growing consensus that the information system has to be audited. Focus on information system auditing will provide important assurance on the quality of data fed into the COA system.
Research Issue: Determine the tradeoffs and complementarities between system structure auditing and transactions auditing. It would be important to analyze whether both transactions and system structure have to be controlled and subjected to high frequency auditing. Since the enterprise information system can be assumed to be fairly stable over time, its high frequency auditing can be simply reduced to continuously monitoring that the system has not changed (e.g., by using cryptographic techniques of digital signatures). In traditional EDP auditing this function would be accomplished by controlling the size of the executable files.