The use and misuse of intelligent systems in accounting: the risk of technology dominance

Designing intelligent systems to enable less experienced staff to make decisions normally made by more experienced staff is possibly not a good strategy. However, there appears to be potential for success in using intelligent systems to complement and support experts’ decisions.

by Stewart Leech

Judgment and decision-making in accounting

Accountants and auditors are called upon daily to use their judgment to make crucial decisions. However, questions arise. How good is that judgment? What factors affect it? How do they combine and mentally weigh the numerous factors that lead them to make a decision? What do we really know about how an auditor makes a going-concern judgment or about how an insolvency practitioner makes a decision to trade on or liquidate a business in financial difficulty?

Over the past twenty years, major accounting firms turned to technology to assist them with making such judgments. Computer-based intelligent systems were developed with the aim of making more consistent decisions and sharing expertise; and in the hope that novice staff could make the same decisions as experts. What do we know about the use of intelligent systems in accounting and how might they be used more effectively?

An intelligent system is a computer-based system intended to replicate the decisions of a human expert. We expect an intelligent system to show some form of intelligence – that is, it stores expertise and, given the factors about a particular case, undertakes reasoning and makes recommendations. They include the audit support systems used by the Big Four and other major audit firms, although some are more ‘intelligent’ than others.

The Theory of Technology Dominance

In theory, it was assumed that intelligent systems would allow relatively junior or novice staff to make the same decisions as experts. This assumption was questioned in the Theory of Technology Dominance, proposed in the late 1990s by Arnold & Sutton as ‘a model for understanding the conditions under which success (of intelligent systems) is more likely to occur.’ The theory attempts to understand the impact of intelligent systems on a decision maker’s judgments, including the short-term impacts and the long-term implications.

Firstly, a basic requirement of success for using an intelligent system is reliance on the system by a user. Reliance implies two conditions – acceptance and influence on the decision outcomes. To enhance the likelihood of reliance on the system:

The task should be highly complex;
The system should be familiar; and
There should be good cognitive fit between system and user.

The second part of the theory explores the short-term impacts. Here, the theory posits that when the expertise of the user matches the level of the intelligent system, decision-making will be enhanced; but where the intelligent system has more expertise than the user, the system can lead to poorer decisions. Thus, technology dominance is the state of decision-making whereby the intelligent system, rather than the user, controls the decision-making process, and a user with limited expertise is unable to use the system properly and might misinterpret its output.

The theory then goes on to examine the epistemological implications of using intelligent systems in the long term. Here, the theory predicts that continued use of intelligent systems could affect accounting expertise negatively in the longer term. How do we test a theory that addresses the use and misuse of intelligent systems in accounting? What intelligent system would we use? And in what field of accounting? While work on most intelligent systems has been in audit and tax, my colleagues and I decided to concentrate on a little-researched field in accounting that relies on substantial human judgement – that of corporate recovery and insolvency.

Corporate recovery and insolvency

We set off on a journey to first build an intelligent system (“INSOLVE”) that would allow insolvency practitioners to input factors about a company in financial distress, while the system would use its expert knowledge to make a recommendation on how to proceed – to either liquidate or trade the company on – and to provide the reasons and explanations for the decision.

We were motivated by two objectives: firstly, a cognitive modelling rationale since we were interested in understanding how expert insolvency practitioners made decisions about companies in financial distress; and secondly a behavioural science rationale that could test the theory of technology dominance. Under what conditions is an intelligent system likely to lead to improved decision-making?

In building INSOLVE, knowledge was acquired from 23 experts from major accounting firms and banks. INSOLVE was then extensively validated against insolvency cases, resulting in a high level of agreement between the experts and INSOLVE.

How are companies in financial distress dealt with by insolvency practitioners? An initial decision is made to either liquidate or trade on the business. The objectives of trading-on are:

Reconstructing the business prior to returning the business to directors;
Enhancing and/or preserving the sale value of the business as a going concern; or
Complete work prior to liquidation.

Such decisions may extend over a considerable period and are made on the basis of both financial information and qualitative judgments about the business, stakeholders and the business environment.

The initial assessment is based on several factors: the business has ceased or cannot become viable; there is no cash and no way to generate cash; key staff vital to the business have left and cannot be replaced; or essential customers and/or suppliers will not support trading-on. If the decision is made to trade-on, an assessment is made of the stakeholders – including the directors, staff, secured creditors, customers, suppliers and unions – and the financial situation, which involves a comparison of the auction value of the assets, the sales value as a going concern and the future projected profitability of the business. All of these factors were included in INSOLVE. Once provided with the facts of an insolvency case, the system uses its inference engine to undertake reasoning and produce a recommendation. INSOLVE produces a report that recommends either liquidate, sell as a going concern or hand the business back to the directors.

Testing the Theory of Technology Dominance

We then used INSOLVE to test the short-term propositions of the Theory of Technology Dominance, that is:

When there is a strong match between the user and an intelligent system, the judgment of the user will improve;
When there is a mismatch between the user and an intelligent system in terms of expertise, the risk of poor decision-making increases.

The difficulty in testing these two propositions is the ambiguity in defining a better or worse decision in domains that are highly subjective. One of the challenges was to place the decision environment into an observable and measurable context.

One approach recommended in decision-making research is to focus on a specific source of judgment error. We used a similar approach by examining certain specific types of judgment error during the completion of a complex task by insolvency practitioners – some aided by INSOLVE and some without INSOLVE. The research hypotheses can be summarised as follows: for novices being aided by INSOLVE, there will be an increase in judgement error; for experts being aided by INSOLVE, there will be reduced judgement error.

A training session (experiment) was used to test the hypotheses. A real, reconstructed insolvency case was used with 80 insolvency practitioners having access to INSOLVE and with 87 who did not use INSOLVE. Experts were partners and managers. Novices were staff and seniors. In each of three stages of the insolvency case, the participants were asked to give an assessment of whether they would trade-on or liquidate. The results indicated the existence of a detrimental effect of the intelligent system on the decision-making processes of novices. On the other hand, the intelligent system was effective at reducing the judgment error in the decision-making processes of experts. These results supported the Theory of Technology Dominance.

INSOLVE was a basic intelligent system lacking a substantial explanation facility. We needed to address the question: would the provision of a fully functional explanation facility in an intelligent system have an effect on the results so far? This meant providing users with four types of explanations: definitions, rule trace, justification and strategic. Once developed, we tested INSOLVE II (with the explanation facility) in two further studies.

The first study was designed to see if and how the explanation facility affected the decision-making behaviour of both novices and experts. The results showed overall that both novices and experts are more likely to rely on the recommendation of the intelligent system when explanations are provided. However, novices still tended to accept INSOLVE’s recommendation and move towards reliance, supporting the previous research.

The second study used INSOLVE with major accounting firms in Singapore and compared the results with Australia. Intelligent systems developed in one country are often used in other countries by large accounting firms. The results showed that the overall judgements were no more consistent between Singapore and Australia when using INSOLVE. However we did find that the intelligent system developed in Australia altered the way the Singaporeans evaluated some of the evidence of an insolvency case, leading them to be more in line with Australian decision-making processes. INSOLVE did change the Chinese culture/attitude to aspects of insolvency decisions, which means that an intelligent system could lead to more consistency in judgement across cultural boundaries, if indeed that is considered desirable.

Other evidence from the US using intelligent tax systems generally supported our results. In summary, designing intelligent systems to enable less experienced staff to make decisions normally made by more experienced staff is possibly not a good strategy. On the other hand, there appeared to be potential for success in using intelligent systems to complement and support experts’ decisions.

Finally, I turn to the possible long-term implications of the widespread use of intelligent systems in accounting. Here, the Theory of Technology Dominance posits that continued use of an intelligent system will result in the de-skilling of users’ abilities and have a negative effect on the growth and advancement of knowledge.

There has been virtually no research in this area until recently, partly because it is a very difficult question to research. There was some evidence from one of the US tax studies, where participants using the intelligent tax system had difficulty in completing the tax returns manually, whereas those who completed manual tax returns first could then easily use the intelligent system.

We tested the association between the extent of decision-support embedded in the audit support systems of three major audit firms and the declarative knowledge processed by long-term users (Dowling, Leech, and Moroney, 2007). We required auditors, without the aid of their firms’ audit support systems, to list the key business risks common to clients in an industry familiar to them. We found that auditors who normally use an audit support system that was not an intelligent system were able to list more relevant risks than auditors who normally use an audit support system that was more of an intelligent system.

While this was a very simple exploratory study, it does ring some alarm bells in the direction of the theory, and suggests that the way audit support systems are designed has a role to play in providing sufficient opportunities for auditors to develop their knowledge.

Summary

Intelligent systems and the way they are designed do have an impact on the decision-making behaviour of accountants. The expectation that novice accountants can use intelligent systems and perform like experts is not supported by the evidence and can be a possible dangerous assumption in the use of such systems. There needs to be a good match between the expertise embodied in the intelligent system and the user. A good explanation facility can make a difference to the decisions being made, and an intelligent system developed in one country can affect the decisions being made in a country of a different culture. Finally, while the jury is still out on the longer-term consequences, there are possible alarms bells that we need to heed.

Top ^

A condensed version of the 68th Annual CPA Australia/University of Melbourne Research Lecture delivered at the University of Melbourne on 10 September 2007.

Professor Stewart Leech is in the Department of Accounting and Business Information Systems at the University of Melbourne. This is a report of research into intelligent systems in accounting that Professor Leech has undertaken over many years in collaboration with co-researchers, Phil Collier, Vicky Arnold, Steve Sutton, and more recently Carlin Dowling.