International Journal of Information Management 37 (2017) 1380–1390 Contents lists available at ScienceDirect International Journal of Information Management journal homepage: www.elsevier.com/locate/ijinfomgt Revisiting the information audit: A systematic literature review and synthesis Robert B. Frost, Chun Wei Choo∗ Faculty of Information, University of Toronto, 140 St. George. St., Toronto, ON M5S 3G6, Canada a r t i c l e i n f o Article history: Received 25 May 2016 Received in revised form 8 September 2016 Accepted 11 October 2016 Available online 25 October 2016 Keywords: Information audit Information management Information quality Quality management Literature review methodology a b s t r a c t The purpose of this paper is to revitalize the theory and practice of the Information Audit (IA) by connecting it with recent developments in information management theories and methods While the IA is a powerful information management practice, the methods and applications of IA have not been wedded to recent developments in the study of information management capability and information quality management. This study addresses that gap. The paper also introduces and applies a systematic methodology for conducting literature reviews that combines concept mapping, review scoping, and a structured search and analysis process. The resulting search in Scopus and Proquest and subsequent analysis of the recent literature (2011–2016) on IA and quality, evaluation, measurement, and maturity in the context of information management yielded the following findings and recommendations. IA research and practice could do well to: pursue contingency frameworks rather than seek universal standardization; investigate the relationship between IA and the dimensions of information quality and information management quality; undertake case studies that apply more foundational IA methodologies in full; develop theories of IA maturity and IA maturity modelling methods; recognize that measurement and evaluation of information management quality and information quality are necessary elements of the IA and should be explicitly incorporated into IA methodology. © 2016 Elsevier Ltd. All rights reserved. 1. Introduction The information audit (IA) has been defined by Buchanan and Gibb (2007) as “a holistic approach to identifying and evaluating an organization’s information resources and information flow, in order to facilitate effective and efficient information systems” (p. 171). The IA provides “an invaluable structure of knowledge” in formulating an organizational information strategy (Orna, 2004; p. 105), and as Buchanan and Gibb (2008) note, the IA’s influence on information management, technology, systems, and content are well established in much of the foundational literature on IA (Buchanan & Gibb, 1998; Burk & Horton, 1998; Henczel, 2001; Orna, 1999). In its fullest form, the IA encompasses all the methods and tools needed to catalogue, model, evaluate, quality-control, and analyze an organization’s information assets and information management. In their comparative analysis of the common IA methodologies established in the late 1990s and early 2000s, Buchanan and Gibb (2008) propose a seven step methodological baseline for the IA and find that the methodologies of Orna (1999) and Henczel (2001) ∗ Corresponding author. E-mail addresses: robert@rbfrost.com (R.B. Frost), cw.choo@utoronto.ca (C.W. Choo). cover every step of the baseline. Henczel’s methodology involves seven stages (pp. 18–19): 1) Planning the audit by setting objectives, identifying stakeholders, scoping the project and allocating resources, selecting a methodology, and developing communications and business strategies. 2) Collecting data in an information resources database, designing and distributing questionnaires, holding focus groups, and conducting personal interviews. 3) Analyzing the collected data and research. 4) Evaluating gaps and duplications in information, mapping and interpreting information flows, formulating recommendations, and developing a change management plan. 5) Communicating recommendations to stakeholders through written reports, presentations and seminars, webpages, and personal feedback. 6) Implementing recommendations through implementation programs, formal change plans, post-implementation strategies, and information policies. 7) Ongoing information service management to measure and assess the changes through a regular information audit and service evaluation cycle. http://dx.doi.org/10.1016/j.ijinfomgt.2016.10.001 0268-4012/© 2016 Elsevier Ltd. All rights reserved. R.B. Frost, C.W. Choo / International Journal of Information Management 37 (2017) 1380–1390 1381 Other established methodologies cover all seven stages of the baseline in different sequences or with added stages (e.g. Orna, 1999), do not provide guidance for the planning the audit (e.g. Buchanan & Gibb, 1998), or do not provide guidance for planning or change management following a report of the audit’s findings (e.g. Burk & Horton, 1998). Griffiths (2012) notes that the traditional, established IA methods are generally concerned with Hard IA, involving “notions of compliance, regulation and accuracy” (p. 43), rather than improving the usability of information assets, increasing the efficiency of information use, or finding opportunities for business innovation by changing information management practices (what Griffiths calls Soft IA). Henczel’s later emphasis on user-centric interviews and focus groups, information flow mapping, as well as integration with change management and service management showed that the IA could have applications beyond the “hard” realm of accountability and compliance. While the IA is a powerful information management practice, the methods of both hard and soft IA described in the literature have not been wedded to recent developments in the study of information management capability (as described by Mithas, Ramasubbu, & Sambamurthy, 2011) and information quality management (as conceptualized in the framework of Baskarada & Koronios, 2014). Furthermore, measurement and evaluation techniques such as information asset registration and maturity modelling remain largely absent from most IA methodologies, their synthesis with IA representing a direction for further research (Griffiths, 2012). With the advent of widespread digital transformation and the rise of big data, it is now more important than ever for organizations to have methods and tools for auditing and evaluating their information assets. With the aim of better connecting the IA literature with recent information management theories and methods, this paper poses three research questions for investigation: RQ1: What recent research (from 2011 to 2016) has been done on IA? RQ2: What recent research (from 2011 to 2016) has been done on quality, evaluation, measurement, and maturity in the context of information management? RQ3: In the future, how might IA researchers and practitioners synthesize the recent research on IA with the recent research on information management quality, evaluation, measurement, and maturity? This paper will address the above three research questions through a systematic literature review, present and discuss the results of the review, outline the implications of the review for IA researchers and practitioners, and describe the limitations of the review. To better aggregate the large volume of articles under review, the discussion of results will provide a high-level overview of recent trends in IA methods and theories rather than a complete analysis of every reviewed article. The paper offers unique contributionsin both its literature review methodology and its findings on recent trends in IA and information management quality, evaluation, measurement, and maturity. 2. Concept mapping In preparation for the systematic literature review, a conceptual map of the linkages between IA, information quality, information management quality, evaluation, measurement, and maturity was created through a preliminary analysis of pivotal works from the IA and information management literature. Mindful of the fact that researchers and practitioners in different domains often utilize different terminologies for similar concepts, the need to establish working understandings of quality, evaluation, measurement, and maturity was recognized. Applying the working understandings to the conceptual mapping process led to the development of an analytical framework, which would ensure a consistent approach to analysis in later stages of the literature review. Two perspectives on quality were considered: information quality and information management quality. Information quality was understood with respect to Floridi’s (2013) characterization of information quality as the categories, dimensions, purpose-depth, and purpose-scope that shape a unit of information. Information management quality was understood with respect to Mithas, Ramasubbu, and Sambamurthy’s (2011) characterization of highquality information management capability as “the ability to provide data and information to users with the appropriate levels of accuracy, timeliness, reliability, security, confidentiality, connectivity, and access and the ability to tailor these in response to changing business needs and directions” (p. 238). Measurement and evaluation were understood interchangeably as appraisals of a quality criterion with reference to a specific performance indicator (e.g. the appraisal of a form’s accessibility with reference to the quantity or severity of access barriers it contains). Maturity was understood with respect to Marchand, Kettinger, and Rollins’ (2001) view of information orientation maturity as a mixture of highly developed information capabilities. Benchmarking was understood as an alternative approach to maturity modelling in which an organization’s internal measurements are compared to external measurements, rather than compared to internal targets. With the key concepts for analysis defined, their links to an IA were established in relation to Buchanan and Gibb’s (2007) description of the IA as “a holistic approach to identifying and evaluating an organization’s information resources and information flow, in order to facilitate effective and efficient organizational information systems” (p. 171). In deconstructing Buchanan and Gibb’s description of the IA process, four implications for the linkages between an IA and the other concepts under investigation in this paper become evident: Implication 1: The IA requires an evaluation of information resources and flows. Implication 2: The evaluation of information resources entails a measurement of information quality. Implication 3: The evaluation of information flows entails a measurement of information management quality. Implication 4: The measures of information quality and information management quality are determinants of maturity, contributing to effective and efficient information systems. The four linkages derived from Buchanan and Gibb’s (2007) definition were formalized in a concept map, shown in Fig. 1. The concept map illustrates the causal relationships between each of the concepts under analysis, and also provides a basic rationale model of IA goals and IA-maturity links for use in future maturity modelling. With working understandings of the concepts under investigation established and formalized in the concept map as a result of the preliminary analysis, it was possible to proceed with the creation literature review using a well-defined and consistent analytical framework. 3. Methodology The literature review was systematic and performed in accordance with the methodology described by vom Brocke et al. (2009). The methodology is rigorous in its approach, prescribing five phases to the systematic literature review: scoping, conceptualization, literature search, analysis/synthesis, and stating the expected contributions of the review to the broader research agenda. This literature review largely followed the prescriptions of vom Brocke 1382 R.B. Frost, C.W. Choo / International Journal of Information Management 37 (2017) 1380–1390 Fig. 1. Concept map of the linkages between IA and the other concepts under investigation in this paper, with reference to Buchanan and Gibb’s (2007) definition of the IA. et al., with some liberties being taken to adapt the methodology to the specific needs of this study. To scope the review, vom Brocke et al. recommend using the taxonomy of literature review characteristics proposed by Cooper (1988). Table 1 utilizes Cooper’s scoping table to outline the overall scope of this review, producing the following methodological decisions: Decision 1: To maintain a focus on methodological and conceptual trends, the review would focus on methods and theories of IA, information quality, information management quality, measurement, evaluation, maturity, and benchmarking. A thorough discussion of the individuals, industries, and/or sectors involved in the application of methods and theories would be beyond the scope of the review, as would detailed analysis and criticism of the research outcomes of an empirical study. Decision 2: The goal of the review would be to integrate those methods and theories into insights on recent developments in IA and points of connection between IA and other recent areas of information management research. Decision 3: The results and discussion of the review would be organized conceptually, with reference to the conceptual mapping performed during the review’s preliminary analysis. Decision 4: The perspective of the review would ultimately be argumentative rather than a neutral representation of the literature, espousing a position on the best paths forward for future IA research based on the review’s results. Decision 5: The audience for the review would be specialized IA scholars and practitioners. Decision 6: The extent of the review’s coverage of the literature would be representative, seeking a large enough sample of the literature to be representative of current trends rather than an exhaustive search of all recent literature. After scoping the review, vom Brocke et al. recommend conceptualizing the topic(s) to be studied—this phase of the methodology development was already performed in the preliminary analysis, with the creation of the concept map. Accordingly, the concept map provided a conceptualization of the topics, which could be used to inform the analysis and synthesis phase. In developing the literature search process, an adaptation of the search process proposed by vom Brocke et al. was used, specifically, the adaptation used by Kowalczyk, Buxmann, and Besier (2013) in their systematic review of the business intelligence and analytics literature. The literature search process began with the creation of inclusion criteria to determine the conditions under which an Table 1 The scope of this review using the scoping table recommended by Cooper (1988), with this review’s characteristics highlighted in grey. R.B. Frost, C.W. Choo / International Journal of Information Management 37 (2017) 1380–1390 1383 Fig. 2. The information flow of the literature search process used in this review, with the entire process being performed once to address RQ1 and a second time to address RQ2. article would be included in or excluded from analysis. The five inclusion criteria decided upon were: Criterion 1: The article was published in a peer-reviewed scholarly journal, a trade journal, or the proceedings of an academic or trade conference. Criterion 2: The queried data source provided full text of the article or a link to the full text of the article. Criterion 3: The article was written in English. Criterion 4: The article was published between 2011 and 2016. Criterion 5: The article examines one or more of the concepts included in the concept map from an information management perspective (papers on the topic of maturity or quality measurement from an operations management perspective with no relationship to information management, for example, were excluded). Next, query structures were iteratively developed through trial and error. Query structures were experimentally entered into the search engines of Scopus and ProQuest until a set of query structures, which were optimal for addressing the research questions and inclusion criteria, had been identified. Two structured queries were created to extract articles from Scopus and ProQuest, which could provide potential answers to RQ1, and another two structured queries were created to extract articles from Scopus and ProQuest, which could provide potential answers to RQ2. All four query structures are documented in Appendix A of this paper. RQ3 was to be addressed through a discussion of the literature review’s results rather than through a query of its own. To ensure the results more accurately reflected the inclusion criteria, the queries were accompanied by manual selections of inclusion criteria in the Scopus and ProQuest user interfaces: only articles published between 2011 and 2016 in scholarly journals, trade journals, and conference proceedings were included in the search results. With the query structures optimized and defined, two separate instantiations of the literature search process were carried out in response to RQ1 and RQ2, respectively. Both search processes were carried out in careful conformance with a predefined process model (depicted in Fig. 2). Both search processes consisted of five steps: Step 1: After querying the Scopus and ProQuest search engines, article information was extracted from Scopus and ProQuest and results counts were recorded in an extraction log. Step 2: The extracted articles had their titles and abstracts analyzed to assess their degree of fit with the concept map and their potential value in answering the research question at hand. Articles that were not well-suited to the conceptualization of the topic or had little value in answering the research question at hand were excluded from further consideration, and the count of articles remaining after the title and abstract analysis was recorded in the extraction log. Step 3: The remaining articles had their full text analyzed in full to assess their degree of conceptual fit and potential value in greater detail. Again, articles that were not well-suited conceptually or had little value in answering the research question at hand were excluded from further consideration. The count of articles remaining after the full text analysis was recorded in the extraction log. Table 2 Concept matrix template used to record and count conceptual information about the articles. The actual concept matrix resulting from the review can be found in Appendix B. Concept Matrix Template Year Author(s) Research Focus Info. Audit IM Quality Info. Quality Measure- ment/Evaluation Maturity/ Bench- marking (Year) (Authors) 1384 R.B. Frost, C.W. Choo / International Journal of Information Management 37 (2017) 1380–1390 Table 3 Extraction log of the results from each step of both the literature search addressing RQ1 and the literature search addressing RQ2. Extraction Log Research Question Database Name Structured Query Results Title/Abstract Results Full Text Results RQ1 Scopus 98 18 9 RQ1 ProQuest 194 6 4 RQ2 Scopus 289 5 4 RQ2 ProQuest 416 11 6 997 40 22 Table 4 Total counts of the amount of articles adopting particular research and conceptual foci from the pool of 22 articles remaining after the full text analysis. Concept Matrix Totals Research Focus Article Count Outcomes 4 Methods 15 Theories 14 Applications 9 Concept Matrix Totals Conceptual Focus Article Count Information Audit 12 Information Management Quality 3 Information Quality 10 Measurement/Evaluation 13 Maturity/Benchmarking 5 Step 4: All of the articles remaining after the full text analysis had their authorship information and year of publication recorded in an adaptation of the concept matrix template (depicted in Table 2) provided by Webster and Watson (2002). The articles were then reviewed in full a second time to discern their focus or foci with reference to the categories the scoping table: outcomes and results of research studies, methods of conducting research and analysis, theories that conceptually model and explain the nature of a target of analysis, or applications of research methods and theories to specific case studies or contexts. The second full text review was also used to discern the extent to which they analyzed the concepts outlined in the concept map. The research focus/foci and concept map relationships of each article were recorded in the concept matrix. Step 5: The counts of each research focus and concept map relationship were tallied and used for analysis and synthesis in Section 5 of this review. The literature searches in response to RQ1 and RQ2 were both carried out over the span of one week in March 2016. The rigor, reproducibility, and research question orientation of the methodology was expected to answer all three research questions posed by this study, offering three sets of unique contributions to the IA research agenda. 4. Results Between both search processes, 997 initial results were returned, 40 remained after title and abstract analysis, and 22 remained after full text analysis. In accordance with the methodology, a breakdown of the step-by-step results was recorded in an extraction log, which can be found in Table 3. All 22 of the articles remaining after the full text analysis had their authorship information and year of publication entered into a concept matrix. Their full text was then reviewed a second time to determine the research and conceptual foci of each article. After the research and conceptual foci of each article had been recorded in the concept matrix, the results of the review were sorted in descending order based on year of publication in order to facilitate analysis of trends in the literature over time, and the results of the research focus and conceptual reviewing for each article were tallied. Table 4 depicts the total counts for each research and conceptual focus, which were observed in each of the 22 reviewed articles. Note that it is possible for a single article to have multiple research or conceptual foci, and thus the research and conceptual foci counts do not add up to 22. The research foci most often represented in the 22 reviewed articles were methods and theories, with 15 articles adopting a methods focus and 14 articles adopting a theories focus. The conceptual foci most often represented in the 22 reviewed articles were measurement/evaluation and information audit, with 13 articles analyzing and discussing measurement/evaluation in detail and 12 articles analyzing and discussing information audit in detail. A more detailed discussion of the trends revealed by the concept matrix and the implications of those trends for researchers and practitioners will be presented in later sections of this paper. A full version of the concept matrix with research and conceptual foci counts for each of the 22 articles can be found in Appendix B of this paper. Considering the initial queries for RQ1 and RQ2 resulted in a total of 997 articles being extracted from Scopus and ProQuest, it may seem surprising that so few articles remained after full text analysis was performed. The significant gap between initial query results and full text results is in large part attributable to the fact that many initial query results did not approach the concepts under investigation from an information management perspective, and as a result, use terms such as “information audit”, “information quality”, and “measurement” in senses, which are unrelated to the prevalent conceptualizations of those terms within the information management literature. For example, in one excluded article, which had its title and abstract analyzed in response to RQ1, Yanjun (2013) discusses highly technical methods of auditing the functionality of computer systems, and in doing so, uses the phrase “massive information audit” (p. 409) to describe the need for computer system auditors to develop new computer auditing methods in response to the big data paradigm. In another example of an excluded article, which had its title and abstract analyzed in response to RQ2, Orozco, Tarhini, Masa’deh, and Tarhini (2015) briefly discuss concepts of measurement, evaluation, and maturity, but only in the context of improving IT governance architectures through increasing information system and business alignment. The article does not analyze those concepts of measurement, evaluation, and maturity models in much detail, nor does it link its domain-specific perspective to a broader information management perspective or the relationships expressed in this paper’s concept map. Despite a smaller sample of results remaining after the full text analysis than was initially anticipated, the results are representative enough of recent trends in the IA literature and literature related to the concept map established in the preliminary analysis to be able to answer RQ1 and RQ2. Recent developments and trends in IA literature (addressing RQ1) and relevant information management literature (addressing RQ2) will be discussed in the following section with reference to the literature search processes, and ultimately, the findings from the two bodies of literature will R.B. Frost, C.W. Choo / International Journal of Information Management 37 (2017) 1380–1390 1385 be synthesized into implications for researchers and practitioners, which address RQ3. 5. Discussion Given the number of papers found in the literature search, it is not feasible to elaborate on the content of each item. Instead, we provide a big-picture view, which highlights the main contours of the IA and information management terrain covered by recent published research. In analyzing the literature, we sought out the main areas of research emphasis, as well as the areas that require research but have hitherto received little attention. 5.1. Recent literature on IA To address the first research question (RQ1), our analysis of the literature on IA leads us to make the following four observations that we believe characterize the state of recent research on the subject. 5.2. Firstly, much more attention has been given to theories and methods of IA than IA research outcomes and applications The recent theory-building observed in the IA literature always goes hand-in-hand with more prescriptive guidelines and methods of IA practice. For example, Fraser-Arnott (2014) develops a theoretical framework of technical, interpersonal, analytical, and knowledge competencies within librarianship, which—when fused with knowledge management techniques—form domain-specific methods and tools of knowledge auditing. A second example of IA theories and methods being fused include Buchanan and McMenemy’s (2012) discussion of the nature of elicitation, decomposition, and representation in process modelling methods used to model digital library information flow. In a third example, Griffiths (2012) analyzes of the fundamental, common characteristics between IA methodologies, the division between “hard” IA focused on compliance and “soft” IA focused on information mapping, and the manner in which that division and the characteristics of hard and soft IA are expressed in practical techniques of IA such as compliance management, information systems audit, and information asset registration. 5.3. Secondly, there has been limited research that focuses on IA applications and case studies Among the 22 articles remaining after the full text analysis, only five have a focus on IA applications (Ariffin, Latif„ Faudzi, Shariff, & Nadzir, 2014; Jones, Mutch, & Valero-Silva, 2013; Mircea, GhilicMicu, & Stoica, 2012; Ocholla, 2011; Vo-Tran, 2011). Of those five articles, only Jones et al. (2013) have a pure focus on application, applying the well-established IA methodology of Buchanan and Gibb (1998) to a case study of information flow mapping at the Nottingham City Homes public housing service. Their case study follows three steps: identifying the organizational context and process structure by reviewing business documentation; interviewing staff members to determine the processes/activities involved in their role, the information flows they mediate, and the information resources they utilize; analyzing and reporting on recommendations, then seeking the implementation of those recommendations. Jones, Mutch, and Valero-Silva conclude that the results of their application “affirm the value of the [IA] approach” (p. 297), but suggest that challenges exist in convincing key stakeholders to recognize the value of and engage with the information audit process. 5.4. Thirdly, there have been few notable attempts to extend or modify established IA methodologies for specific industries or specialized domain areas The research foci of the remaining four IA applications are divided between methods and applications, with the authors proposing their own extensions or modifications of established IA methodologies, then applying their modified methods to a case study. Ariffin et al. (2014) propose modifications of IA methods to better suit the electricity supply industry, determining that IA implementation is currently a challenge in the industry due to the limited linkage of existing IA frameworks with systems development processes, the lack of scoping or resource sampling guidelines in most IA methods, the lack of standardized IA methodologies, and the lack of change management guidelines to facilitate the implementation of IA-driven recommendations. Mircea et al. (2012) devise a variation of the IA referred to as a “decision process audit” with five steps: planning the audit, identification of key decisions, description of decision processes, evaluation of decision processes, and drafting the audit report. They then apply the decision process audit method to a case study of a public procurement contract for photocopy repair and maintenance services in the Bucharest University of Economic Studies. Ocholla (2011) leverages IA methods such as process mapping and resource inventory to propose research audit methods for mapping and auditing scholarly research, then conducts a research audit of the research projects and research output of the University of Zululand between 1994 and 2008. Vo-Tran (2011) combines Henczel’s (2001) IA methodology with an action research methodology, modifying the “implementing recommendations” stage of Henczel’s methodology to be better applicable to architectural design. Vo-Tran then applies the hybrid methodology to a case study of the architectural design process of a new building at an Australian university. 5.5. Fourthly, there is a near absence of research that focuses on IA implementation and outcomes Only one article extracted from the literature search has a focus on IA research outcomes: Carvalho and Esteban-Navarro’s (2016) study of intelligence system auditing, in which the results of a literature review, participant observation, and case study are triangulated, enabling the authors to propose empirically-grounded methods of auditing the informational and operational aspects of business intelligence systems. In their literature review, Carvalho and Esteban-Navarro separate audit methods into information resources audit, information audit, knowledge audit, communication audit, and intelligence audit, recognizing the unique value of each set of audit methods as part of a more holistic audit of intelligence systems. Their incorporation of “information ecology theory, information asset audits, user studies, knowledge management and intelligence studies” (p. 58) into traditional IA methods, inspired by the outcome of their research, implies that traditional IA methods must be responsive and amenable to the growing importance of knowledge management and business intelligence systems if IA is to remain a valuable information management practice in the current business environment. The lack of any outcome-focused empirical studies of IA implementation beyond Carvalho and Esteban-Navarro’s indicates a significant research gap for IA. Although many articles focus on methods and theories, which extend the capability of the traditional IA methods, there is little research being done to analyze, criticize, and develop new methods and theories out of the planning and implementation of specific IA projects. In studying applications of IA methods and theories, it is worth noting that compared to the early IA methodologies, the objectives of audits are increasingly taking a hybrid approach, auditing for hard IA reasons of compli- 1386 R.B. Frost, C.W. Choo / International Journal of Information Management 37 (2017) 1380–1390 ance and accountability, as well as soft IA reasons of usability and innovation. However, there does not yet exist a contingency model outlining which hard IA or soft IA methods an organization should adopt based on the audit context. 5.6. Recent literature on quality, measurement/evaluation, & maturity/benchmarking In response to the second research question (RQ2), and based on our analysis of the recent literature, we again make a number of observations that we believe characterize the state of recent research on quality, measurement/evaluation, maturity/benchmarking in the information management context. First, we note that the greatest research attention has been given to measurement/evaluation. Of the concepts from the concept map which were investigated in the literature search process addressing RQ2, the concept with the greatest attention paid to it was measurement/evaluation, with a total of 13 articles including it as a conceptual focus (see Appendix B for the full mapping of every article aligned with the measurement/evaluation focus). Of those 13 articles, however, only four discuss measurement/evaluation from an information management perspective in addition to discussing the concept of IA (Buchanan & McMenemy, 2012; Jones et al., 2013; Mircea et al., 2012; Shamel, 2014). Shamel (2014) situates the IA within a larger knowledge assessment process, first formulated by Henczel (2000), in which the information audit is preceded by needs analysis and followed by a knowledge audit. Shamel then suggests methods of measuring the gap between information and knowledge management goals and the actual performance revealed by the audit, as well as a list of common strengths and weaknesses to be vigilant for during evaluation of the audit results. Jones et al. (2013) describe the methods that they used to have staff members of an organization, which is undergoing an IA, evaluate their organization’s information resource usage with reference to a set of best practices. Buchanan and McMenemy (2012) discuss measures of success and common pitfalls in modelling information flows as part of an IA, such as failing to elicit vital information from staff, failing to decompose information flow processes appropriately, and underrepresenting less tangible activities or resources. Mircea et al. (2012) discuss the five steps of evaluating decision process quality as part of a decision process audit: identifying the dimensions of the managerial situation, developing alternatives, analyzing and evaluating alternatives, choosing and implementing the best alternative, and evaluating the overall quality of the decision. Other articles in the segment of measurement/evaluation literature, which do not share a conceptual focus on IA, tend to focus on the management of information systems. For example, Visser, van Biljon, and Herselman (2013) analyze frameworks and survey tools for evaluating the performance of management information systems in Further Education and Training colleges. Petter, DeLone, and McLean (2012) review the historical evolution of information systems success measures followed by the suggestion of research directions for new success measures. Mohammed and Yusof (2012) offer criteria for effective information quality management in health information systems. All of these articles present novel approaches to the measurement and evaluation of information systems, but are not connected to the literature on IA. However, all three of the above examples share an additional conceptual focus on information quality, indicating the importance of not only managing information systems, but managing the information they contain as well. Second, we note a surprising lack of overlap between information quality and IA as conceptual foci in the literature. On the whole, information quality also received a great deal of attention in the reviewed literature: 10 articles, which analyzed or discussed information quality, were reviewed in total. Of those 10 articles, seven conceptually overlap with measurement/evaluation (including the three examples of information systems management articles noted above). This close relationship between information quality and measurement/evaluation is somewhat unsurprising, given the fact that information quality is aptly defined by one of those 10 articles as “a measure [emphasis added] of where the information consistently meet [sic] all user expectations” (Gunawan & Suhardi, 2014, p. 1). The lack of overlap between information quality and IA conceptual foci, though, is more surprising: out of all of the articles reviewed, only Griffiths (2012) discusses both information quality and IA. In fact, in discussing the need to combine business intelligence and compliance functions to better manage information quality, Griffiths voices “concern that IA practitioners may be placing emphasis on the quality of the subject organization’s data processing when the key element should be an assessment of the quality of the data itself” (p. 45). While Griffiths does not provide a framework for assessing information quality as part of an organization’s IA, his concern about data and information quality being overlooked within IA practice signals the need for researchers and practitioners to begin bridging this conceptual gap. Third, there has been limited research attention on maturity/benchmarking in the information management context. In contrast to measurement/evaluation and information quality, the concept of maturity/benchmarking in the context of information management received relatively little attention in the reviewed literature: only five articles analyze and discuss maturity/benchmarking (Chuah, 2014; Gunawan & Suhardi, 2014; Zou, Flanagan, Jewell, & Tang, 2013; Chuah & Wong, 2012; Pranicevi´c, Alfirevi´c, & ˇStemberger, 2011). Following an initial study (Chuah & Wong, 2012), which determined that further exploration of the dimensions of business intelligence quality and maturity were needed, Chuah (2014) constructs a maturity model for enterprise business intelligence, linking the development of organizational and information-managerial capabilities to the maturity level of the enterprise business intelligence system. Gunawan and Suhardi (2014) create a theoretical and methodological framework for total information quality management, combining quality management principles from the total quality management and information quality management schools into an evaluable, five-level maturity model. Zou et al. (2013) create an information maturity model tailored to the decision-making needs of the construction industry, proposing that a unit of information’s overall maturity is composed of four measures: its format, informational value, informational quality, and the timeliness of its delivery to end users. Based on their empirical study of the relationship between information system maturity and hotel performance measurement, Pranicevi´c et al. (2011) introduce a maturity model for information systems in the hospitality industry. Despite the breadth of domain-specific theories and methods of maturity modelling displayed throughout the reviewed literature, no article was found with conceptual overlap between IA and maturity/benchmarking, leaving a gap in the research to be filled by a maturity model of IA practice. Finally, the concept with the least attention paid to it was information management quality. Only three articles analyze and discuss the concept (Fraser-Arnott, 2014; Sheriff, Bouchlaghem, El-Hamalawi, & Yeomans, 2012 ;Griffiths, 2012). Sheriff et al. (2012) explore the drivers of and barriers to effective information management in UK-based architecture and engineering organizations, determining from the outcome of their study that the ability to improve end products, the ability to improve processes, the effective transfer of learning, legal and regulatory compliance, and risk mitigation are all drivers of information management quality. However, Sheriff et al.’s unique perspective on information management quality does not share a conceptual focus on IFraser-Arnott (2014) and Griffiths (2012) do exhibit conceptual foci on both IA and informa- R.B. Frost, C.W. Choo / International Journal of Information Management 37 (2017) 1380–1390 1387 tion management quality, but their engagement with the drivers, barriers, and dimensions of information management quality is not as direct and thorough as Sheriff et al.’s. As a result, there is a gap in the research to be filled by an outcome-focused empirical study of the causal relationships between IA methods and information management quality. 6. Implications for researchers Our characterization of the recent literature in IA and information management allows us to identify the areas of emphasis in the research, as well as the gaps and opportunities for further research. Based on the discussion in the last section, we propose four directions for future IA research that would address RQ3, that is, how might IA researchers better synthesize the recent research on IA and information management? 6.1. Direction 1: pursue contingency frameworks rather than standardization Ariffin et al. (2014) and Griffiths (2012) have recently lamented that the lack of standardization in IA methodologies is a central reason for IA failing to be put into widespread practice. Buchanan and Gibb’s (2008) “methodological baseline” for IA practice contends that IA methodology selection should be based on organizational requirements and auditor experience, but stops short of building a contingency framework of context-specific methods and tools, which could supplement the baseline. This review’s finding that most recent IA research focuses on methods, signals that the proliferation of different methodological variations and permutations of the IA has become too spread out between different use cases and industry-specific needs to be effectively standardized. In light of that finding, it appears that the pursuit of cross-industry or industry standardization is a dead end. Future research should instead focus on designing contingency frameworks for IA practice, with different sets of methods and tools being prescribed for different industries, business scenarios, and information cultures. For example, aligning particular sets of methods and tools with the relationship-based, risk-taking, rulefollowing, and results-oriented information cultures typologized by Choo (2013) could prove more widely valuable and applicable than a one-size-fits-all approach to IA methodology. 6.2. Direction 2: explore the relationship between IA and quality dimensions in more detail Despite the close conceptual link between IA, information management quality, and information quality established in this paper, the relationship between those concepts is scarcely explored in the reviewed literature. Only Fraser-Arnott (2014) and Griffiths (2012) discuss the nexus of IA and information management quality, and only Griffiths discusses the nexus of IA and information quality. Future research, which seeks a deeper understanding of the causal relationships between IA and quality, will not only provide novel contributions to the research agenda, but would also validate the power of the IA if a causal link between IA practice and heightened quality could be empirically demonstrated. 6.3. Direction 3: apply more foundational IA methodologies in full to case studies There is a shortage of case studies applying foundational IA methodologies such as Burk and Horton (1998), Buchanan and Gibb (1998), Orna (1999), and Henczel (2001). In the reviewed literature, researchers will often propose their own modifications of foundational IA methodologies and apply the modified components to a case study—as exemplified in Ariffin et al. (2014), Mircea et al. (2012), Ocholla (2011), and Vo-Tran (2011)—but there is only one instance of a foundational IA methodology being applied to a case study in full, as exemplified in Jones et al. (2013). Moreover, there are few articles focused on the research outcomes of empirical studies of IA implementation compared to articles focused on IA methods, theories, and applications. Future research focusing on the outcomes of applying foundational and recent IA methods and theories could make a significant contribution to the IA literature by reporting on the strengths and weaknesses of IA methodologies in different use contexts, resulting in more empirical studies of IA outcomes. 6.4. Direction 4: develop theories of IA maturity and IA maturity modelling methods In the reviewed literature, there is no article which exhibits conceptual overlap between IA and maturity. Griffiths (2012) has suggested that the development of a maturity model represents a future direction in his research, and indeed, this review suggests that theories of IA maturity and IA maturity modelling methods are future research directions, which still remain unrealized. 6.5. Contribution to research methodology In addition to its implications for the content of future IA research, this literature review claims a unique methodological contribution, which stands to benefit all information management researchers. vom Brocke et al. (2009) provided the overall structure of this review’s methodology, Kowalczyk et al. (2013) provided a refinement of the literature search process, Webster and Watson (2002) provided the concept matrix template, and Cooper (1988) provided the scoping table template. However, to the knowledge of the author, the creation of a concept map and its direct integration into research questions, an extraction log, concept matrix, and results-driven conceptual analysis and synthesis has not previously been performed in the same manner as this review. Additionally, this review’s use of a detailed data flow model to illustrate the specific actions involved in the literature search process and make the process easily replicable from the diagram alone is unique in the information management literature. This methodology is ideal and easily replicable for potentially any literature review, which aims to sample only a representative portion of the literature and synthesize the findings, but strive for a high degree of rigor in doing so. Because this methodology is so easily replicable, it is an ideal candidate for the methodology of future IA reviews. An update to this literature review could be conducted years after the publication of this review using the exact same methodology, and an expansion of this literature review to analyze more data sources and concepts could easily be done by making minor adjustments to the concept map, extraction log, and concept matrix. 7. Implications for practitioners In a series of post-hoc discussions, the authors of the paper reflected further on the findings of our literature survey, and arrived at three recommendations for IA practitioners. Recommendation 1: Measurement and evaluation of information management quality and information quality are necessary parts of the IA. Define information management quality and information quality dimensions in advance of the audit, then measure and evaluate those quality dimensions as part of the audit. Mithas et al. (2011) implicitly offer insights into the quality dimensions of information management through their examination of information management capability factors; Floridi (2013) offers a readily 1388 R.B. Frost, C.W. Choo / International Journal of Information Management 37 (2017) 1380–1390 useable typology of information quality dimensions. Measurement and evaluation of the quality dimensions can then be incorporated into the phase of an IA methodology, which involves evaluation and qualitative judgment of the audit’s findings, such as Henczel’s (2001) “data evaluation” phase or Buchanan and Gibb’s (2008) “account” phase. Recommendation 2: In the reviewed literature, modelling and diagramming techniques are rarely recommended or situated within a larger IA process, but the use of these techniques is essential to the holistic mapping of information resources and flow. Observing Buchanan and Gibb’s (2007) separation of resource, process, and strategic perspectives when conducting the IA, there is a need for using different modelling and diagramming techniques when auditing within each of the three perspectives. Use entity-relationship diagrams and/or classification schemes to model information resources when working in the resource perspective; use data flow, UML, and/or BPMN diagrams to model information flows when working in the process perspective; use Allee’s (2000) value network diagram and/or the strategic view diagram template found in Buchanan and Gibb (2007, p. 169) to model alignment between information resources, flows, and strategies when working the strategic perspective. Recommendation 3: Document and publish more applications of IA methods so that researchers and practitioners have a greater knowledge base to draw from in studying the performance of different IA method and in innovating new IA methods and tools. 8. Limitations In advance of this review being conducted, the coverage of this literature review was predefined as representative rather than exhaustive, as outlined in Section 3. Accordingly, only two major academic databases (Scopus and ProQuest) were queried as part of the literature review. The body of literature on IA is quite small and highly specialized, so an exhaustive review encompassing more data sources will likely not produce any significant insights into RQ1 beyond what has been covered in this review. Nonetheless, this review’s coverage of only two data sources does constitute a limitation of the study. A more significant limitation arises from the query structures, which were used to address RQ2 (found in Appendix A). Those query structures were designed to limit the query results to articles, which discussed information management, information quality, and one other conceptual keyword, with the assumption that those articles would be of greatest relevance to the scope of this literature review. In the experimental development of the query structures; query structures seeking a match for only one conceptual keyword were tested. Unfortunately; each of those query structures resulted in tens of thousands of results being returned from both Scopus and ProQuest. If the number of data sources were expanded beyond two; the total number of query results pertaining to any one conceptual focus could easily extend into the hundreds of thousands. Although an exhaustive review of the literature on information management quality; information quality; and measurement/evaluation/maturity/benchmarking in the context of information management would surely reveal enormous potential for cross-pollination with IA theories and methods; such a review would likely not be possible without a data mining and analysis tool custom-built for the research project. 9. Conclusion This literature review posed and answered three research questions, which aimed to better discern recent developments in the IA literature, recent developments in quality, evaluation, measurement, and maturity in the information management literature, and how those two sets of developments could be synthesized with one another. A conceptualization of the topics was mapped out, and a systematic literature review methodology was described. The review was conducted and the results of the review were presented and discussed, with special attention paid to the research foci and conceptual foci of the reviewed literature. Implications of the review for researchers and practitioners were then outlined, and the limitations of the review were acknowl- edged. This literature review indicates that interest in IA research has remained highly specialized yet steady since 2011. Embedding recent developments in information management, information quality, measurement and evaluation, and maturity and benchmarking more deeply into IA theories, methods, and applications could propel interest in IA research up to higher levels. The practicality of IA methodologies could be improved by having more IA case studies on record, as well as through the design of IA contingency frameworks, IA maturity models, and information/information management quality measurement techniques. The IA is clearly an extremely valuable information management practice, but there is still much work to be done by both IA researchers and practitioners to make the practice more widely known and used. It is unfortunate that at a time when information management has become a necessity in nearly every industry and sector, IA research and practice have lagged behind other facets of information management. There now exists a great opportunity for an intellectual renewal of IA research and practice, and it is hoped that this literature review will serve as a guidepost for anyone seeking to push the IA agenda forward. Funding This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. Appendix A. Query Structures The query structure used to extract articles from Scopus in response to RQ1 was structured as follows: {information audit} The query structure used to extract articles from ProQuest in response to RQ1 was structured as follows: “information audit” NOT auditory The query structure used to extract articles from Scopus in response to RQ2 was structured as follows: {information management} AND {information quality} AND (“evaluat*” OR “measure*”) AND (“maturity” OR “benchmark*”) The query structure used to extract articles from ProQuest in response to RQ2 was structured as follows: (“information management” AND “information quality”) AND (“evaluat*” or “measure*”) AND (“maturity” OR “benchmark*”) R.B. Frost, C.W. Choo / International Journal of Information Management 37 (2017) 1380–1390 1389 Appendix B. Full Concept Matrix Concept Matrix Reference Research Focus Info. Audit IM Quality Info. Quality Measure- ment/Evaluation Maturity/Bench- marking Carvalho and Esteban-Navarro (2016) Outcomes, Methods x Ayyash (2015) Theories x Fraser-Arnott (2014) Theories, Methods x x Ariffin et al. (2014) Methods, Application x Shamel (2014) Theories, Methods x x Chuah (2014) Theories x x Gunawan and Suhardi (2014) Theories, Methods x x x Bastos, Moreira, Bruno, Filho, and Filho (2014) Theories, Methods, Application x x Jones, Mutch, and Valero-Silva (2013) Application x x Zhi-hong, Da-wei, and Hong-yun (2013) Methods x Visser et al. (2013) Theories, Methods, Application x x Zou et al. (2013) Theories, Application x x x Mohammed and Yusof (2013) Methods, Application x x Sheriff et al. (2012) Outcomes, Theories x Reifsnider (2012) Methods x Buchanan and McMenemy (2012) Theories, Methods x x Griffiths (2012) Theories, Methods x x x Chuah and Wong (2012) Outcomes x x x Petter et al. (2012) Theories x x Mircea et al. (2012) Methods, Application x x Ocholla (2011) Theories, Methods, Application x Vo-Tran (2011) Methods, Application x Pranicevi´c et al. (2011) Outcomes, Theories x x 12 3 10 13 5 References Allee, V. (2000). Reconfiguring the value network. Journal of Business Strategy, 21(4), 36–39. Ariffin, I., Latif, A. A., Faudzi, M. A., Shariff, S. S., & Nadzir, M. M. (2014). Information audit in electricity utilities: Roles, methodologies, issues and challenges. 2014 international conference on computer and information sciences (ICCOINS), 1–6. Ayyash, M. M. (2015). Identifying information quality dimensions that affect customers satisfaction of e-banking services. Journal of Theoretical and Applied Information Technology, 82(1), 122–130. Baskarada, S., & Koronios, A. (2014). A critical success factor framework for information quality management. Information Systems Management, 31, 276–295. Bastos, C. A. M., Moreira, M. R., Bruno, A. C. M., Filho, S. M., & Filho, J. R. F. (2014). Information flow modeling: A tool to support the integrated management of information and knowledge. KMIS 2014–International conference on knowledge management and information sharing, 76–86. Buchanan, S., & Gibb, F. (1998). The information audit: An integrated strategic approach. The International Journal of Information Management, 18(1), 29–47. Buchanan, S., & Gibb, F. (2007). The information audit: Role and scope. International Journal of Information Management, 27, 159–172. Buchanan, S., & Gibb, F. (2008). The information audit: Methodology selection. International Journal of Information Management, 28, 3–11. Buchanan, S., & McMenemy, D. (2012). Digital service analysis and design: The role of process modelling. International Journal of Information Management, 32, 251–256. Burk, C. F., & Horton, F. W. (1998). InfoMap: A complete guide to discovering corporate information resources. Englewood Cliffs, NJ: Prentice-Hall. Carvalho, A. V., & Esteban-Navarro, M. (2016). Intelligence audit: Planning and assessment of organizational intelligence systems. Journal of Librarianship and Information Science, 48(1), 47–59. Choo, C. W. (2013). Information culture and organizational effectiveness. International Journal of Information Management, 33, 775–779. Chuah, M., & Wong, K. (2012). A framework for accessing an enterprise business intelligence maturity model (EBI2M): Delphi study approach. African Journal of Business Management, 6(23), 6880–6889. Chuah, M. (2014). The impact of capability level on maturity of business intelligence initiatives. IJCSI International Journal of Computer Science Issues, 11(6–1), 144–149. Cooper, H. M. (1988). Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society, 1, 104–126. Floridi, L. (2013). Information quality. Philosophy and Technology, 26(1), 1–6. Fraser-Arnott, M. (2014). Moving from librarian to knowledge manager. Partnership: the Canadian Journal of Library and Information Practice and Research, 9(2), 1–10. Griffiths, P. (2012). Information audit: Towards common standards and methodology. Business Information Review, 29(1), 39–51. Gunawan, I. G. N. A. R., & Suhardi. (2014). Designing process maturity measure of total information quality management (TIQM). 2014 2nd international conference on business and information management (ICBIM), 47–52. Henczel, S. (2000). The information audit as a first step towards effective knowledge management: An opportunity for the special librarian. INSPEL, 34(3), 210–226. Henczel, S. (2001). The information audit: A practical guide. Munich: K.G. Saur. Jones, A., Mutch, A., & Valero-Silva, N. (2013). Exploring information flows at nottingham city homes. International Journal of Information Management, 33, 291–299. Kowalczyk, M., Buxmann, P., & Besier, J. (2013). Investigating business intelligence and analytics from a decision process perspective: A structured literature review. ECIS 2013 completed research [Paper 126]. Marchand, D. A., Kettinger, W. J., & Rollins, J. D. (2001). Information orientation: The link to business performance. New York: Oxford University Press. Mircea, M., Ghilic-Micu, B., & Stoica, M. (2012). Information audit for decision processes. Economic Computation and Economic Cybernetics Studies and Research, 3, 41–58. Mithas, S., Ramasubbu, N., & Sambamurthy, V. (2011). How information management capability influences firm performance. MIS Quarterly, 35(1), 237–256. Mohammed, S. A., & Yusof, M. M. (2013). Towards an evaluation framework for information quality management (IQM) practices for health information systems – Evaluation criteria for effective IQM practices. Journal of Evaluation in Clinical Practice, 19, 379–387. Ocholla, D. N. (2011). An overview of issues, challenges, and opportunities of scholarly publishing in information studies in Africa African Journal of Library. Archives and Information Science, 21(1), 1–16. Orna, E. (1999). Practical information policies (2nd ed.). Aldershot: Gower. Orna, E. (2004). Information strategy in practice. Aldershot: Gower. Orozco, J., Tarhini, A., Masa’deh, R. M. T., & Tarhini, T. (2015). A framework of IS/business alignment management practices to improve the design of IT governance architectures. International Journal of Business and Management, 10(4), 1–12. Petter, S., DeLone, W., & McLean, E. R. (2012). The past, present: And future of IS success. Journal of the Association for Information Systems, 13, 341–362. Pranicevi´c, D. G., Alfirevi´c, N., & ˇStemberger, M. I. (2011). Information system maturity and the hospitality enterprise performance. Economic and Business Review, 13(4), 227–249. Reifsnider, C., 2012. Instituting a business intelligence process. Online, May–June 2012, 28–32. Shamel, C. (2014). Planning for knowledge management: Conducting a knowledge assessment. In U. de Stricker (Ed.), Knowledge management practice in organizations 59–97. Hershey, PA: IGI Global. Sheriff, A., Bouchlaghem, D., El-Hamalawi, A., & Yeomans, S. (2012). Information management in UK-based architecture and engineering organizations: Drivers, constraining factors, and barriers. Journal of Management in Engineering, 28(2), 170–180. Visser, M., van Biljon, J., & Herselman, M. (2013). Evaluation of management information systems: A study at a further education and training college. SA Journal of Information Management, 15(1) [Art. #531]. 1390 R.B. Frost, C.W. Choo / International Journal of Information Management 37 (2017) 1380–1390 vom Brocke, J., Simons, A., Niehaves, B., Niehaves, B., Reimer, K., Plattfaut, R., et al. (2009). Reconstructing the giant: On the importance of rigour in documenting the literature search process. ECIS 2009 proceedings [Paper 161]. Vo-Tran, H. (2011). Adding action to the information audit. The Electronic Journal Information Systems Evaluation, 14(2), 167–282. Webster, J., & Watson, R. T. (2002). Analyzing the past to prepare for the future: Writing a literature review. MIS Quarterly, 26(2), xiii–xxiii. Yanjun, T. (2013). Study on computer audit based on OLAP technology. Journal of Digital Information Management, 11(6), 409–417. Zhi-hong, Z., Da-wei, X., & Hong-yun, L. (2013). Research on intellectual property information audit based on independent innovation. Advanced Materials Research, 860–863, 3100–3103. Zou, R. R., Flanagan, R., Jewell, C., & Tang, L. C. M. (2013). An exploratory study of information maturity in construction and developing a decision-making model. In S. D. Smith, & D. D. Ahiaga-Dagbui (Eds.). Proceedings of the 29th annual ARCOM conference, 69–80. Robert B. Frost is an information management researcher with a Master of Information degree from the University of Toronto. His research interests include methods of auditing information assets, quality dimensions of information management, and methods of leveraging information assets to improve service quality. His Masters degree culminated in a capstone project, which involved the design and pilottesting of a holistic audit framework for the cataloguing, modelling, and quality management of information assets. Chun Wei Choo is Professor of the Faculty of Information at the University of Toronto. His recent books include The Inquiring Organization (2016, Oxford University Press); The Knowing Organization (2nd ed., 2006, Oxford University Press); The Strategic Management of Intellectual Capital and Organizational Knowledge (2002, Oxford University Press); and Information Management for the Intelligent Organization (3rd ed., 2002, Information Today).