|The Magic, Myth, and Reality of CD-ROM Tax Research Tools|
Copyright 1995, Robert O. Graves, CPA
The tax publishing industry has revolutionized the field of tax research. Through the efforts of several major players in the industry, the CD-ROM medium has been transformed from a novelty for the technologically adventurous to an every day facet of life for many (if not most) tax practitioners. The industry has created a vast new information infrastructure by converting massive amounts of print based materials into digital form capable of dissemination either through on-line or CD-ROM distribution channels. There can be no doubt that CD-ROM is here to stay and will figure prominently into the working lives of all tax practitioners. The tax publishing industry deserves high marks for its efforts in bringing about this transformation.
Now that the practitioner community has worked with CD-ROM research products for a few years, it seems appropriate to take a reflective view of the medium to determine what these products can really do. It seems appropriate to consider the realistic capabilities and limitations of CD-ROM research products. It also seems appropriate to question whether the CD-ROM medium has any unique characteristics practitioners should take into account in using these products.
The tax publishers have invested vast amounts of energy and capital to develop and market CD-ROM tax research products. One can hardly fault the publishers for trumpeting the virtues of their products, but the keenness of the competition between the various vendors has resulted in a good deal of unintended misinformation about the products. In effect, the marketing efforts have endowed these products with almost magical capabilities. As a result, many practitioners have unrealistic expectations about CD-ROM tax research products. More importantly, few practitioners appreciate what is really involved in migrating from print based research to research in the CD-ROM medium. Most importantly, absent a proper understanding of some of the hidden pitfalls of CD-ROM tax research products, the unsuspecting user can easily make serious research errors.
The following article is an attempt to strip away the marketing hype and expose some of the myths surrounding these products. In the end, the aim is to understand the true magic as well as the reality of using CD-ROM tax research tools.
Magic: CD-ROM research products provide the tax practitioner with the possibility that relevant research materials can be located in a fraction of the time it would have taken to find those same items in a print based medium. This bit of magic drives practitioner fascination with CD-ROM research products and quite understandably has become the basis of the marketing efforts on behalf of the product vendors. The magic is genuine: CD-ROM research products really do provide that possibility.
Myth: With CD-ROM research products, you can conquer all your tax research problems effortlessly and in matter of minutes by simply entering a few key words into the CD-ROM research software.
Reality: In my experience, CD-ROM research techniques do produce almost magical results in as much as one-third of all situations. In a significant proportion of other cases, CD-ROM techniques can materially speed up the overall research process. On the whole, CD-ROM techniques represent unprecedented power in the hands of a skilled researcher. As an additional bonus, CD-ROM products eliminate the need for significant shelf space and the problem of filing loose-leaf updates.
However, even with all the pluses, CD-ROM products do not represent a "magic bullet" capable of effortlessly slaying every tax research problem. CD-ROM tax research is still a labor-intensive process requiring considerable skill, judgment, and patience. CD-ROM tools simply eliminate some practical problems (shelf space and updating) and provide the possibility that you can locate some items much more easily than in the print medium.
Myth: In order to take full advantage of the capabilities CD-ROM has to offer, it is simply necessary to purchase a CD-ROM tax research product and learn to use the search software.
Reality: CD-ROM tax research products consist of one or more discs containing the research information, search software which allows you to query the information contained on the disc(s), and various user manuals. The user literature focuses almost entirely on the issue of how to manipulate the search software. There is virtually no guidance on:
- the differences between research in the CD-ROM medium and the print medium and how these differences should affect your approach to tax research;
- special problems or risks inherent in the CD-ROM medium and how to avoid them; and
- special techniques necessary for success in the CD-ROM medium.
Reality: There is a body of knowledge unique to the CD-ROM medium that has nothing to do with the keystrokes for the search software of your particular CD-ROM tax research product. This body of knowledge is essential to safe and effective use of CD-ROM research products. Without an appreciation of the unique powers and limitations of the CD-ROM medium, you can make egregious research errors. It is alarmingly easy to inadvertently overlook critical research material absent an understanding of the unique nature of the CD-ROM medium.
Myth: It is acceptable to learn effective CD-ROM research techniques through on-the-job-training based on trial and error learning.
Reality: Professional tax advisers depend heavily on their ability to reliably locate relevant tax research materials in order to provide competent advice to their clients. If the state of knowledge about tax research in the CD-ROM medium is such that vast numbers of practitioners are using research tools they don't fully understand, costly and professionally embarrassing errors will necessarily result. After all, the technique isn't called trial and error learning for nothing.
Magic: It is not terribly difficult to assimilate the knowledge and skills necessary for success in the CD-ROM medium. It is merely necessary that the appropriate information be defined and widely disseminated.
Reality: The vendors of CD-ROM tax research products have been slow to recognize the need for additional information about their products. To date, no vendor has attempted to define the essential body of skills and techniques necessary for safe and effective use of their products.
Magic: The key word search is the principal tool responsible for the new capabilities available to CD-ROM tax researchers. The key word search has existed for many years in the world of on-line research services such as Lexis, Westlaw, and more recently CCH Online, but the high cost of these services has made extensive use of electronic research methods prohibitively expensive in many cases. The advent of CD-ROM products means that electronic research tools are available without the burden of on-line charges. For the first time ever, the key word search is a tool for finding answers to routine tax questions.
Myth: The transition from print based research to research in the CD-ROM medium is virtually painless.
Reality: With the possible exception of the very few practitioners who have used on-line services quite extensively, no one has any real experience with using the key word search as a research tool. Accordingly, migration from print based research to the CD-ROM medium means using, probably for the first time, a new research tool. Like any new tool, the technique requires some getting used to; and since the answers to all tax research questions do not simply appear as if by magic upon entering just a few well-chosen key words, the new CD-ROM user is in for a period of frustration and adjustment.
Reality: Viewing tax research results through the medium of the computer screen is a new experience for most practitioners. Having become so accustomed to dealing with information on the printed page, dealing with the same information on the computer screen can be frustrating, at least initially.
The computer screen is capable of displaying about one-third of a page of text. The print based researcher is able to simultaneously look at two whole pages. The greatly reduced field of view in the CD-ROM medium feels quite confining to most users and is initially very annoying.
Print based researchers make considerable use of peripheral vision. Oftentimes, you "stumble" onto relevant research materials simply because they are physically located in close proximity to something else you happen to be reading. The ability to see a considerable field of text is essential to the effectiveness of peripheral vision. Naturally, because the computer screen materially decreases the user's field of view, peripheral vision is all but eliminated in the CD-ROM medium thus considerably degrading research performance. Special techniques are needed in order to compensate for this problem.
Reality: Using print based materials and traditional research finding tools (index, table of contents, etc.) you always know where you are in the research material you are using. Having found the material you are viewing either through an index or table of contents, you automatically know what the context of the material should be.
For example, if you are researching the home office deduction and you access your research materials by looking up that phrase in the table of contents or index, you very much expect that the area in the research materials to which you were directed by the index or table of contents will discuss some aspect of the issue home office deduction.
In contrast, a key word search almost always retrieves materials containing the desired key words (called "hits") from disparate, far-flung parts of the research materials some of which will be totally irrelevant to the issue at hand. For example, a search using the key words home, office, and deduction would necessarily produce hits connected with the home office deduction. In addition, however, it is probable that other hits, totally unrelated to the home office deduction issue, will be retrieved as well; accordingly, each hit must be evaluated for relevance.
The difficulty is as follows: since any key word search hit may be from absolutely any part of the research material under consideration, the electronic researcher cannot assume that the material retrieved by the search will actually pertain to the topic at hand, and it may be difficult to determine the actual context of the hit especially when restrained to viewing only a few lines at a time.
Using an obviously oversimplified example to make this point, consider that a key word search containing the search terms "home, office, and deduction" would retrieve a document in which both the following two issues are discussed: a) the tax ramifications of selling a home; and b) a deduction for office expenses. All three key words are necessarily present in such a document, yet it has absolutely nothing to do with the home office deduction.
The issue becomes how long does it take and how difficult is it to determine whether or not a hit is really relevant? In practice, sometimes the determination is easy and sometimes not. Pure happenstance is usually the difference between the easy determinations and the difficult ones.
The key word searcher's problem is similar to that illustrated in the following example. Suppose you have a book containing extremely detailed street maps of major cities in the United States. Randomly open the book to any map page, and restrict your view of the page to only a small area, about one-third of a page. Now, what area are you viewing? If you are lucky, the portion of the map you happen to be viewing might contain some landmark you recognize. If not, it is necessary to scan the map, again with your very narrow field of view, until something recognizable appears. Whether you happen to land on something recognizable is a matter of pure luck and how much you happen to know about the mapped area.
This sense of disorientation caused by restricted field of view in conjunction with the existence of essentially random hits can be frustrating, especially at first, and special techniques are needed to combat this problem.
Myth: The key word search technique is a sophisticated, intelligent process.
Reality: In reality, a key word search is nothing more than a request to see a list of documents containing the desired key words. As discussed above, it is quite possible to retrieve documents that do indeed contain the desired key words but yet have absolutely nothing to do with the issue at hand.
In the print medium, you conduct research by visually scanning the reference material for possibly relevant items. Your mind, conditioned as it is by much tax knowledge and years of research experience, is capable of recognizing any relevant materials that fall under your eyes. In contrast, the computer is not so intelligent. The computer does not know relevant material when it "sees it". The key words you choose are simply an indirect, and inefficient means of communicating to the computer the concepts you wish to see.
The difficulty is that you as a researcher do not care about key words, per se. The office in the home example above is proof of that. You as a researcher want to see anything relevant to a given concept. Key words are nothing more than a proxy used to represent the concept you want to see. Unfortunately, the device is far from perfect. A key word search will almost invariably retrieve irrelevant documents. More importantly, however, a key word search (no matter how skillfully constructed) will frequently miss important, relevant materials. Again, special techniques are required to deal with this problem.
Here is an example of an actual tax research project gone wrong as a result of the problem set forth above.
The issue is whether a C corporation contractor with sales of less than $5 million can use the cash method of accounting. Obviously, Section 448 is inapplicable because of the low level of gross receipts. Exhaustive key word searches based on words such as contractor and builder in conjunction with words and phrases such as inventory, method of accounting, or clear reflection of income produced nothing of interest.
There is an extremely important case exactly on point. Consider the following annotation from CCH Standard Federal Tax Reporter at Paragraph 20,613.40
The IRS did not abuse its discretion in requiring a taxpayer in the roofing repair business to use the accrual method of accounting. Noting that the materials utilized by the taxpayer in the course of its business were held for sale and produced income, the court determined that the taxpayer was required to maintain inventories in order to accurately reflect income. [Emphasis added.]
J.P. Sheahan Associates, Inc., 63 TCM 2842, Dec. 48,174(M), TC Memo 1992-239.
(Reprinted with permission from Standard Federal Tax Reports published and copyrighted by CCH Incorporated, 2700 Lake Cook Rd., Riverwoods, IL 60015.)
The IRS contention in this case is that roofing materials are in effect inventory. Notice the way the taxpayer's business is described as a roofing repair business. Is this relevant to a contractor? Of course. Would you have been able to anticipate this type of problem in constructing your own key word search?
This problem resulted from a rather unexpected but otherwise quite ordinary description of what amounts to a roofing contractor as a roofing repair business.
Myth: A key word search can be constructed to find any document.
Reality: Under normal research conditions using key word searches, it is impossible to find between 5% to 30% of all documents in the given research materials.
The English language is very versatile: there are many ways to express a given concept. In order to be successful with a key word search, you have to guess how an author might express himself when writing about the concept you are researching.
If you guess correctly, you find
what you are looking for.
If not, you don't
Consider the following two sentences.
A) Bob and Mary drove to a restaurant to have dinner.
B) Bob and Mary took their car to Pizza Hut and ate their evening meal.
Both sentences express exactly the same concept using slightly different language.
Remember, when you perform a key word search, you have no way of knowing how the writer of the materials you are using would express the thoughts you need to see. What key words would locate both sentences A and B? This is precisely the problem you face each time you do a key word search.
Conduct an experiment using your own CD-ROM tax research product. Choose one or more current matter updates. Make sure the updates are old enough that the items discussed will have been incorporated into the main body of your research materials. Randomly choose ten items from the update(s), and read them for content. Now use key word searches to locate those items in your own CD-ROM tax research materials. Be careful that you don't inadvertently "fudge" the results. Use only key words you would have been able to choose without first having read the material.
Depending on your skill and the type of material you have, somewhere between 5% and 30% of the items will be very difficult or impossible to find using ordinary key word search techniques. A sample of ten items is far too small for a statistically valid result; accordingly, your results may vary from these figures. Also, the type of material for which you are searching makes a big difference. In general, the shorter the document you are seeking, the more difficult it is to find. For example, if the items you search for consist of two-to-three sentence annotations of cases and rulings, your results will tend toward the higher end of the range. On the other hand, if the items you pursue in this experiment consist of multi-paragraph or multi-page material such as full text cases or encyclopedia-style commentary, your results will tend to fall into the lower end of the range.
Never mind the statistics. The important point is not the precise frequency with which this phenomenon occurs, but rather the fact that there is a very high probability that your tax research materials contain some things you simply cannot retrieve with a key word search.
Myth: If a key word search fails to uncover relevant information, it means that your research materials do not contain information on the given subject.
Reality: Obviously, if it is impossible to retrieve some materials with a key word search, failure to produce relevant materials with a key word search means absolutely nothing. Very few CD-ROM users understand this, but failure to appreciate this unique aspect of electronic tax research products can doom the user to making serious research mistakes.
Magic: The developers of CD-ROM tax research products very wisely left the more traditional research finding tools (index, table of contents, code section classification, etc.) in place when CD-ROM research products were created. These more traditional tools can and should be used to supplement or substitute for key word search techniques.
Myth: The results of a well-planned, well-executed key word search must necessarily represent the answer to your tax question.
Reality: CD-ROM research techniques allow the user to quickly produce lists of possibly relevant materials. You must still evaluate all hits for:
- relevance--does the material really speak to my issue;
- completeness--are there other materials that also bear on my issue;
- timeliness--has this material been superseded by some subsequent material; and
- authority--what is the source of the material and its authority relative to other pronouncements on the subject.
Be careful to avoid thinking that the technological success of actually locating something relevant with a key word search is the same as reaching a supportable conclusion for your research question. A successful key word search is merely the beginning of the process of evaluation and careful consideration that is necessary for the professional tax adviser.
Conclusion: CD-ROM tax research products are powerful tools in the hands of skilled, well-informed users, and a source of potential professional liability problems in the hands of those who lack the proper understanding of the unique aspects of the new medium. Just like any powerful tool, CD-ROM tax research products must be understood in order to be used safely and to the greatest advantage. In some cases, special techniques are needed to deal with the problems faced by CD-ROM and other electronic tax researchers. Discussion of such techniques is beyond the scope of this article. In other cases, however, mere awareness of potential pitfalls is the most effective weapon the researcher has. Hopefully this article will serve as a catalyst for practitioner dialogue on these matters.
About the Author: Robert O. Graves, CPA is a professional tax researcher and the President of The Tax Resource Group in West Los Angeles, California. Mr. Graves is also the author of CD-ROM Tax Research, A Comprehensive Guide to Tax Research in the CD-ROM Medium, published by TRG Publishing. 800-578-3498.