The Process of Research Writing | Steven D. Krause | Spring 2007 | Home

 

Chapter Nine

The Categorization and Evaluation Exercise
 
•    Revisiting your Working Thesis
•    Why Categorize and Evaluate Evidence?
•    Dividing, Conquering, Categorizing:  A Few Rules to Follow
•    Some Sample Categories
•    Charting Your Categories
•    Assignment: Writing the Categorization and Evaluation Exercise    
*    Questions to consider as you write your first draft
    *    Review and Revision
*    A Student Example:  “Categorizing My Research on Drug Advertising” by Jeremy Stephens
 
Revisiting your Working Thesis Again
 
Before you start working on the categorization and evaluation exercises, you should revisit the progress of your working thesis.  In chapter eight, I began the discussion of the antithesis essay by encouraging you to first take a moment to take stock of the current version of your working thesis.  It’s important to embark on research projects with some sense of where you’re going, and the main advantage and goal of a working thesis is it establishes a direction for you to pursue your research.
 
As I’ve also said before, your working thesis will almost inevitably change a bit as you work your way through the process of research writing and the exercises in this book.  You begin in one place with some sense of direction about what you want to research, but when you start gathering and examining your evidence and as you work through the exercises, it’s important to be willing and able to change directions.  In other words, a working thesis is where you start your research project, but it isn’t necessarily where you end your research project.
 
 
 
Exercise 9.1
Either as a short writing exercise or with a group of your peers, consider once again the evolution of your working thesis.  Where did it start out and how has it changed to what it is now?  What sparked these changes in your working thesis and your point of view on your topic?  If your working thesis has not changed yet, why do you think this is the case?  If you did revisit your working thesis at the beginning of chapter 8, did the antithesis essay project (also in Chapter 8) make you reconsider your working thesis again?  Why or why not?
 
 
 Why Categorize and Evaluate Evidence?
 
We divide things into categories in order to make some sense of and interpret all sorts of different things.  Stores are arranged according to categories that tend to make sense of what’s in them for shoppers—for example, department stores divide their merchandise up into categories like women’s clothing, hardware, sporting goods, housewares, and so forth.
 
We also expect things to be categorized in a descriptive and sensible way.  Department stores tend to arrange things by what you might use them for and who might use them:  kitchen things are in one part of the store, sheets in another, women’s clothing in one part, and men’s clothing in still another part.  These categories aren’t the only way the department store owners could arrange things.  They could arrange things by color—all of the blue things in one part of the store (blue cookware, blue sheets, blue shirts, etc.), all of the white things in another part of the store, and so forth.  While that might make for a visually interesting store, it would be very difficult for customers to find anything in such an arrangement.
 
Categorizing your research will:
 
•    Help you (and eventually your readers) make better sense of what sort of evidence you have.  
 
•    Enable you to compare and contrast different pieces of evidence and to evaluate your research, which is an essential step in the process of research writing.  
 
•    Give you get a clearer sense of the evidence that you have and the evidence you are lacking.  
 
Dividing, Conquering, Categorizing: A Few Rules to Follow
 
While there are no formal rules for categorizing your research, there are a few guidelines that you need to consider as you begin to categorize your research for the purposes of writing about and evaluating it.
 
•    You have to have a significant body of research to categorize in the first place.  Hopefully, you have started compiling an annotated bibliography (see chapter six) and you have been working on adding to your annotated bibliography as you have progressed through the other exercises and projects in The Process of Research Writing by gathering materials from the library, the Internet, interviews, and so forth.  If you haven’t done these things yet, you probably aren’t ready for the categorization and evaluation essay exercise.
 
•    Each piece of research has to fit into a category.  No matter how you decide to categorize your research, be sure that all of it can be put into at least one category.  
 
As you try to meet this guideline, be careful to follow the next one as well:
 
•    As much as possible, each category should have at least two pieces of research.  Avoid having categories with just one item.  One item categories don’t allow you to make comparisons or generalizations about how things might be similar; they only demonstrate how things are different, which is only one of the functions of categorizing your research.  Also, if you allow yourself one item categories, it can often be a little too tempting to make too many one item categories.
 
If you get completely stuck with what categories to put some of your evidence in, you can create a “miscellaneous” category, though I would encourage you to avoid it if you can.  Having categories that are more specific than “miscellaneous” will help you in writing about these categories and what they mean for your research.
 
•    Categories should be as distinct and different from each other as possible.  If there is no difference between the items that you put in the category “from newspapers” and those from the category “from nonacademic sources,” then put all of the sources from both categories into only one category.
 
•    Last but not least, categories should make sense and tell you and potential readers about what your think of your evidence.  It probably wouldn’t make much sense and wouldn’t be very meaningful to have a category consisting of articles that appeared on page four of newspapers, or a category consisting of articles that were published in journals with titles that begin with the letter “R.”
 
Sometimes, categories that might seem to be illogical actually make sense once they are explained.  It might not seem to make much sense for a writer to categorize his evidence according to the gender of the authors.   But if the writer is trying to make a point about how men and women hold different attitudes about the topic of the research, it might make quite a bit of sense to have at least one category that examines the gender of the source.
 
Some Sample Categories
 
Beyond the few general rules I just described, categorizing things can be a very idiosyncratic and specific activity.  But to get you started in coming up with categories of your own, I’d like to suggest a few ways to categorize your research that should be applicable for most research projects:
 
Categories of the Author
•    “Academic” or scholarly writer
•    Non-expert writer (a magazine writer or writers with no stated credentials, for example)
•    “Non-writers” (that is, pieces of evidence where no author is named)
 
Categories of Source
•    Primary Sources
•    Secondary Sources
(See the discussion in chapter one on the differences between primary and secondary sources)
•    Academic journal or book
•    Non-academic or popular press magazine or book
•    Newspapers
•    Internet-based resources
•    Interviews (or other primary research you may have conducted)
 
Other Potentially Useful Categories
•    Date of publication—either a particular year, before or after a particular event, etc.  For example, if your working thesis was about gun control and teen violence, it might be significant to compare the research you have that was published before the 1999 Columbine High School shootings to the research that was published after the shootings.
•    Research that generally supports your working thesis
•    Research that generally supports antithetical arguments to your working thesis (see chapter eight)
 
Of course, not all of these sample categories will work equally well for all research projects, and it is possible that the categories you will find most useful for this exercise are ones that are very specific to your own research project.
 
 
Exercise 7.2
Which of the previous sample categories seem to be most potentially useful for your research project?  What other ideas do you have for other categories on your research?  Working alone or in small groups, consider as many categories for your evidence as possible.
 
 
Charting Your Categories
 
Once you have some ideas about what categories you think will be useful for dividing your evidence, you have to figure out how you want to do it.  I recommend you create a table or chart, either by taking advantage of the table function of your word processor, using a spreadsheet software, or just good old-fashioned paper and pen or pencil.  Write your categories across the top and some basic citation information-- author, title, publication, etc.-- about each piece of your evidence along the left side of the table.  In each “cell” of the table or chart created by this arrangement, indicate if the article falls into that category and make any other notation that you think will help explain how the article fits into that category.
 
The example below is part of a categorization chart that explores the topic of computer crime and computer hacking.  The writer’s current working thesis at this stage of the project was “While many hackers commit serious computer crimes and represent a serious Internet security problem, they can also help law enforcement officials to solve and prevent crime.” The left-hand column lists the title of the articles that the writer is categorizing, while the categories themselves are listed across the top row.
 
There are other possibilities for categories not included here of course, and I would encourage you to come up with as many categories as you can for this step in the process of writing a categorization essay.  There are ten different pieces of evidence being categorized here.  You could do more or less, though again, though for this exercise to be effective, you should chart at least five or six pieces of evidence.  
 
As you can also see here, most of the entries include at least a few extra notes to explain why they are in different categories.  That’s okay, and these notes might be helpful to the writer later on when he puts together his categorization and evaluation essay.
 
A Categorization Chart Example
 
Evidence:
Web-based Sources
Academic/Trade Sources
Gov. Doc Sources
Popular Sources
Hackers always bad
Hackers sometimes good
Enforce-ment/ fighting crime
Brenner, Susan
cybercrimes.net, 01
XX
XX
(Law school)
   
XX (Legal issues/laws against)
 
XX (courts, laws, etc.)
Cameron, Al
“Fighting Internet Freud”
Business Credit, 02
 
XX
(Trade Pub)
   
XX (Money &
business)
 
XX (cops, company software)
“Cybercrime.gov”
US. Gov., 02
XX
 
XX
(Dept. of Justice)
 
XX
(terrorism, fraud)
 
XX (FBI, etc.)
“Cybercrime soars”
Info Management Jrnl, 02
 
 
XX
(Trade pub)
   
XX
   
Markoff, John.
“New Center...”
NYT, 10/99
 
     
XX
XX (business)
 
XX (private business)
Neighly, Patrick
“Meet the hackers”
America’s Network, 00
 
 
XX (??)
     
XX (“hanging out” with hackers)
 
Palmer, CC.
“Ethical Hacking”
IBM Sys. J, 01
 
XX
(Trade pub)
     
XX (can help with business)
XX (hackers fighting crime)
Sauer, Geoffrey
“Hackers, Order, Control”
Bad Subjects 2/96
 
XX
(Culture studies)
     
XX (the “culture” of hacking)
 
Speer, David
“Redefining Borders:”
C, L & S C, 00
 
 
XX
(Crimin-ology)
   
XX (business but individuals, too)
 
XX
(abstract ideas)
 
“World Cybercrime...”
CNN, 10/02
XX
(CNN web site)
   
XX
XX (business, terrorism)
 
XX (inter-national effort)
 
Presumably, you are not familiar with the specifics about these pieces of evidence; but for the purposes of this example, it’s more important that you understand the categories and the process the writer must have gone through to come up with this chart.  The number of observations that can be made from a chart like this could be explored in more detail in a categorization and evaluation essay.  You’ll use your own chart to complete such an essay later in this chapter.
 
•    While the reasons for the articles for being put into the category “Hackers always bad” are similar (fear of damage to business and the potential for terrorism), the reasons why the articles were put into the category “Hackers sometimes good” vary.  The Palmer essay suggests that hackers might be beneficial (when they work “ethically,” as the title says) in order to help protect business from the attacks of “bad” hackers.  While both the Neighly and Sauer articles make distinctions between “good” and “bad” hackers, these essays are more focused on hackers as people than as criminals.
 
All of this suggests that if the writer wanted to continue exploring this idea of “hacking,” it might be wise for the researcher to carefully consider how hacking is discussed.  For example, how does each article define “hacking?”  How does each article assess the potential threat or potential benefit of computer hacking?
 
•    With the possible exception of the Neighly essay, the three essays that describe computer hacking as something that is sometimes good are from academic or “trade” publications.  The writer put question marks in his chart in the “Academic/Trade Sources” category next to the Neighly essay because it was a difficult to categorize source that seemed to fit best here. .  Interestingly enough, one of the “hackers sometimes good” publication was produced by the computer company IBM.  The professional and trade publications that suggest computer hacking is always bad focus on the issues of the law, law enforcement, or criminology.  
 
•    Almost all of the evidence included here is concerned with enforcing the laws and fighting against cybercrime, but there seems to be little consensus as to how to do it.  Some of the resources are advocating for tougher U.S. federal laws; one is advocating international action; and some are suggesting that enforcement must come mainly from the Internet business community.
 
•    There is only one government publication listed on this categorization chart, which suggests that either the U.S. government has not published many documents on computer crime and hacking, or the researcher ought to consider conducting some more research that focuses on government documents.
 
The same can be said in some ways about Web-based resources:  all of the Web-based research portrays computer hacking as an unlawful and criminal act.  Considering the fact that the World Wide Web is a space with many divergent views (especially about topics like computer crime and computer hacking), it seems logical that there may be worthwhile to see what other evidence is available on the Web.
 
This process of charting your categories is one that can go much further than suggested here.  For example, perhaps your initial categories have prompted you to consider new ways to categorize your evidence, which might lead to additional relationships between your sources.  You might also include more evidence, which again might lead to different observations about your evidence.  
 
Ultimately, you have to write about the results of your categorization in the form of an essay.  I will describe this in more detail in the next section of this chapter, but you might want to consider two strategies as you move from the “charting” phase of this exercise to the “drafting” phase:
 
•    You will have to explain the significance of your different categories and groupings of evidence in your essay for this exercise, perhaps more than you might think.  As the writer, the division of the evidence might make perfect sense to you, but that “sense” often is not as accessible to your readers.  This potential of missing your audience is possible with any writing project, but it is something to be especially mindful about with this exercise.
 
•    Charting of evidence will probably yield many different and interesting points of comparison and evaluation, but you should focus on the points of comparison you think are the most significant.  In other words, you probably shouldn’t talk about each and every category you chart.
 
 
Exercise 9.3
Try creating a categorization chart of your own.  Working alone or in small collaborative groups, group your sources according to categories that make sense to you, perhaps the ones you developed in the previous exercise.  On a piece of paper or on a computer using a spreadsheet or table-making software, create a chart that looks similar to the one in this section.  Do you notice similarities or differences between your evidence you didn’t notice before?  Are there any short-comings or other imbalances between your categories that might help you better target what you need in any additional  research?  What other sorts of observations can you make about your research?
 
 
Assignment:  Writing the Categorization and Evaluation Essay
 
Write an essay that categorizes the evidence you have up to this point in order to assess the strengths and weakness of various types of evidence, to draw some conclusions about your evidence and topic, and to take inventory of your research.  Be sure to explain the categories you establish for comparing and contrasting your evidence and to make some sort of conclusion based on your criteria.
 
In this writing exercise you need to be especially careful about understanding your audience.  If your main audience for this project is a group of readers who are already familiar with the evidence you will be comparing (because they are classmates that you’ve been collaborating with all semester, for example) and the purposes of your comparison, then you may not have to provide much summary of the research you are categorizing and evaluating.  
 
On the other hand, if your main audience for this project is not already familiar with your research or the process you’ve gone through to categorize your evidence, you might have to provide both a detailed explanation of the process you went through to categorize your evidence and a summary of the evidence you are categorizing.  When in doubt, you should assume that your readers are not familiar with the process of categorization or the evidence being categorized and evaluated.
 
Another important part of this writing exercise is focusing in on just a few categories in order to make an overall evaluation of the evidence.  Remember:  the goal of categorizing your evidence the way you have here is to make evaluations of your evidence that are interesting to you and potential readers.  In the example discussed in the previous section of this chapter, there are five different “observations” or points that could be the focus of evaluation.  While some of these observations could be combined for the purposes of an essay for this project, it would be very difficult for the writer to talk about all of these points and still have a focused and clear essay.
 
Questions to consider as you write your first draft
•    Have you revisited your working thesis yet again?  Based on the research and the writing that you have done, has it changed since the beginning of your project?  Has it changed since chapter four?  How?
 
•    Have you gathered enough research to effectively categorize and evaluate it, at least five or six different pieces of evidence (and ideally more)?
 
•    What sorts of categories are you using to “divide and conquer” your evidence?  Which of your categories seem unique to your research project?  Have you considered some of the categories suggested in the “Some Sample Categories” section of this chapter?
 
•    Have you followed the guidelines discussed in the “Dividing, Conquering, and Categorizing: A Few Rules to Follow” section of this chapter?  Can you fit all of your research into at least one of your categories?  Have you avoided single item categories or “miscellaneous” categories?  Is there a clear difference between your categories?  Do your categories help you and your potential readers make sense of the evidence you are comparing?
 
•    Did you chart your categories using a word processor’s table function, a spread sheet, or paper and pen/pencil as suggested in the “Charting Your Categories” section?  Would additional evidence or categories make your comparisons more useful?  If you didn’t create a chart similar to the example in this chapter, how did you decide to categorize your research in order to evaluate it?
 
•    What observations did you make about your categorization chart?  Were there relationships, comparisons, contrasts, or other connections between evidence and categories that you were expecting?  Were there ones you weren’t?  Did your categorization chart give you a better sense of the kinds of evidence you have?  Did you get a sense of the kinds of evidence that you don’t have and perhaps need to research further?
 
•    What sort of evaluations can you make about your evidence based on these categorizations?  Do you notice any patterns within categories or between different categories?  Did you find yourself making evaluative statements similar to the examples at the end of the “Charting Your Categories” section of this chapter?
 
•    What do you think your audience will see as the one or two most important points of evaluation that you’ve learned from categorizing your evidence?
 
Revision and Review
If you made a chart to categorize your evidence as you wrote a draft of your essay, you might want to share that with your peers in the revision process.  They might see something about the relationship between your pieces of evidence that you haven’t noted in your essay.
 
Here are some questions you and your classmates want to consider as you revise your critique essays (of course, you and your teachers might have other ideas and questions to ask in review too!):
•    Is the writer’s evaluation and comparison of the research clear to readers?  Do readers understand the point the writer is trying to make with this categorization and evaluation essay project?  What would make this evaluation clearer?
 
•    Is the writer providing sufficient summary and explanation of the research being categorized and evaluated for this group of readers?  What additional information might some readers need to understand the writer’s point?  Is there too much summary for the writer’s intended audience?
 
•    Does the writer explain the categorization process they went through in evaluating their research?  Do the categories make sense in understanding the research?  As a reader, do you have any other suggestions for ways the writer could categorize their research?
 
A Student Example:  
“Categorizing My Research on Drug Advertising” by Jeremy Stephens
 
For this assignment, Jeremy was required to write an essay similar to the assignment outlined above, to categorize his research and to draw some conclusions about his evidence based on these categories.  “This was a hard assignment, and I’m not sure if I did it right,” Jeremy wrote in a memo that introduced this project.  “It did help me to see more clearly what evidence I had and what I needed.”
 
Categorizing My Research on Drug Advertising
 
    When I started to take a closer look at the different sorts of evidence I had gathered for my research project on the problems of drug advertising on television, I noticed several different trends.  To get a better understanding of the evidence, I began by categorizing all of my evidence by the type of media-- books, web sites, articles from academic and professional sources, and articles from more popular sources.  From there, I divided the evidence into two additional categories: those that supported my working thesis on limiting drug advertisements and those that did not support my working thesis.  
One of the things I noticed is that I had not realized how much evidence I had from trade and professional sources, things that weren’t really academic but that weren’t from popular sources either.  I’ve decided to focus on these sources and some web site sources too because they have made me think more carefully about my topic.
    My working thesis is that drug commercials on television ought to be severely limited because they are misleading and make false or exaggerated claims about the benefits of the drugs.  Some of the articles in professional and trade publications disagreed with this thesis.  For example, Carol Rados wrote an article called  “RX Ads Come of Age,” published in FDA Consumer, which is a publication of the Food and Drug Administration. Rados wrote “There seems to be little doubt that DTC advertising can help advance the public health by encouraging more people to talk with health care professionals about health problems, particularly undertreated conditions such as high blood pressure and high cholesterol” (22).  While Rados does note that there has been a lot of criticism of drug ads on TV, she makes it clear that the benefits actually outweigh the harms of these ads.
    However, many of the professional sources agreed with my thesis.  For example, Emma Dorrey’s brief article in Chemical and Industry titled “FDA sends 23 warning letters to drug companies” supported my thesis because it points out that there have been a number of problems with the ads.  Dorrey reports that the drug industry claims to work hard at self-regulating and that the companies say the ads educate consumers.  However, despite the laws and the efforts of the FDA, there are still a lot of misleading ads:
One of the problems, according to Barbara Mintzes of the Center for Health Services and Policy Research at the University of British Columbia in Canada, is that the FDA can only regulate after the fact. And  “companies do not face any sanctions other than needing to withdraw the ad if the information is inaccurate or misleading”(6).
    I also noted that I had two articles from trade publications that focused on media, publishing, and advertising, both of which supported my working thesis.  The first came from the publication Broadcasting and Cable, which I accessed via the WilsonSelect database.  In the article titled “Relaxed Rules on Drug Ads Find Allies,” Bill McConnell reports on a move by the FDA to relax the rules for drug companies to list the side effects of their medications, a move that would help the drug companies.
    The second was an editorial by Allan Wolper in Editor and Publisher titled “Accepting Drug Ads a Risky Proposition.”  Wolper tells the story of a controversial cholesterol medication that was being simultaneously criticized and advertised in The New York Times in November 2004. As Wolper points out, “pharmaceutical ads present an ethical problem for newspaper sales acceptability departments, which love the revenue the ads bring in but worry that the claims associated with them will hurt the credibility of their news organization” (22).  Both of these articles were published in trade journals for the media, which benefits by the money drug companies pay them to advertise their products.  However, both of these articles express how these ads can ultimately hurt their credibility, too.
    Almost all of the web sites I came across supported my working thesis too.  I looked at a lot of different sites, but I rejected any site that did not name the author or who had an author that wasn’t familiar to me because I just wasn’t sure if they were credible.  I also rejected web sites created by drug companies because of the obvious bias of these sites.
    Instead, I focused on web sites maintained by news organizations or other organizations I had heard of and that seemed credible.  For example, I came across an article on the Consumer Reports web site called “Free rein for drug ads?”  The article, published in February 2003, says that there has been a decrease in the number of drug ads being reviewed by the FDA, and this drop-off of the number of letters sent from the FDA to drug companies about their ads “has raised concerns among some legislators and policy researchers because it leaves potentially false or misleading drug information in the public eye for longer periods.”
    I also read a transcript of an internet chat with Dr. Jeffery Kahn, who was CNN.com’s bioethics columnist.  Kahn chatted over the internet with all kinds of different people about drug advertising.  Kahn said that he thought drug companies were “overzealous in how they market, leading to misunderstanding and confusion for patients.” Judging from the rest of the transcript, it appears that most of the participants agreed with Kahn.  One of the things that I thought was interesting about this piece of evidence was how the source made it more credible.  If it had just been a chat session somewhere out on the internet, it wouldn’t have been as good of a source.
    Categorizing my evidence was a helpful exercise for me.  I knew that I had evidence from a variety of different kinds of sources, but by focusing on trade publications and credible internet sources, I feel like I am in a good place to start my research project.  Looking again at these professional publications and web sites has made me think about my working thesis more carefully.
 
Works Cited:
“Chat Transcript: Jeff Kahn on Drug Marketing Ethics.”  CNN.com 16 August 1999. 20 June 2005 <http://www.cnn.com>.
Dorrey, Emma.  “FDA Sends 23 Warning Letters About Drug Ads.”  Chemistry and Industry 7 February 2006: 6.
“Free Rein for Drug Ads?”  ConsumerReports.org February 2003.  15 June 2005 <http://www.consumerreports.org>
McConnell, Bill.  “Relaxed Rules on Drug Ads Find Allies.”  Broadcasting and Cable 19 January 2004:34.  Electronic.  WilsonSelect.  Eastern Michigan University Halle Lib. 15 June 2005.
Rados, Carol. “Truth in Advertising:  Rx As Come of Age.”  FDA Consumer July-August 2004: 20-27.
Wolper, Allen.  “Accepting Drug Ads a Risky Proposition.”  Editor and Publisher January 2005:  22.