Harris Research Partners: Developing Online Research Methods & Tools

Developing Online Market Research Methods and Tools

by: Cheryl Harris, Ph.D.

Paper presented at: ESOMAR "Worldwide Internet Seminar," Lisbon, July, 1997; and published in Marketing and Research Today, Vol. 25, #4, November 1997, p. 267.

Abstract

There have been a large number of conflicting working models and assumptions proposed for interactive media, suggesting that there are problems in the way we conceptualize "interactivity" in general. Work in interactive media, including research, design and strategic planning, faces disruption from a lack of appropriate theoretical development. Such theory-building can be facilitated by taking a close look at lessons learned from real-life observations of media users. This study will summarize recent consumer research in interactive media, including the practical and theoretical lessons from which we can learn in developing research methods and tools relevant to new media, such as the WorldWide Web (WWW.) Based on accumulated studies in online consumer research over an 18-month period, the author will examine various methodological solutions to the challenge of the online environment. The paper will also suggest how the research community as a whole may cooperate in setting an agenda for an ethics of online research practice. Theorizing Interactivity: Models and Cases to Consider in Developing Online Market Research Methods and Tools There have been a large number of conflicting working models and assumptions proposed for interactive media, suggesting that there are problems in the way we conceptualize "interactivity" in general. Work in interactive media, including research, design and strategic planning, faces disruption from a lack of appropriate theoretical development. Such theory-building can be facilitated by taking a close look at lessons learned from real-life observations of media users. This study will summarize recent consumer research in interactive media, including the practical and theoretical lessons from which we can learn in developing research methods and tools relevant to new media, such as the WorldWide Web (WWW.)

DISCUSSION

In October of 1996 The Council of American Survey Research Organizations (CASRO) released the results of a survey in which they had asked a wide range of executives and research professionals at Fortune 2000 companies across North America a series of questions about their assessment of online research, particularly survey research, for application in their organizations. Although many had concerns about current methodological practices in online research, the majority, an astonishing 64%, say that they expect to conduct or commission online survey research in the near future. About the same time, Forrester Research reported their startling projections for the future of corporate investment in the WWW, anticipating that US$10 billion will be spent on WWW development services within the next 2-3 years, and placing the current average cost for developing a website at US$267,000 (Forrester, September 1996.) These statistics suggest that there will indeed be a bright future for online research, as the intense and escalating investment in the WWW force companies to institute programs of continuing evaluation and accountability around their online activities. However, there is evidence that there is already a crisis in new media, with conflicting pronouncements about what does and does not work online, and daily complaints in the trade press of disappointments with website performance. The fabulously expensive, but abysmally unprofitable WWW site is more common than not. And no wonder, with online gurus such as Doubleclick contributing to an avalanche of descriptive data of various types, such as: "Simple animation can increase response rates by 25%...", "Using questions in a banner increases clickthrough 16%," and "Statements creating a psychological sense of urgency actually seems to decrease average response rates." Webmasters and designers blindly apply these pointers, while the crisis in content and performance grows ever deeper. While gathering such observations is helpful, it does not tell us anything to help understand the motivations behind these behaviors. It is relatively easy to track behavior in virtual environments, but much more difficult to thoughtfully understand this new environment and the motives and concerns of the people who engage in it. To do so will require developing a research agenda that will include careful theoretical and methodological development attuned to the critical differences in interactive environments. When interactive media design and strategic planning may benefit from the kind of stringent and rigorous audience testing that traditional advertising in other media routinely employs, then we as an industry may have a shot at creating online environments that meet our objectives for them. However, right now we are possibly the only discipline actively engaged in "practice without theory" underscoring the fact that despite the 3-5 year presence of the Internet as a public relations, marketing and communications tool, we have not progressed very far in adapting our methods to it.

The Harvard Review of Business in December of 1996 made the following statement: "A profession, no less than a craft, is shaped by its tools. The professions of advertising and marketing, our theories, practices and even the basic sciences that it draws on are determined by the tools at its disposal at any moment." The applicability of this notion to new media is rather striking. Naturally when the tools change, the discipline must adjust, "sometimes quite profoundly and usually quite belatedly." The introduction of television and television advertising some 50 years ago in the U.S. was just such a disruptive event, and advertising theory and practice is still responding. One just has to take a cursory look at the huge body of literature surrounding the identification and measurement of television effects to see many examples of conflicting models and paradigms for what seems to be a simple question (but in fact is wildly complex) - how does watching television affect the individual? So, it is not too surprising that the development of appropriate theories and models in interactive media should be in an elementary state. Nevertheless, can we afford the incredibly high investments in entering the realm of new media, either as marketers, content developers, or technologists, which we are either contemplating or already making, without an improved, more clarified set of assumptions?

Beyond the investment of capital in online media, there are other imperatives driving the need to engage in research and theory-building, such as the recent disclosure at a New York City workshop sponsored by the Advertising Research Foundation (ARF) in which a marketing director of a Fortune 100 firm claimed that her firm had definitely seen a marked erosion in their brand equity after developing an online presence. The reason seemed to be related to poor WWW design and content at their site which undermined millions of dollars and years of attention to developing brand equity and imagery. Failure to understand the interactive environment and the online population segment being addressed apparently can damage the considerable branding investments made over time and across numerous other media - reason enough to think carefully about what one is doing online. Finally, it is not difficult to see that incorrect market and audience analysis has contributed to market failures in the past. We might say that we in interactive media have been lucky so far in that the phenomenal growth, journalistic interest, and even the "goldrush mentality" attached to all things Internet has fueled the attraction of capital and talent to the field. To succeed for the long term, we need to move forward to next steps: We need a real commitment to high quality (as opposed to ad-hoc) research that takes the global online audience into account and seriously addresses the problems with current online research efforts, including the development of a framework of theories and models that are adaptive to a new media environment. We must learn to integrate what is useful from existing models of audience and advertising theory, but also be willing to reject those that are not a good fit with the unique demands of an evolving interactive environment. We must take the time to gather knowledge, build informative and effective models, and then test them well. Every discipline must do this to rise above a merely reactive stance. We urgently need an accepted ethics of online research and online marketing behavior. Without this, unscrupulous marketing tactics and resultant consumer fear will make performing online research impractical.

THEORETICAL PROBLEMS IN ONLINE RESEARCH

A starting point for the development of a research agenda for a new discipline is to examine the terms that are in common use. Although I have used the term "interactive" freely throughout this paper and at least half a dozen times on the preceding page, I find myself more and more troubled about what it really means. Recently, I completed a survey of new media professionals (such as webmasters, web art directors, content writers, and consultants) and asked them to define the term "interactivity." Concurrently, I sampled two separate online panels for a total sample size of n=12,000 online users with the same query. The results were quite remarkable, and emblematic of the variety of problems just mentioned. Strangely, the new media professionals were frequently confused about the notion of interactivity, and lacked a strong consensus about how they deployed the concept in their work. For example, one prominent webmaster wrote that "I construe 'interactive' as giving people what they want, when they want it. However, we still don't know what [giving them what they want] really means..." Another said "we thought that interactivity means providing a lot of levels, buttons to push, and so on, but our logfiles show that these features aren't getting used." Yet another said "when you find out, let me know." In contrast, online users were intensely interested in this question and had high expectations for their online experiences (which many noted bitterly were not being met.) Online audience members used words such as "involvement," "influence", "participation", "control," "real communication", "responsive", "customization," "choice," "Real time" and "active exchange." The lack of convergence among these two groups - developers and audiences - concerning the primary feature of online media suggests that we have no working definition as a field of its key concept. It is also clear that without an ongoing feedback loop between these two parties strategic development of web content and design will continue to experience failure. The first task, then, should probably be to return to the basic concepts which impact the field - interactivity being critical among them - and begin to theorize them. This requires the introduction of a critical discourse around current and future practice that will be necessarily interdisciplinary, involving the best thinking from the areas of Computer-Mediated Communication (CMC), human and interpersonal communication, Human-Computer Interaction (HCI), virtual reality specialists, interface designers, cultural theorists, artists, writers, advertising and marketing professionals and others in the academy and industry with an interest in the evolution of interactive media and virtual environments.

METHODOLOGICAL ISSUES IN ONLINE RESEARCH

Earlier I mentioned that integration of the models of traditional research in new media is also important, but perhaps more critical is to presence the rigor of traditional research methods in online research models. Sampling and Online Panels The credibility of online research has suffered from a number of serious problems that are theoretical as well as methodological. Probably the most significant problem is the lack of a master database for the Internet population. In basic terms, there is no reasonably accurate sampling frame, nor is there likely to be one anytime soon. This is true for two reasons: the rapid growth of the Internet population as well as its international scope makes the development and maintenance of such a list a Herculean task, even if it were not for the second reason, which is the notorious intolerance online users have for anything regarded as unsolicited communications. Invitations to participate in research, even those originating from well-known and respected firms, are still considered "spamming" and often have very negative consequences. These consequences have included the withdrawal or threatened withdrawal of the researcher's ISP (Internet service provider) access, effectively putting the researcher out of business, or other problems such as the offended party or parties bombing the originating server with mail, virus-infected files, or fax communications. So, online research cannot currently draw a probability sample, and at the same time cannot easily conduct survey or other research work utilizing unsolicited email as a recruiting or surveying tool. Without the ability to draw a probability sample, obviously the validity of online survey research is suspect. This alone has led to the rapid and somewhat panicked development of online panels, some quite massive, by small and large research firms alike. Large syndicated research firms such as Simmons Market Research Bureau (SMRB) and National Family Opinion (NFO) have recently identified the subset of their standing panel members who say they have online access and are marketing these members as "online panels." Other firms, like my own, are constructing panels based on an assumed profile of the online Internet population. Both approaches to online panel construction have their pitfalls, however. First, quite apart from the lack of a master database, there are currently no accepted statistics for the characteristics of the online population. Within the past year a variety of studies have been released which have both conflicted with each other and which have been attacked on methodological grounds, the Nielsen Media Research study of 1996 which was debunked by a team of Vanderbilt University researchers being a primary example. Naturally, all attempts to profile and segment the online audience suffers from the same sampling constraints described above. Therefore even a triangulated profile, drawing from the areas of convergence among the key studies, as a guide to the construction of an online panel will have sampling error attached to it which currently cannot be estimated. Panels which are subsets of traditional panels also have problems. Traditional panel companies are rightfully concerned about panel maturation effects as well as self-selection bias in their general panels. As it is reasonable to believe that highly mobile members of online panels must be refreshed more often and that the known demographics of online users (younger, more educated, more affluent) do not map well onto the demography of traditional research panelists, subset panels may not be a very good representation of the online population either. Moreover, online panel member identity must be validated at least as carefully - and probably even moreso - than traditional panelists. The well-known joke "....on the Internet nobody knows you're a dog" has an element of truth to it. It is critical to rigorously validate online panelist identity, possibly each time there is a communication or research task to be performed. It may be necessary to do so in a variety of ways, making online panel maintenance a labor-intensive task. It is also important to realize that online panels must be rather large to be effective, both for purposes of segmentation and to achieve adequate response rates. Online Surveys Although most completed online surveys returns occur in just 48-72 hours, making turnaround incredibly fast compared to other methods, early research suggests that response rates for online surveys may be dramatically lower than for any method other than mail. Online surveys are also hampered by constraints on length and complexity. Response rates for un-incentivized questionnaires drop off precipitously after about 10-15 questions. In fact, response rate seems to be directly and negatively correlated with questionnaire length. The advantages of attention-grabbing mailing materials, paper selection, and other methods designed to increase the response rates of mail surveys, for example, do not extend to e-mail delivered surveys. Web-based surveys have the distinction of being able to introduce multimedia elements, but further complicate self-selection biases, as the respondent must "go to" the website where the survey resides. It is also worth noting that the elaborate information-condensing grids of CATI phone and mail surveys do not translate well to either an e-mail or web-based delivery environment, also decreasing the amount of information which may be obtained in online surveys.

SUMMARY OF ADVANTAGES AND DISADVANTAGES IN ONLINE SURVEY RESEARCH Advantages · Large samples are possible in a short amount of time · Can in most cases be performed more cheaply than surveys using other methods (although low-incidence samples remain a problem) · Data may be analyzed continuously; can port directly into statistical tools and databases · Anonymity effect can be helpful for some topics; some evidence that interview bias is reduced or eliminated in online surveys Disadvantages · Similar problems as with all self-administered surveys · Length directly correlates with response rates · Difficulties in incentivizing (response rates cannot be estimated well in advance, therefore incentive costs are unknown; also influence of incentives on performance are unknown.) · Identity validation a problem · Panels seem to be a necessity but little is known about realities of developing/maintaining online panels · Probability sampling is not yet (and may never be) achievable; all samples have high self-selection bias · Error cannot be calculated, therefore data quality is suspect.

METHODOLOGICAL ISSUES IN ONLINE QUALITATIVE RESEARCH

For the same reasons that researchers have always chosen qualitative approaches, such as the need to conduct exploratory research, or obtain detailed, unstructured feedback, the online research toolbox needs a set of qualitative methodological principles. The most pressing practical need is for the ongoing testing of web design concepts and features with the target audience which attempts to understand the unique nature of the online experience and audience expectations of it. This is currently one of the most active areas of our research practice and one which we expect to grow, certainly if the projections of web development costs and investment mentioned earlier in this paper prove to be accurate. The range of topics which may be explored online with qualitative methods is diverse, and should prove to be an important arena of theory development as we seek to understand more about the motivations and behaviors of online audiences. Still, compared to online survey research there has been far less attention paid to the progress of interactive qualitative research methodologies. Much of what has been done may be described as falling into two categories: 1) interactive media research using "offline" methods, such as bringing together a group of people into a computer laboratory and observing their behavior with a post-test "debrief" interview (either collective or individual.) This is common for web designers, CD-ROM developers, and web advertisers. This approach lacks the synergism of being executed in the same environment as the test stimulus (i.e., online or in front of a private, home or office-based workstation) and also fails to appreciate the many problems historically associated with experimental lab research. Geographically-restricted tests will not replicate the global marketplace for interactive products and online services. For some groups, such as children, response patterns are most likely seriously affected by the presence of peers and the unfamiliar setting;. 2) It has become popular to attach "chat rooms" to websites which invite visitors to discuss issues in a free-form discussion. Although occasionally moderated by the website sponsor or a "special guest", these have been erroneously dubbed "focus group research" by companies eager to retrieve information that will be helpful in future marketing from their visitors. These chat-room environments do provide an important public-relations function as visitors appreciate that their opinions are solicited, and they do provide a sense of bonding, loyalty and community which is critical for the ongoing health of the site. However, they are not informed by models of rigorous research practice, lacking sampling specifications, controls on participation and respondent security, and a trained moderator capable of focusing the discussion in a productive and objective manner. Online focus groups or depth interviews (1:1 interviews) even more than survey research will require careful attention to the challenges of virtual environments. Questions which must be answered in order to conduct online qualitative research successfully are both practical and theoretical in nature: What happens when a computer intervenes in a human communication exchange? Are the gains from the online "anonymity effect", which makes the discussion of sensitive topics often more productive, adequately offset by the loss of nonverbal communication data, such as facial expression and voice tonality? How does an online moderator best handle the anti-hierarchical nature of online communication, in which people resist the introduction of authorities and all communications are treated as equally valid? Although online focus groups may cut across time zones and occur in a virtual, global environment, they also seem to be more effective when both shorter (an hour or less) and smaller (a maximum of 6-8 participants) than traditional focus groups. This means more groups must be done in order to obtain adequate data. Early findings on incentivization also suggest that online respondents must be well-compensated in order to contribute fully, despite the fact that they are not burdened with having to travel in order to participate. Apparently, keeping a "date with the computer" as opposed to a "real person" is less of a commitment and consequently focus group recruitment for online groups must anticipate high rates of attrition. Already a number of issues have emerged in the 18-24 month history of experimental online focus groups which call for a reconceptualization of traditional qualitative research practice, which also extends to the guidelines for training the prospective online moderator. Online moderators will need a detailed understanding of the cultural and social constructs of the online environment as well as extensive computing experience. They will also need to be well-trained in traditional qualitative research theory and practice and able to adapt creatively to new models for interactive media as they develop. A tolerance for "multi-tasking" is helpful as in some cases moderators must manage more than one group simultaneously as well as interact with observing clients who can message the moderator continuously during the group on private channels in some online software applications. Needless to say, these paragons are currently in short supply. Research companies will need to actively recruit and train moderators for this new function. It is unknown to what extent the lack of a perceived physical presence may impact the quality of research. This was of course an argument in the research industry when it made the painful transition from door-to-door interviews as a standard practice toward telephone and mail surveys. Virtual reality overlays (or a yet to be discovered VR-based Internet development platform), which will allow the online focus group to occupy a "virtual space" in real time in which participants may be visually represented and experience each other as physically present, will probably contribute significantly to the long-term viability of online qualitative research. Already some firms are experimenting with the impact of "respondent avatars" in which respondents are permitted to introduce their own visual representation into the focus group environment. These avatars range from cartoon characters, animals, and objects to photographic images of the actual respondent. This raises interesting questions, in that one of the key advantages of online communication at present seems to be that communication inhibition is decreased when one feels anonymous. Although this perception is incorrect (just about everything one does online leaves a "digital vapor trail" and can be tracked, so therefore one cannot be truly anonymous), it contributes to the noticeable ease with which online strangers meet and feel comfortable disclosing personal information. The neutrality of most of the online research environments being used, which avoid social cues such as real names or other identifying information about the respondent, also help enhance the quality of the disclosures. The introduction of images which mark the respondent with social cues will have measureable and significant effects and these effects will need to be considered. At even a basic level, the types of social cues which avatars introduce include social class (for example, the image resolution of an avatar reveals the relative access to computing hardware and software, as well as the respondent's proficiency levels with these) and cultural background (choosing a Japanese anime tells others one thing, as opposed to choosing an American television star to stand in for the respondent.) All of these reasons makes the analysis of qualitative data obtained online extremely challenging.

A SUMMARY OF ADVANTAGES AND DISADVANTAGES OF ONLINE FOCUS GROUPS Advantages · Anonymity (reduces communicative inhibition) · Sensitive topics seem to be productive · Multiple groups may be done simultaneously (although this requires special software and a skilled moderator). Larger overall sample sizes are possible. · Lower costs (no moderator travel; national or international groups can be done quickly) · Transcripts are automatically generated; turnaround can be very accelerated · It is not necessary to segregate groups by age or gender, although other segmentations may be desirable · Multiple moderators may be used across different time zones and may extend interviewing in an international study through 24-hours · Moderator/interviewer bias eliminated or substantially reduced Disadvantages · Sampling difficulties suggest panels must be used; however, extensive use of a panel for qualitative research may lead to rapid panel wearout · Identity validation must be rigorous and ongoing · Theory development is still at a primitive stage · Moderators must be trained interdisciplinarily and have a variety of computing skills few professional moderators are likely to possess now. · Substantial Over-recruitment is necessary as attrition rates are quite high · Groups must be both smaller and shorter than traditional groups

THE ETHICS OF ONLINE RESEARCH

The danger of online research being conflated with "spamming" activities will continue to be an area with which researchers should be actively concerned. In the 1970's and 1980's in North America, research organizations were forced to collaborate on setting standards of research practices as distinct from marketing, because of the drastically increased use of the form of the "telephone survey" by unscrupulous marketers as a guise for an aggressive marketing pitch. As a result of this phenomenon, consumer households began to refuse survey participation at much higher rates, which wreaked havoc on sampling designs and data quality. Legislation was introduced in the U.S. prohibiting the practice of using survey administration as a blind for telephone marketing ("telemarketing") and gradually confidence in telephone survey research improved. This confidence is at least partially due to surveyors learning the importance of reassuring respondents of the research firm's commitment to research ethics, such as protecting the confidentiality of respondent data. The CASRO and Marketing Research Association (MRA) separate codes of research conduct to which their members must adhere helped increase the awareness of the research industry, their clients, and the public as to the responsibilities of researchers. Unfortunately, no such set of standards has yet been developed for the practice of online research. Marketers are presenting visitors with questionnaires, secretly "trapping" information about them (although newer WWW browsers attempt to prevent this trapping), and selling information to third parties. Companies which call their "chat rooms" and informal site-based surveys "market research" and then sell this data or make other questionable use of it do not help the reputation of online research. Overall, there is a certain amount of deserved as well as undeserved hysteria online and in the press concerning the lack of privacy on the Internet, and its potential hazards and risks, and as a result there is also resistance to additional intrusions, even by legitimate researchers. It is clear that the research industry must be proactive in organizing around the protection of online information and must develop a coherent ethics for the practice of online research. If we fail to do so soon, we may find ourselves online pariahs and unable to answer even the most basic questions for ourselves and our clients.

ENDNOTES

1. Some organizations have conducted online research loosely based on a qualitative paradigm for far longer than 18-24 months. These include companies such as Minitel in France, Yankelovich/Cyber Dialogue (via AOL) and Nickelodeon (the cable television channel in the U.S.), via Compuserve. However, some of these tests resembled chat-room experiences, at least in their early stages. Furthermore, it is only in the past two years that an "online research industry" and the marketplace to demand it has evolved.

References

Alber, A. (1993). Interactive Computer Systems, Plenum Press. Bittle, S. (1995, December 14).

Surveying Customers on the Internet. Travel Weekly 54(99), p. 23.

Connolly, T. (1990). Effects of Anonymity and Evaluative Tone on Idea Generation in Computer-Mediated Groups, Management Science, 36, p. 97.

Ekman, P. (1980). Relative Importance of Face, Body and Speech in Judgments of Personality and Affect. Journal of Personality and Social Psychology, 47, p. 382.

Forsyth, D. (1983). An Introduction to Group Dynamics. Brooks/Cole.

Geller, V.J. (1981). Mediation of Social Presence: Communication modality Effects on Arousal and Task Performance, Bell Laboratories.

Harris, C. (1996) An Internet Education: A Guide to Doing Research on the Internet. Wadsworth.

Hiltz, S. (1986). Experiments in Group Decision Making: Communication Process and Outcome in face-to-face versus computerized conferences. Human Communication Research, 13, p. 225.

Hiltz, S. and Turoff, M. (1993). The Network Nation: Human Communication via Computer, MIT Press

Jones, S. (1995). Cybersociety: Computer mediated Communication and Community. Sage.

Kiesler, S. (1984, October). Social Psychological Aspects of Computer mediated Communication, American Psychologist, 39(10), p. 1123.

MacEvoy, B. (1994, January). Change Leaders and the New Media. American Demographics.

O'Malley, K. (1995, June 12). Marketing to Youth. Chicago Tribune. P. 1C.

Peebles, J. (1996, March 11). Online Technology Creates Research Tools. Marketing News, p. 5.

Reeve, V. (1995, December 1). Marketing to Women Online. Multimedia and Videodisc Monitor, 12(13), p. 1.

Roller, M. (1996, January 15). Virtual Research Exists, but how real is it? Marketing News, p. 13.

Rosen, S. (1994, August). Online Options for Focus Group Research. Communication World. 7(11), p. 33.

Thomas, P. (1995). The Social and Psychological Dimensions of Human-Computer Interfaces, Cambridge University Press.

Walther, J. (1992, February). Interpersonal Effects in Computer Mediated Interaction. Communication Research, 19(1), p. 52.

(1995, September 12). DEC's Online Research Program.

Newsbytes News Network. (1996). Competing Estimates of Internet Size Reported. Internet World Online (http://www.iworld.com). (1996).

SRI International, Inc. The Internet Values and Lifestyles Survey. (http://future.sri.com). (1996).

Commercial Uses of the WWW. University of Michigan Business School (http://www.umich.edu/~sgupta/hermes/) (1996).

WWW User Profiles.Georgia Institute of Technology (http://www.cc.gatech.edu/gvu/user_surveys/survey-04-1995) (1996).

Commercenet/Nielsen Internet Demographics Survey. (http://www.commercenet.com). (1996).

Interactive Consumers: Insights into Demand for Digital Content, Products and services.

FIND/SVP Research. (http://etrg.findsvp.com/iac_mprr/IC_1.1.html)

About the Author

Cheryl Harris, Ph.D., is an experienced e-business executive and entrepreneur, as well as a respected educator. A former professor at California State University and Parsons School of Design, New York, a published author and frequent international public speaker, she is well-known as one of the leaders in user experience and usability research. In 1996 she founded Northstar Interactive, an online research and consulting firm, and led the firm to its successful acquisition in February, 2000. Northstar developed web-based software and usability tools and consulted on strategy + design issues for such clients as Procter & Gamble, Motorola, Sprint, IBM, Netscape, Sony, AT&T, Time Warner, Roadrunner, Ogilvy & Mather, Grey, Modemmedia, Monsterboard, Mastercard, Citibank, eBay, Office.com, Insweb, Ziff Davis, Conde Nast, NBC, HBO, Discovery, and CNBC. She was also SVP, Interactive Strategy at Datek Online where her redesign of the online brokerage's site resulted in a doubling of customer accounts in less than four months. The new site was recognized or received top awards from Money magazine, TheStreet.com, Gomez Advisors, PC Computing, Red Herring, and several others. She is on the boards of several institutions, including the Lower Manhattan Cultural Council, the University of Massachussets IT initiative, WNET reelnewyork, and is a juror for several digital media festivals. Her publications include three books: An Internet Education (International Thomson Press, 1996) Theorizing Fandom (Hampton Press, 1998) as well as numerous articles. She received her Ph.D. from the University of Massachusetts-Amherst in 1992.