The Social Life of Information Systems Research: A Response to Benbasat and Zmud\u27s Call for Returning to the IT Artifact

Benbasat and Zmud (2003) argue that there is an identity crisis within the Information Systems discipline and, as a solution to the crisis, propose a focus on ¡°the IT artifact and its immediate nomological net¡± (p. 186). Using Aldrich¡¯s (1999) articulation of organizational evolution, they note the need for greater cognitive legitimacy as a driving force for sustainability of the discipline. They recommend that researchers and journal editors set the boundaries of the field more firmly so that greater attention is given to the IT artifact rather than to structure, context, or other phenomena that lie distant from the artifact. An alternative analysis of the IS field can be made through the lens of community of practice. Here the indicators suggest more positive progress toward legitimacy of the IS field and a path toward improvement via boundary enhancement rather than constraint. Other recommendations for improving the sustainability of the discipline include greater attention to research questions of current interest, even if they are peripheral to the artifact, greater communication of theory and empirical research results, and continued attempts to build and sustain active membership

Mobilizing Informational Social Capital in Cyber Space: Online Social Network Structural Properties and Knowledge Sharing

Online networks can be construed as social networks in which people engage in interactions, build rela- tionships, share information, and request and extend assistance to each other using electronic communication technologies. Is social capital embedded in online networks? How is such social capital mobilized (i.e., shared)? What structural properties of cyber networks are associated with mobilization of social capital? These questions have drawn the attention of researchers in the areas of social networks, social capital, and online communication. Our research is an initial effort to touch upon these three questions. Whereas most previous research on both conventional and online social networks has favored analysis of either egocentric (i.e., individuals) or bounded (i.e., groups or organizations) networks as the primary unit of analysis, this study investigates online discussion forums that span formal boundaries of organizations, examining their structural properties and patterns of information exchange. We employ a network-based approach to the study of social capital, postulating that mobilization of social capital is contingent on social network properties. Using data from professional online forums devoted to knowledge management, we find that two network properties, core–periphery structure and centralization, are related to the mobilization of informational social capital in online networks. Implications and limitations of this study are discussed and suggestions for future research are provided

An Investigation of the Tables Versus Graphs Controversy in a Learning Environment

The study of computer graphics as decision aids has become popular among MIS researchers in the last several years. However, this area of research, like many others in management information systems, has been plagued with methodological problems and contradictory findings. In light of these difficulties, the current study examined the tables versus graphs controversy within a learning environment. Seventy-five MBA students were exposed to one of three experimental treatments and asked to develop financial forecasts for fictitious companies over five experimental trials. Following their forecasts for each firm, participants were provided with feedback on the quality of their decisions. The information presentation treatments were as follows:(l) traditional spreadsheet (tabular), (2) graphs using standard scaling, and (3) graphs using nonstandard scaling. Results suggest that, although graphics may initially demonstrate no advantage over tables, they do show an advantage i f decision makers are repeatedly exposed to the novel format and given feedback on their performance. L. arning will occur even when improper scaling is used. The implication is that the effectiveness of graphics as decision aids depends on practice. Researchers are encouraged to employ repeated measures, or longitudinal, designs when examining the tables-versus-graphs controversy

The Dynamics of Teams and Technology: A Field Study of Groupware in a Network Organization

Groupware technology, such as electronic communication systems, discussion databases, collaborative writing tools, and workflow applications, has been viewed by both scholars and practitioners as having the potential to facilitate productive teamworkandenableanorganization’stransitiontotheso-callednetworkdesign. Networkedorganizationsrelyonmultiparty cooperative relationships across structural and geographic boundaries, yielding dense, flexible communication patterns. Because groupware systems provide a platform on which teams can support their communication needs and shared work obligations, teams which use groupware should experience improved information exchange and fewer coordination problems than those that do not. Further, organizations that invest in groupware systems should make more rapid progress toward a network form than those that do not. The added value of groupware, relative to more primitive forms of communication support, should increase as team members become more facile in new technology use and modify their work practices to accommodate computer-mediated collaboration

Panel 11 The Web and IS Academics: An Opportunity or a Trap?

The area of Information Systems research has been blessed — or cursed — with fast evolving subjects of study that have created a lot of hype, captured the attention and imagination of wide audiences, and generated a significant demand for corresponding literacy and skills. At the moment, it seems that none of these — the use of computers for the management of large scale operations, the augmentation of human intelligence, and personal computing — has become such a broadly referred phenomenon as the Internet and its related uses, most prominently the World Wide Web

Understanding the Effectiveness of Computer Graphics for Decision Support: A Cumulative Experimental Approach

A total of 840 junior and senior-level undergraduate business students participated in three experiments which compared computer-generated graphical forms of data presentation with traditional tabular reports. The first experiment compared tables and bar charts for their effects on readability, interpretation accuracy and decision making. No differences in interpretation accuracy or decision quality were observed for the two groups, although tabular reports were rated as more easy to read and understand than graphical reports. The second experiment compared line plots with tables for their effects on interpretation accuracy and decision quality. Subjects with graphical reports outperformed those with tables. There were no meaningful differences in interpretation accuracy across treatment groups. The third experiment compared graphical and tabular reports for their ability to convey a message to the reader. Only in situations in which a vast amount of information was presented and relatively simple impressions were to be made, did subjects given graphs outperform those using tables. This program of cumulative experiments indicates that generalized claims of superiority of graphic presentation are unsupported, at least for decision-related activities. In fact, the experiments suggest that the effectiveness of data display format is largely a function of the characteristics of the task at hand, and that impressions gleaned from one shot studies of the effectiveness of the use of graphs may be nothing more than situationally dependent artifacts

USING A GDSS TO FACILITATE GROUP CONSENSUS: SOME INTENDED AND UNIMTENDED CONSEQUENCES

RATIONALE AND PURPOSE OF THE STUDY The empirical research examining group decision support systems suggests that many of the hopes for GDSS can be realized. For example, Lewis (1982) and, more recently, Gallupe (1985) both found that groups supported by a GDSS made higher quality decisions than groups without GDSS support. Applegate (1986) and Steeb and Johnston (1981) have demonstrated the viability of GDSS in live planning situations. Positive effects of a GDSS on groups have also been reported by Gray et al. (1981), Turoff and Hiltz (1982), and Siegel et al. (1986). Computer support has been shown to foster a democratic approach to the decision process, with more equality of participation among members (Siegel et at. 1986), to improve satisfaction with the decision process (Applegate 1986), and to result in a greater shift away from initial individual preferences (Siegel et al. 1986). These intended effects of the technology have been demonstrated for a limited number of task types. To date, positive effects of GDSS have been observed for idea generation (Applegate 1986; Lewis 1982), problem finding (Gallupe 1985), intellective choice (i.e., selection of a correct answer among a given set of alternatives) (Hiltz and Turoff 1982), and planning tasks (Applegate 1986; Steeb and Johnston 1981). In two of these studies, group members were dispersed and interacted with one another via a communication network (Hiltz and Turoff 1982; Siegel et al. 1986), while in the other studies group members met in a face-to-face (i.e., conference room) setting. In all cases, each member had direct interaction with the GDSS, and in most of the studies the performance of the group was compared to an objective measure of decision quality. Of course, many organizational meetings occur without prior or post knowledge of the correct outcome of a group meeting. For this reason, the current study aimed to build on the available knowledge of GDSS impacts by examining the usefulness of the technology in situations where a group must resolve competing personal preferences and maximize agreement on a solution to a problem. In such situations, achieving high decision quality is not the primary goal of the group meeting. The theory of GDSS would argue that the technology should be as useful in achieving consensus as in identifying correct solutions. In either situation, the GDSS should foster more even participation in the decision and a more systematic, or structured, group decision process (DeSanctis and Gallupe 1987; Huber 1984a). For the most part GDSS research is being conducted in laboratory settings where the organizational context and other factors can be controlled so that the impact of the technology on group outcomes can be carefully assessed. The current study aimed to build on the available GDSS research by systematically comparing groups supported with a GDSS with groups that had either no support whatsoever ( baseline groups) or a paper-and-pencil ( manual ) support system, that contained the same decision structure as the GDSS (cf. Lewis 1982). The purpose of having two control groups was to determine whether increments or decrements in outcomes were due to the GDSS or simply due to imposing a problem-solving structure on the group. Three major hypotheses were investigated: HYPOTHESES Hl. The degree of post-meeting consensus will vary as a function of the type of support given to the group. Hla. Post-meeting consensus will be higher in the GDSS groups than in the manual support or baseline groups, controlling for initial level of conflict. Hlb. Post-meeting consensus will be higher in manual support groups than in the baseline groups, controlling for initial level of conflict. H2. The equality of influence will vary as a function of the type of support given to the group. H2a. Influence will be more even in the GDSS groups than in the manual support groups. H2b. Influence will be more even in the manual support groups than in the baseline groups. H3. Attitudes toward the group process will be different in the GDSS groups than in the manual system and baseline groups. METHOD Forty-four three-person and 38 four-person groups participated in the study. Group size in this study was similar to that in previous research (Lewis 1982; Gallupe 1985; Siegel et al. 1986). The groups were made up of undergraduate and graduate students enrolled in introductory MIS classes. Many of the students were employed full-time in business settings, and most were working at least parttime. On average, the participants were 24 years of age with slightly more than two-and-a-half years of work experience in a business or related setting. All of the groups were live groups in that they were actively working together as teams on class assignments. In this way, the initial socialization that occurs early in group formation could be avoided during the data collection. THE GDSS The GDSS, called Computer Assisted Meeting (CAM), was designed, coded, and tested by a research team at the University of Minnesota. The system is described in DeSanctis and Dickson (1987) and is being used for several related studies of group DSS (Poole and DeSanctis 1987; Watson 1987; Zigurs 1987). Basically, the system incorporates a rational problem-solving agenda (Dewey 1910). The software is similar to that used by Lewis (1982) and Gallupe (1985) in that it performs the basic functions of recording, storing, and displaying problem definitions, criteria for evaluating solutions, alternative solutions, and a final group decision. Group members can enter relative weights for solution criteria, and the system will aggregate and display average group weightings. In addition, the system will cumulate and display ratings, rankings, and votes associated with one or more alternative solutions to a problem. These features have been identified as appropriate for supporting the communication needs of groups (Huber 1984b; DeSanctis and Gallupe 1987; Joyner and Tunstall 1970). Experimental Task and Procedure The research task required subjects to allocate a given sum of money among six competing projects that have requested funds from a philanthropic foundation. Conflict arises because the team members have varying preference structures that result in different allocation patterns. The projects that subjects can fund are based upon the personality components scheme described by Spranger (1928), who asserts that there are six basic interests or motives in personality: theoretical, economic, aesthetic, social, political, and religious. The six projects that can be funded correspond to Spranger\u27s six personality traits. Correlation analysis based on the 300 experimental subjects was used to check that the amount allocated to a project by an individual was highly correlated with that person\u27s values as measured by the Study of Values instrument (Allport et al. 1970). The strengths of the task are twofold. First, it produces conflict in a group. Second, the source of the conflict is identifiable; it is based upon different preference structures arising from varying personality traits. The task and its validation are further described in Watson (1987). The experimental procedure was as follows: 1. Subjects listened to a standard introductory script read by the administrator of the experiment, and then read a background statement. 2. Subjects completed a consent form, a background questionnaire, and the Study of Values instrument. 3. Subjects individually allocated funds to the six projects requesting support from the philanthropic trust (these measures were used to calculate pre-meeting consensus). Subjects also allocated funds to five other sets of six projects each in order to give them practice and to help stabilize their reasoning processes. 4. Groups allocated funds to the six projects requesting support from the philanthropic trust. 5. Subjects completed a post-meeting questionnaire for measuring an individual\u27s perception of the group\u27s decision-making process, and individually allocated funds to the six projects requesting support from the philanthropic trust (these were used to calculate post-meeting consensus). 6. The administrator conducted a debriefing of the subjects. During section step 4 of the experiment, the group decision-making phase, teams were given one of the three treatments discussed previously. In the case of the manual groups, subjects were provided with a eleven-page handout outlining the same agenda that was on the GDSS. Each page of the handout explained an agenda item, giving details on how to accomplish the item parallel to those in the submenus of the GDSS. Manual groups were given a flip chart to display ideas publicly. Every effort was made to ensure that manual groups had the same structural aids as the GDSS groups, the only difference being that the manual groups operated without computer support. GDSS groups were provided with a 20-minute training session on use of the system, manual groups were also trained in how to use the meeting structure. Baseline groups were given no structure, flip chart, or training. They were told to operate with their own resources. FINDINGS This investigation identified some intended and unintended effects of using a decision support system for groups. Overall, the results on consensus and equality of influence for the GDSS and manual conditions tended to be similar, showing different patterns than the results for the baseline condition. As intended, the presence of a suggested structure for the group meeting improved the degree of post-meeting consensus. Also, in contrast to the baseline and manual system group meetings, users of the GDSS reported more input into the group\u27s solution and were less likely to perceive that there was a leader in the group. The relationship between pre-meeting and post-meeting consensus was similar in GDSS and manual groups, but post-meeting consensus was not significantly higher in the GDSS groups than in the baseline or manual groups. Although the structure provided in the GDSS and manual conditions reduced the variance across groups on their equality of influence, use of the GDSS did not result in more equal influence of group members on the final solution. The most surprising unintended effect was that GDSS users, compared to the other experimental groups, perceived the issues discussed in the group meeting to be more trivial and the group\u27s problem solving process to be less understandable. Other observations of the study were that use of the GDSS tended to reduce face-to-face interpersonal communication in the group; use of the GDSS presented a challenge to the groups, thus making their meeting task more difficult than groups without the GDSS; and groups using the GDSS appeared to become very procedure-oriented, rather than issue-oriented, in their discussions. In the future, GDSS research should press further to sort out what Kiesler calls intended technological effects (faster processing, fewer errors, more equal participation), unintended social effects (heightened conflict), and transient effects (effects that will diminish with group experience with the system) of the technology on groups. ACKNOWLEDGEMENT This project was funded by NCR Corporation, the MIS Research Center, and the Graduate School of the University of Minnesota

Methodological Issues in Experimental IS Research: Experiences and Recommendations

Within the last ten years, the use of experimental methodology in information systems research has substantially increased. However, despite the popularity of experimentation, studies suffer from major methodological problems: (1) lack of underlying theory, (2) proliferation of measuring instruments, (3) inappropriate research designs, (4) diversity of experimental tasks, and (5) lack of internal validity. These problems have led to an accumulation of conflicting results in several areas of IS research, in particular, research in the area of graphics and information presentation. This paper uses the area of information presentation format to explore the nature of the methodological problems mentioned above and to suggest potential remedies: 1. Due to the lackoftheoretical basis, informationpresentationresearchers do nothave any common ground for conducting and interpreting their results. This has resulted in oneshot, ad-hoc studies that do not build on the work of others. No state of relatedness among studies has emerged. Only through progrums of research can we hope for an underlying theory to emerge. 2. The proliferation of measuring instruments, many of which may have problems with reliability and validity, has plagued IS research. Again, only through a program of research can we hope to construct a set ofmeasuring instruments applicable and easily adaptable to a large number of studies 3. With regard to research design, simplistic and nonpragmatic studies as well as poorly controlled experiments have impeded the progress of IS research. Suggested remedies include the adoption of multivariate designs, use of decision maker productivity as a dependent vatiable, and more effective experimental control through measurement of factors that are 1mown from previous research to influence decision performance. 4. The presence of a multitude of task environments has also posed problems. The employment of diverse tasks makes comparisons of results across studies inappropriate. A taxomony of tasks must be developed before we can meaningfully integrate research findings. 5. Many studies have suffered from internal validity problems. A remedy for this requires more effective precautions to ensure that the findings of a study are due to the factors researched rather than to accidents. Tb illustrate this last problem of internal validity and the steps needed to improve the experimental studies involving mangerial graphics is described. The research study conducted at the University of Minnesota was initially set up to investigate the relationship between graphical decision aids, task complexity, and decision maker performance. First, a task, and a case that was to provide a task setting, were developed. Also, questionnaires and tests were constructed to gather information on the (1) backgroundof subjects, (2) motivation of subjects, (3) subjects\u27 satisfaction withthe graphs, (4) perceived complexity and difficulty of the problem solving task, and (5) the subjects\u27 interpretation accuracy in reading graphs After the development of the tasl and other experimental material, the experiment was pretested The results from the pilot study gave the authors every reason to believe that the task did not have any major validity problems. However, whenthe experiment was actually given to 63 graduate students, the data didnotreveal any consistent patterns due to graphical and task treatments. This, of course, concerned the authors, and, as a result, attention was directed toward improving the experimental task, research design, and measurement A second experiment was conducted to test whether the insignificant results in the first experimentwere causedbythegraphsorbymisleadingorconfusinginformationinthetask. The data from the second experiment collected on 20 experimental subjects, convinced the authorsthatthemainproblemcausingtheinsignificantresultshadnotbeenthepoorquality of the graphs, but the fact that, in general subjects were just not able to perform the task However, the authors did not know whether this poor performance was due to an overly difficult task or to misleading or confusing information within the task. Therefore, a third experiment was conducted to resolve this question. The third study used 17 managers as experimental subjects. It was assumed that if the managers couldsatisfactorilycompletethetasktheauthors couldconcludethatthe taskwas valid, but too difficult for graduate students. The analysis of the data collected from the third experiment confirmed, however, that serious problems existed with the task itself. Debriefings of the managers indicated that the case description, in combination with the presented data on marketing variables, included confusing and misleading data Obviously, thetaskwasnotprovidingthebasisforansweringtheresearchquestionontherelationshipof task, presentation format, the decision performance. Thus, a majorrevision of the task was undertaken. The revised material is currently undergoing pretesting. In summary, the authors have gone through several experiments in searching and testing for valid measurements. During this process we have learned an invaluable lesson that we hope willbe usefulto othersintheirresearch endeavors. We discoveredthatthe process of coming up with an effective taskand variable measurementis lengthy, costly, and mayhave uncertain outcomes even if considerable precautions are taken. For experimental IS researchers particularly those performing studies on the use of managerial graphics, cautions and guidelines are provided to help them address more effectively the common methodological

THE IMPACT OF COMPUTER-BASED SUPPORT ON THE PROCESS AND OUTCOMES OF GROUP DECISION MAKING

Interactive computer-based systems to support group decision making (group decision support systems or GDSS) have received increased attention from researchers and practitioners in recent years. Huber (1984) argues that as organizational environments become more turbulent and complex, decisions will be required to be made in less time and with greater information exchange within decision making groups. Thus, it is imperative that studies be undertaken to determine the types and characteristics of group decision tasks most appropriate for support by a GDSS and to determine the features of a GDSS that will support those tasks. A number of prominent researchers in the field of group decision making (Shaw, 1973, 1981; Hackman and Morris, 1975; Fisher, 1974) agree that the decision task itself is probably the most important factor in determining group decision making effectiveness. The characteristics of group decision tasks are many and varied, but according to Shaw (1973) the level of difficulty/complexity of the decision is a fundamental factor in influencing the performance of the group. Some decisions are characterized by information that is clear, concise, easily communicable, and where relationships between important factors in the decision are easily understood. In short, these decisions require relatively little effort to make and are therefore called easy decisions. Decision tasks where the information to be considered in making the decision is incomplete, difficult to understand, and where complex relationships exist within the information available are called complex or difficult decisions. The role of decision task difficulty in the effective use of GDSS is considered ih this study. This research is an initial experimental study, exploratory in nature, that aims to get a first-level understanding of the impact of a computer-based DSS on group decision making. The group decision support system that is used in this study has only those features that specifically support group decision making (alternatives generation and communication, preference ranking and voting support). The reason for this approach is to start a program of research with a simple system in order to determine the particular impact of these features on, not only the outcomes of group decision making (such as decision quality), but on the process of group decision making as well. A controlled 2 x 2 factorial experiment was used to compare the decisions made by groups which had GDSS support with those groups that had no GDSS support and those with a high difficulty task to those with a low difficulty task. Figure 1 shows the relationship among the main variables in the study. The experimental task was a marketing business case in which the company was experiencing declining profits. Each group was asked to find the problem which was causing the declining profits. Difficulty was manipulated by modifying the data in the case. The setting for this experiment was a decision room designed and set up to accommodate face-to-face group interaction. The GDSS treatment entailed the use of one computer terminal per group member so that the GDSS could be used to support group decision making. Each group member in the GDSS treatment also had the use of a pencil, paper, a hand calculator, and a blackboard. For the non GDSS treatment, the terminals were removed and the group used just pencils, paper, hand calculators, and a blackboard to assist in making the decision. The computer hardware consisted of a DEC VAX 11/780 timesharing system using the VMS operating system, and DEC VT-102 terminals. The terminals were connected to the VAX 11/780 using 2400 baud direct lines. The GDSS called Decision Aid for Groups (DECAID) was designed, coded, and tested to make sure that it worked in the experimental setting. The approach to design was to implement the features, and then to refine the system through testing to make those features work as efficiently as possible. The GDSS software performed the basic functions of recording and storing and displaying alternatives that were entered by group members, aggregating and displaying preference rankings that had been entered for those alternatives, and recording votes (either publicly or anonymously) for the various alternatives generated. The system was easy to use and menu driven. Eighty four senior undergraduate business administration students participated in the study. These subjects had taken at least one course each in management science/decision analysis techniques, marketing, management theory/organizational behavior, and all had exposure to case analysis techniques. All subjects had been given training in the use of the GDSS. Measures were taken of decision outcomes (decision quality, decision time, decision confidence, satisfaction with group process, and amount of GDSS usage), and decision process variables (number of issues considered, number of alternatives generated, and participation in the decision making). Decision quality was measured along two dimensions: (1) decision content - how close did the group\u27s decision come to that made by a panel of experts; and (2) decision reasoning -- how similar the group\u27s reasoning in arriving at their decision was to the reasoning of the experts. Decision time was defined as the length of time it took the group to reach a consensus decision. Decision confidence and satisfaction with the group process were measured by individual responses to a post- test questionnaire. The individual responses were then aggregated to give a group value. The amount of GDSS usage was measured by examining the computer logs that were kept during the GDSS sessions. Decision issues were defined as factors that were important in the analysis of the case. Decision alternatives were defined as those issues in the case that the group analyzed as being the possible major problems in the case and hence, possible solutions to the decision task. Participation was measured by counting the number of task related comments made by each individual group member. Issues, alternatives and participation were determined by analysis of the video and audio tapes that were made of the experimental sessions. The major findings of the study are: 1. Decision quality is enhanced when decision making is supported by a GDSS, particularly for high dificulty tasks. 2. Decision time is not affected by use of a GDSS. 3. Confidence in the group decision and satisfaction with the decision making process are reduced when a GDSS is used, irrespective of task difficulty. 4. The number of alternatives considered is increased when a GDSS is used to support group decision making. 5. Participation in the group decision making process is unaffected by GDSS support or by decision task difficulty. The paper concludes by suggesting directions for future research into GDSS. Work is needed to determine the effectiveness of additional features of a GDSS (such as other communication features, modeling features, etc.), to understand the impact of GDSS on the different phases of decision making, and to examine the effect of repeated use of a GDSS on the quality of group decision making