ABSTRACT In this paper we describe a new class of collaborative scientific applications that incorporate heterogeneous devices, such as shared supercomputing or visualization resources, personal computers, and mobile devices, present some classes of collaboration tasks that could profitably make use an application infrastructure that connects heterogeneous devices, and identify some particular applications that fall into these classes. We also describe in greater detail one potential application, natural science field research that employs sensor networks. We also discuss some of the problems that need to be addressed in building an application that allows such heterogeneous device collaboration and some benefits to digital science that could be realized by building collaborative applications in this fashion

Mary C Whitton

Richard Skarbez

CiteSeerX

 1
Enabling Distributed Collaboration  
among Heterogeneous Devices 
Richard Skarbez 
University of North Carolina at Chapel Hill 
Department of Computer Science 
skarbez@cs.unc.edu 
 
Mary C. Whitton 
University of North Carolina at Chapel Hill 
Department of Computer Science 
whitton@cs.unc.edu 
 
ABSTRACT 
In this paper we describe a new class of collaborative 
scientific applications that incorporate heterogeneous 
devices, such as shared supercomputing or visualization 
resources, personal computers, and mobile devices, present 
some classes of collaboration tasks that could profitably 
make use an application infrastructure that connects 
heterogeneous devices, and identify some particular 
applications that fall into these classes. We also describe in 
greater detail one potential application, natural science field 
research that employs sensor networks.  We also discuss 
some of the problems that need to be addressed in building 
an application that allows such heterogeneous device 
collaboration and some benefits to digital science that could 
be realized by building collaborative applications in this 
fashion. 
Author Keywords 
Distributed collaboration, collaborative science, mobile 
devices, visualization, human-computer interaction. 
ACM Classification Keywords 
H5.3. [Information interfaces and presentation (e.g., HCI)]: 
Group and Organization Interfaces---collaborative 
computing.  
INTRODUCTION 
The “big shared resource” (BSR) has long been an 
important part of scientific research. In the past, this may 
have been a large-scale telescope or a particle accelerator. 
Today, it could be any of these traditional devices 
augmented with computer control, or it might be a 
supercomputer or a sophisticated visualization environment, 
such as a dome or a CAVE. Regardless of type, BSRs are 
generally expensive, are difficult to install and maintain, 
require specialized expertise to operate, and are often 
physically distant from researchers and scientists who 
would like to use them. Enabling collaboration using the 
BSR has been achieved by networked interfaces or bringing 
all collaborators to the BSR’s location. This is expensive, 
inconvenient, and inefficient. Today, virtually every 
scientist has access to a personal desktop or laptop 
computer, and there have been efforts to enable researchers 
to do collaborative science using desktop computers, 
including the Collaborative nanoManipulator (CnM) [1], a 
networked interface for controlling an atomic force 
microscope which also served as a platform for analysis of 
distributed collaboration. Projects such as that one address 
some of the collaboration difficulties discussed above. 
Scientists today have access to increasingly powerful 
mobile devices. These devices are available, relatively 
cheap, and network-enabled. Many “smartphones” possess 
sufficient audio/video and computing capabilities to run 
fairly heavyweight applications such as computer games or 
video players, or, potentially, scientific applications. That 
said, these devices still lag far behind desktop computers or 
workstations in terms of processing power, display size, 
network bandwidth and user interface sophistication. 
An application infrastructure that connects these different 
classes of devices (BSRs, PCs, and mobile devices) and 
enables collaborations among their users would benefit 
distributed science. In the remainder of this position paper, 
we present some potential benefits of such an infrastructure, 
applications where such technology would benefit scientific 
research, and discuss some problems that need to be 
addressed and ideas for building such an infrastructure. 
POTENTIAL BENEFITS 
A collaboratory based around a BSR would naturally 
benefit from allowing users on less-capable devices to 
interact with the BSR, or other users of the BSR, over the 
network. There are some problems of collaboration that 
could be directly addressed with such technology, and there 
are some types of collaborative activities that would benefit 
more clearly than others. 
The primary problem with BSRs is that they are not easily 
accessible. Allowing remote access on commodity 
computers or mobile devices would help to alleviate this 
 
 
Permission to make digital or hard copies of all or part of this work for 
personal or classroom use is granted without fee provided that copies are 
not made or distributed for profit or commercial advantage and that copies 
bear this notice and the full citation on the first page. To copy otherwise, 
or republish, to post on servers or to redistribute to lists, requires prior 
specific permission and/or a fee. 
CHI 2009,   April 4–9, 2009, Boston, MA, USA. 
Copyright 2009 ACM  978-1-60558-246-7/08/04…$5.00 
 2
problem. This is particularly true in time-sensitive 
situations, where it may be impractical or impossible for all 
collaborators to be in the same place. 
In addition, extending the collaboration to include remote 
users makes it easier to accomplish iterative refinement in 
the sciences, by closing the distance (real or metaphorical) 
between experiment and analysis. For example, consider 
the fields of archaeology and cultural heritage preservation. 
Virtual environments (VEs) are being used as tools in these 
fields. (There is an annual symposium at IEEE VAST on 
the topic.) However, field research is still an essential part 
of study. By bringing mobile users into the collaboration, 
researchers examining a virtual model of a site in a CAVE 
can directly indicate to the mobile user an area of interest, 
and she can examine the location in person, and 
immediately send back measurements or pictures that can 
be incorporated into the model seen in the BSR.  
A similar situation can be imagined in the physical 
sciences: Researchers analyzing the results of an 
experiment (possibly an on-going experiment) in a dome 
display can identify interesting features to a remote user 
who is controlling the instrument, and he can then modify 
the experiment or take additional measurements that can be 
incorporated into the data and visualization. 
Finally, the design and evaluation of collaborative 
applications for heterogeneous devices, including mobile 
devices, have not been the subject of much study. We 
believe that research in these areas would yield productive 
insights into the nature of distributed collaboration, 
scientific or otherwise. 
POTENTIAL APPLICATIONS 
One benefit of enabling heterogeneous device collaboration 
is that data-gathering and analysis can be more tightly 
coupled. That said, it is true that many collaborative tasks 
are either primarily real-time (multiple parties working to 
gather data, run an experiment, or perform a complicated 
task), or primarily post-experiment (multiple parties 
participating in data analysis, or designing the next 
experiment). This distinction can also be used to classify 
BSRs. Traditional scientific instruments are most often used 
in real-time in the conducting of experiments. Visualization 
tools such as CAVEs, domes, display walls, etc. are used 
primarily for post-experiment analysis. Supercomputers can 
be used in both ways, depending on the nature of the task. 
Another useful distinction is whether the collaboration is 
being directed by a user or users local to the BSR, or by a 
remote user. To clarify, we use directing to mean guiding 
the direction of the science. For example, consultation is 
listed as a remote task in Table 1, even though local users 
may very well have initiated the action. 
In a distributed collaboration where the direction is local to 
the BSR and the task is real-time, an expert can be using the 
BSR for display and directing the remote user in 
performance of a difficult task, as in 3D Medical 
Consultation [2]. Also, the task can be divided among 
multiple users, where an expert is controlling the equipment 
and one or more other users are observing the data being 
generated, or controlling other aspects of the experiment. 
The archaeological task discussed in the previous section is 
an example of the case where the direction is local to the 
BSR and the task is post-experiment. The case where the 
direction is remote and the task is real-time is a consulting 
task; an example would be a remote medical consultation. 
Finally, the case where the direction is remote and the task 
is post-experiment could be a situation where a remote 
expert reviews the performance of a task with the personnel 
who were involved, discusses changes, and plans future 
work. 
USE CASE - SENSOR NETWORKS  
An example might help to illustrate the utility of this idea. 
Consider the use of sensor networks for field research in the 
natural sciences, e.g. monitoring animal populations. Such a 
sensor network could generate a huge amount of data, 
which would be transmitted to and stored in some central 
facility. The behavior of the sensor network itself could 
easily be monitored and managed from a variety of devices, 
including mobile devices and regular desktop computers.  
The captured sensor data could be visualized in a CAVE, 
where one or multiple researchers could analyze the results. 
These researchers might find something of interest in the 
data, and could communicate that directly to a field 
researcher on-site with a mobile device, guiding the user to 
the particular item of interest. The field researcher could 
then make adjustments to the sensor behavior using his 
mobile device, and capture pictures or video which could be 
transmitted directly back to the central facility and 
displayed to the researchers in the CAVE. They could then 
immediately determine additional actions to be taken or 
form hypotheses about the data, which could be sent back 
to the on-site researcher (along with any relevant data) to 
get her opinion, and so on. 
In addition, the field researcher could access the BSR from 
her mobile device in order to make use of its more powerful 
computational resources, allowing her to do productive 
analysis work from the field. 
APPROACH 
There are many problems that need to be addressed for such 
a heterogeneous collaboration infrastructure to be deployed 
 Real-Time Post-Experiment 
Local 
Direction 
Direction / 
Doing and 
monitoring 
Directed data gathering 
Remote 
Direction 
Consultation 
Evaluation / 
Planning 
Table 1. Collaborative scientific tasks grouped by the 
location of the expert and the timing of the task 
 3
and used successfully. These problems can be grouped into 
four categories: data management and communication, 
visualization, human-computer interaction, and 
collaboration support. Our research interests are primarily 
in visualization and user interaction techniques for 
collaborative applications on mobile devices, and so that 
will be the focus of the discussion. 
An analogy may help to clarify many of the points below. 
Consider lakes as data, pipes of different sizes representing 
network bandwidth, and computing devices as buckets that 
can do something with the water. A computational BSR has 
a large bucket (advanced visualization hardware, powerful 
computational resources, large data storage) that needs to 
be filled from a small pipe from a small lake (the data sent 
from the mobile device). Meanwhile, the mobile device has 
a much smaller bucket (small display, less computational 
power, etc.), fed by a small pipe from a large lake (the data 
generated by the BSR). Because of this inherent resource 
asymmetry, we as researchers must develop visualization 
and computer-human interaction techniques that can 
overcome these differences and enable effective 
collaboration. 
Data Management and Communication 
Scientific applications often involve large datasets. Mobile 
devices, in general, have limited memory, limited 
computational resources, and limited—as well as 
inconsistent—network connectivity. Applications that allow 
mobile users to interact with large datasets should provide 
them the option of working with portions of the dataset or a 
“lower-resolution” representation of the data in order to 
minimize storage, bandwidth, and power requirements. 
Simply generating a coarser sampling of the data is 
problematic for scientific applications, since doing so can 
eliminate important high-frequency detail, or create 
artifacts that are not in the underlying data. Developing or 
identifying a way of generating multi-scale representations 
of data while preserving “important” features is a crucial 
step in developing this collaboration infrastructure. 
Also, sufficient data should be stored locally to allow the 
mobile user to continue working in cases of poor or 
nonexistent network connectivity. Finally, there may be 
significant network latency in such a collaborative 
application; this must be taken into account in the design of 
the application. 
Visualization 
CAVEs, large-scale display walls, and head-mounted 
displays are now in use in many facilities. Even low-end 
desktop computers generally have displays with greater 
than one megapixel resolution. Meanwhile, mobile devices 
have displays of less than 4” in size with resolutions of 
480x320 or less.  Furthermore, mobile devices have less 
rendering capability than desktop machines or workstations, 
and high-quality rendering often comes at the expense of 
decreased battery life. 
Applications tailored to mobile devices must account for 
the greatly decreased field of view and screen real-estate 
available to users of these devices; at the same time, they 
should take advantage of some of their unique 
characteristics, such as improved resolution (in terms of 
pixels per inch) and ease of repositioning. Such applications 
should also recognize the fact that their users will 
frequently be operating in situations where they need to 
preserve battery life; they should provide users with the 
option of “gracefully degrading” the visualization quality in 
order to minimize power usage. 
Human-Computer Interaction 
Mobile devices employ non-traditional input techniques 
such as touch- or pen-based input. In addition, input 
modalities are not consistent across mobile devices: some 
have keyboards, and some do not; some have touchscreens, 
and others do not; some allow pen-based input, and some 
do not; and so on. These devices also often have many input 
modalities that have not been fully explored for or were not 
applicable to desktop computers; these include cameras, 
GPS units, and accelerometers. 
Application developers for mobile devices should be aware 
of these differences, and should design interfaces that take 
advantage of mobile devices’ unique characteristics rather 
than attempting to mimic the interfaces of desktop 
computers or advanced visualization facilities. We need to 
develop a body of knowledge about usability and interface 
effectiveness for scientific applications so that application 
developers can make informed design choices. 
Collaboration Support 
These ideas are focused on how an application should be 
designed so that the mobile user can interact with users on 
desktop computers, or users in advanced visualization 
environments. Note that we are not talking about other 
sociological factors that come into play in collaborative 
scenarios, such as situational awareness; we are focusing 
only on the technological aspect of how people would work 
together in an environment of heterogeneous resources. 
Users on mobile devices see a very different data rendering 
and user interface than users of more capable devices. 
There are differences in display size and resolution. In 
addition, the mobile user may be interacting with a more 
coarsely sampled version of the dataset, or may be viewing 
the data as a non-photorealistic rendering that may be better 
suited to the limitations of the mobile hardware. As a result, 
the particular presentations that each user sees may be very 
different.  Effective collaboration, however, often requires 
that users are looking at the same thing, or at least similar 
enough representations that the elements that are important 
for the science under discussion are visible to all parties. 
They need to be able to refer to common landmarks. The 
differing representations must be “connected” such that 
users of both can effectively communicate about the data. 
 4
Given what was discussed earlier, it is likely that an 
application designed for a mobile device and a “partner” 
application designed for a desktop computer, or a CAVE, 
will have different user interface techniques and 
affordances, different renderings, and possibly different 
underlying data (in the sense that the mobile user might 
have a coarser sampling of the data or some such). If the 
mobile user performs some manipulation of the dataset, or 
wants to draw his collaborators attention to some feature of 
the data, he must be able to know that his collaborators 
receive and understand that message, and that it is correctly 
mapped onto their representation of the data. The converse 
should hold true as well. The problem here is identifying 
the intent of a user working with one representation of the 
data, and communicating that intent to another user 
working with a different representation. 
CONCLUSION 
Our intentions in this position paper were to identify some 
problems common to distributed scientific collaboration, 
and to point toward a new collaboration paradigm 
incorporating heterogeneous devices that could help resolve 
these difficulties. Big shared resources are essential to 
many scientific pursuits, not only in experimentation but 
also increasingly in analysis, in the forms of visualization 
facilities. Allowing both co-located and remote users to 
observe or control these BSRs from a range of 
heterogeneous devices could increase the productivity of 
existing collaboratories, or make possible collaborations 
that were previously unfeasible. As our particular research 
interest is in enabling collaboration between users of BSRs 
and users of mobile devices, we then pointed out some of 
the difficulties inherent in developing scientific applications 
for mobile devices, and some design rules that might ease 
some of these difficulties. 
ACKNOWLEDGMENTS 
We thank Rob Fowler and Ray Idaszak for helpful 
comments on a very early version of the paper, and the 
Renaissance Computing Institute for supporting us while 
we were developing the ideas contained herein. We also 
thank James Horey for fruitful discussion regarding sensor 
network applications. 
REFERENCES 
1. Hudson, T. C., Helser, A. T., Sonnenwald, D. H., and 
Whitton, M. C. Managing Collaboration in the 
nanoManipulator. In Proc. IEEE VR 2003, IEEE 
Computer Society (2003), 180. 
2. Welch, G., Sonnenwald, D.H., Mayer-Patel, K., Yang, 
R., State, A., Towles, H., Cairns, B., and Fuchs, H. 
Remote 3D Medical Consultation. In Proc. IEEE/ 
CreateNet Int. Workshop on “Telemedicine over 
broadband and wireless networks”, BROADMED 
(2005). 
 


Enabling Distributed Collaboration among Heterogeneous Devices

http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=6196C223B693F9ABE464BF04C6E9C702?doi=10.1.1.1075.8716&rep=rep1&type=pdf

Enabling Distributed Collaboration among Heterogeneous Devices

Abstract

Similar works

Full text

Available Versions

CiteSeerX