9,962 research outputs found
Creating Three-Sixty Objects for Marist Archives Using Open Source JQuery Reel
[Excerpt] In this article I will look into some of the recent 3D digitization projects and highlight numerous technological and financial challenges that would be familiar to many small and medium sized archives. I will further propose jQuery Reel, an open source three-sixty player as a viable alternative to 3D modeling and discuss the workflow we employed at Marist Archives
Environment for Large Data Processing and Visualization Using MongoDB
Data means treasures to both scientists and business people. Scientists can discover significant rules and theories beneath data. People involved in business can find their potential customers and improvement suggestions from exploring data. To uncover these valuable treasures, we need effective and efficient tools. There are some traditional quality data management, processing, and visualization tools and systems. However, most of them are no longer as powerful as before when the data size grows larger. These tools and systems may respond slower or even stop working because of increasingly large data volumes. We are now in the digital era. Data is generated at an amazingly high speed. Remarkably, 90% of the data in the whole world has been generated in the last two years. Â To manage, process, and visualize large data, we need new tools and systems.In this thesis, we introduce the ELDP&V system. It is designed to manage, process, and visualize large data. We used MongoDB to record file paths and stored files in a filesystem to manage data. By using this method, we do not need to create different schemas for different files. The system offers users basic data processing methods, such as distribution frequency histograms. It also provides scientific models for users for data processing. ELDP&V can visualize data with line charts, bar charts, and 2D maps. Most of the visualization results are interactive. We implemented ELDP&V with some new ideas and design workflows, which led to higher performance. For example, in a test we performed the system has visualized 13,455,368 records in 50 seconds. We tested the same dataset with some traditional web-based tools. They stopped working or produced some errors. When ELDP&V visualizes a time-based file, it displays the chosen items based on timestamps. This means the loading time will not be affected by the file size.The thesis presents background on big data, outlines ELDP&Vâs goals and characteristics, the proposed systemâs design, and demonstrates its prototype in action. It also contains a comparison with related work and provides several pointers to directions of future work
The OMII Software â Demonstrations and Comparisons between two different deployments for Client-Server Distributed Systems
This paper describes the key elements of the OMII software and the scenarios which OMII software can be deployed to achieve distributed computing in the UK e-Science Community, where two different deployments for Client-Server distributed systems are demonstrated. Scenarios and experiments for each deployment have been described, with its advantages and disadvantages compared and analyzed. We conclude that our first deployment is more relevant for system administrators or developers, and the second deployment is more suitable for usersâ perspective which they can send and check job status for hundred job submissions
Designing Traceability into Big Data Systems
Providing an appropriate level of accessibility and traceability to data or
process elements (so-called Items) in large volumes of data, often
Cloud-resident, is an essential requirement in the Big Data era.
Enterprise-wide data systems need to be designed from the outset to support
usage of such Items across the spectrum of business use rather than from any
specific application view. The design philosophy advocated in this paper is to
drive the design process using a so-called description-driven approach which
enriches models with meta-data and description and focuses the design process
on Item re-use, thereby promoting traceability. Details are given of the
description-driven design of big data systems at CERN, in health informatics
and in business process management. Evidence is presented that the approach
leads to design simplicity and consequent ease of management thanks to loose
typing and the adoption of a unified approach to Item management and usage.Comment: 10 pages; 6 figures in Proceedings of the 5th Annual International
Conference on ICT: Big Data, Cloud and Security (ICT-BDCS 2015), Singapore
July 2015. arXiv admin note: text overlap with arXiv:1402.5764,
arXiv:1402.575
User Applications Driven by the Community Contribution Framework MPContribs in the Materials Project
This work discusses how the MPContribs framework in the Materials Project
(MP) allows user-contributed data to be shown and analyzed alongside the core
MP database. The Materials Project is a searchable database of electronic
structure properties of over 65,000 bulk solid materials that is accessible
through a web-based science-gateway. We describe the motivation for enabling
user contributions to the materials data and present the framework's features
and challenges in the context of two real applications. These use-cases
illustrate how scientific collaborations can build applications with their own
"user-contributed" data using MPContribs. The Nanoporous Materials Explorer
application provides a unique search interface to a novel dataset of hundreds
of thousands of materials, each with tables of user-contributed values related
to material adsorption and density at varying temperature and pressure. The
Unified Theoretical and Experimental x-ray Spectroscopy application discusses a
full workflow for the association, dissemination and combined analyses of
experimental data from the Advanced Light Source with MP's theoretical core
data, using MPContribs tools for data formatting, management and exploration.
The capabilities being developed for these collaborations are serving as the
model for how new materials data can be incorporated into the Materials Project
website with minimal staff overhead while giving powerful tools for data search
and display to the user community.Comment: 12 pages, 5 figures, Proceedings of 10th Gateway Computing
Environments Workshop (2015), to be published in "Concurrency in Computation:
Practice and Experience
- âŠ