9,962 research outputs found

    Creating Three-Sixty Objects for Marist Archives Using Open Source JQuery Reel

    Get PDF
    [Excerpt] In this article I will look into some of the recent 3D digitization projects and highlight numerous technological and financial challenges that would be familiar to many small and medium sized archives. I will further propose jQuery Reel, an open source three-sixty player as a viable alternative to 3D modeling and discuss the workflow we employed at Marist Archives

    Environment for Large Data Processing and Visualization Using MongoDB

    Get PDF
    Data means treasures to both scientists and business people. Scientists can discover significant rules and theories beneath data. People involved in business can find their potential customers and improvement suggestions from exploring data. To uncover these valuable treasures, we need effective and efficient tools. There are some traditional quality data management, processing, and visualization tools and systems. However, most of them are no longer as powerful as before when the data size grows larger. These tools and systems may respond slower or even stop working because of increasingly large data volumes. We are now in the digital era. Data is generated at an amazingly high speed. Remarkably, 90% of the data in the whole world has been generated in the last two years.  To manage, process, and visualize large data, we need new tools and systems.In this thesis, we introduce the ELDP&V system. It is designed to manage, process, and visualize large data. We used MongoDB to record file paths and stored files in a filesystem to manage data. By using this method, we do not need to create different schemas for different files. The system offers users basic data processing methods, such as distribution frequency histograms. It also provides scientific models for users for data processing. ELDP&V can visualize data with line charts, bar charts, and 2D maps. Most of the visualization results are interactive. We implemented ELDP&V with some new ideas and design workflows, which led to higher performance. For example, in a test we performed the system has visualized 13,455,368 records in 50 seconds. We tested the same dataset with some traditional web-based tools. They stopped working or produced some errors. When ELDP&V visualizes a time-based file, it displays the chosen items based on timestamps. This means the loading time will not be affected by the file size.The thesis presents background on big data, outlines ELDP&V’s goals and characteristics, the proposed system’s design, and demonstrates its prototype in action. It also contains a comparison with related work and provides several pointers to directions of future work

    The OMII Software – Demonstrations and Comparisons between two different deployments for Client-Server Distributed Systems

    No full text
    This paper describes the key elements of the OMII software and the scenarios which OMII software can be deployed to achieve distributed computing in the UK e-Science Community, where two different deployments for Client-Server distributed systems are demonstrated. Scenarios and experiments for each deployment have been described, with its advantages and disadvantages compared and analyzed. We conclude that our first deployment is more relevant for system administrators or developers, and the second deployment is more suitable for users’ perspective which they can send and check job status for hundred job submissions

    Designing Traceability into Big Data Systems

    Full text link
    Providing an appropriate level of accessibility and traceability to data or process elements (so-called Items) in large volumes of data, often Cloud-resident, is an essential requirement in the Big Data era. Enterprise-wide data systems need to be designed from the outset to support usage of such Items across the spectrum of business use rather than from any specific application view. The design philosophy advocated in this paper is to drive the design process using a so-called description-driven approach which enriches models with meta-data and description and focuses the design process on Item re-use, thereby promoting traceability. Details are given of the description-driven design of big data systems at CERN, in health informatics and in business process management. Evidence is presented that the approach leads to design simplicity and consequent ease of management thanks to loose typing and the adoption of a unified approach to Item management and usage.Comment: 10 pages; 6 figures in Proceedings of the 5th Annual International Conference on ICT: Big Data, Cloud and Security (ICT-BDCS 2015), Singapore July 2015. arXiv admin note: text overlap with arXiv:1402.5764, arXiv:1402.575

    User Applications Driven by the Community Contribution Framework MPContribs in the Materials Project

    Full text link
    This work discusses how the MPContribs framework in the Materials Project (MP) allows user-contributed data to be shown and analyzed alongside the core MP database. The Materials Project is a searchable database of electronic structure properties of over 65,000 bulk solid materials that is accessible through a web-based science-gateway. We describe the motivation for enabling user contributions to the materials data and present the framework's features and challenges in the context of two real applications. These use-cases illustrate how scientific collaborations can build applications with their own "user-contributed" data using MPContribs. The Nanoporous Materials Explorer application provides a unique search interface to a novel dataset of hundreds of thousands of materials, each with tables of user-contributed values related to material adsorption and density at varying temperature and pressure. The Unified Theoretical and Experimental x-ray Spectroscopy application discusses a full workflow for the association, dissemination and combined analyses of experimental data from the Advanced Light Source with MP's theoretical core data, using MPContribs tools for data formatting, management and exploration. The capabilities being developed for these collaborations are serving as the model for how new materials data can be incorporated into the Materials Project website with minimal staff overhead while giving powerful tools for data search and display to the user community.Comment: 12 pages, 5 figures, Proceedings of 10th Gateway Computing Environments Workshop (2015), to be published in "Concurrency in Computation: Practice and Experience
    • 

    corecore