Huh, Taesang

English

ZENODO

Study of User-Created Datasets for Belle IITaesang Huh, Sunil Ahn, Junhyun KimSupercomputing  CenterKISTI (Korea Institute of Science and Technology Information)Bell II Computing workshop 2011.05.25EMI INFSO-RI-26161125/05/2011 Study of User-created Datasets for Belle II 2INTRODUCTION• Support of the logical grouping of files into user-named or defined "datasets”• Propose schema for User-Created Datasets(UCD) considering of user scenario• Plan to make new AMGA commands and API • AMGA Manager for Belle IIEMI INFSO-RI-2616113Scenario I – Data Access for Official DataStudy of User-created Datasets for Belle II• Make New UCD (1)- User : A- datasetsTier 0 Tier 0Query* from user AA B, C ∙∙∙ .mdst.mdst• Create metadata for queried results(2)- User : B, C ∙∙∙ (group members)- datasets including queries25/05/2011EMI INFSO-RI-2616114Scenario II – Data Access for Non Official DataStudy of User-created Datasets for Belle II• Make new UCD(1)- User : A- datasetsTier 1 or 2• Retrieve Index file from UCD &Create metadata for Index files(2)- User : B, C ∙∙∙ (group members)- datasets including queriesTier 1 or 2Query* from user AAB, C ∙∙∙ .lst.mdst.mdstSnapshot-like25/05/2011EMI INFSO-RI-261611EMI Hot Topic, JSC, FZJ 5Schema I – UCD schemachecksum dataset_nameuseriddatasetiddatequerylfndescriptionUser datasetFile-level MetadataSchema- metadata for index file25/05/2011EMI INFSO-RI-2616116Schema II• Schema for Caching users’ searching resultsAttribute Name Data Type in AMGA Descriptiondataset_name varchar(32) unique dataset nameuserid smallint ID of a user who create datasetdatasetid smallint dataset iddate timestamp date and time of query creationquery varchar(512) query for deriving a list of fileslfn varchar(512) logical file name for index filedescription varchar(1024) description of user datasetStudy of User-created Datasets for Belle IIThe schema of /belle2/User/[$user_dataset]25/05/2011EMI INFSO-RI-261611• Access AMGAØ belle_amga_access ( ... ) - nowØ Gbasf2.py - future• Manipulate UCDØ belle_amga_create_userdatasetØ belle_amga_list_userdatasetØ belle_amga_remove_userdatasetØ belle_amga_add_userdata• various conditions in the same target datasetØ belle_amga_create_userviewØ belle_amga_list_userviewØ belle_amga_remove_userviewØ belle_amga_access_userview7Study of User-created Datasets for Belle IIWhat will AMGA provide?25/05/2011EMI INFSO-RI-2616118AMGA ManagerSample Data Attributes with data typeCollection treeStudy of User-created Datasets for Belle II25/05/2011EMI INFSO-RI-2616119Contribution : AMGA Manager v1.1 for Belle IIStudy of User-created Datasets for Belle IIFunctions AMGA Manager v1.1 AMGA Manager v1.0DataGrid(SB)• Streaming DataGrid• Play/Pause streaming data• Entries’ limitation of an action∴ possible to view a large scale of streaming metadata in real time• One time∴ impossible to view a large scale of metadata in real timeCollection View• Showing quickly collection view∵ ls –s : get only sub-dir( AMGA ver >= 2.1.1)• Showing slowly  collection view∵ ls –l : get both of entries and sub-dir in the case of the target dir with a large scale of entriesFederation • Site Manager• Federation Manager Null• AMGA Manager v1.0 : general-purpose• AMGA Manager v1.1 : Reflecting Belle II requirements(release  and upload to twiki(Belle2) until early July)25/05/2011EMI INFSO-RI-26161110Further Study• Reflect user requirements in both Scenario and Schema• Make new commands(access, create/read/remove user created dataset, read, remove) • AMGA API (insert metadata into AMGA at Basf2)• Produce user guide line to make UCDStudy of User-created Datasets for Belle II25/05/2011EMI is partially funded by the European Commission under Grant Agreement RI-261611Thank you!

Study of User-Created datasets for Belle II

https://zenodo.org/record/6666/files/Study_of_User_created_datasets_for_Belle_II.pdf

Study of User-Created datasets for Belle II

Abstract

Similar works

Full text

Available Versions

ZENODO