Accessibility navigation


Managing groups of files in a rule oriented data management system (iRODS)

Weise, A., Wan, M., Schroeder, W. and Hasan, A. (2008) Managing groups of files in a rule oriented data management system (iRODS). In: 8th International Conference Computational Science (ICCS 2008), Krakow, Poland, https://doi.org/10.1007/978-3-540-69389-5_37.

Full text not archived in this repository.

It is advisable to refer to the publisher's version if you intend to cite from this work. See Guidance on citing.

To link to this item DOI: 10.1007/978-3-540-69389-5_37

Abstract/Summary

The iRODS system, created by the San Diego Supercomputing Centre, is a rule oriented data management system that allows the user to create sets of rules to define how the data is to be managed. Each rule corresponds to a particular action or operation (such as checksumming a file) and the system is flexible enough to allow the user to create new rules for new types of operations. The iRODS system can interface to any storage system (provided an iRODS driver is built for that system) and relies on its’ metadata catalogue to provide a virtual file-system that can handle files of any size and type. However, some storage systems (such as tape systems) do not handle small files efficiently and prefer small files to be packaged up (or “bundled”) into larger units. We have developed a system that can bundle small data files of any type into larger units - mounted collections. The system can create collection families and contains its’ own extensible metadata, including metadata on which family the collection belongs to. The mounted collection system can work standalone and is being incorporated into the iRODS system to enhance the systems flexibility to handle small files. In this paper we describe the motivation for creating a mounted collection system, its’ architecture and how it has been incorporated into the iRODS system. We describe different technologies used to create the mounted collection system and provide some performance numbers.

Item Type:Conference or Workshop Item (Paper)
Divisions:Faculty of Science
ID Code:14924
Publisher:Springer

University Staff: Request a correction | Centaur Editors: Update this record

Page navigation