edit · history · print

HDFS-based Storage Element

When using HDFS-based SE one needs to correctly report certain information into the BDII infosystem. There are many ways for doing that, e.g. OSG has produced a python library that can provide GIP data. It requires some modifications to be used in gLite-environment, plus it comes with much more functionality than we actually needed.

So we went with our own "bicycle".

Static LDIF data

The basic description of the HDFS SE incl. access protocols is described in static-file-hdfsSE.ldif.

BDII plugin

To get updated values for the dynamic fields BDII is using plugins. In our case these are only the values for the amount of available/free/used space. We are currently not enforcing any limits in the VOs, so each VO can use all available space.

So, we wrote a small perl script (to be placed in the /opt/glite/etc/gip/plugins) that gets the data using the output of the hadoop dfsadmin command.

edit · history · print
Page last modified on April 19, 2011, at 01:19 PM