Store data in HDFS#

Hadoop cluster#

Beagle hadoop clusters has 8 computers (node)

Hdfs health

Hdfshealth

Access HDFS#

Using Hadoop web explorer#

Hadoop web explorer

Using commandline#

Remote access a node in hadoop cluster over ssh#

ssh beagle@165.132.137.241

Access file#

hadoop fs -ls /beagle

folder beagle

Using http api#

  • http://165.132.137.159:8080/api/api/hdfs?hdfsPath=<<hdfs_url>>

Beagle project structure#

  • Each beagle project is located in a folder in HDFS

  • a beagle project has some folders

beagle project structure

  • 360_photos: panorama images

  • _origional: contains uploaded data from web

  • bim: ifc, fbx, …

  • data: point cloud data

  • index: index of point cloud data

  • meta: meta data of point cloud data

  • operation: contains output of other map-reduce job (for example: change detection, RANSAC,…)

  • sample: sample data of point cloud

  • tiling: LOD PCD data