[san-tech][03422] PLFS, a Parallel Log-structured File System, Version 2.1, 2011-10-13

Date: Fri, 16 Dec 2011 21:02:37 +0900
--------------------------------------------------
[san-tech][03429] Re: PLFS, a Parallel Log-structured File System, Version2..1, 2011-10-13
  Date: Thu, 22 Dec 2011 09:32:30 +0900
> "Los Alamos and EMC collaborate to advance high-performance computing"
>  December 21, 2011
>   http://www.lanl.gov/news/releases/los_alamos_emc_collaborate_to_advance_high_performance_computing.html
--------------------------------------------------

ちょっとマニアックかもしれませんが、PLFS, a Parallel Log-structured File System
の開発が進行しています：

PLFS: A Checkpoint Filesystem for Parallel Applications
  http://institute.lanl.gov/plfs/
"PLFS: A Checkpoint Filesystem for Parallel Applications"
 John Bent, Garth Gibson, Gary Grider, Ben McClelland,
 Paul Nowoczynski, James Nunez, Milo Polte, Meghan Wingate
 SC '09
  http://institute.lanl.gov/plfs/plfs.pdf

Abstract
....
"... we have developed a parallel log-structured file system, PLFS,
 which is positioned between the applications and the underlying
 parallel file system. PLFS remaps an application’s write access
 pattern to be optimized for the underlying file system. Through
 testing on Panasas ActiveScale Storage System and IBM's General
 Parallel File System at Los Alamos National Lab and on Lustre at
 Pittsburgh Supercomputer Center, we have seen that this layer of
 indirection and reorganization can reduce checkpoint time by up to
 several orders of magnitude for several important benchmarks and
 real applications."


PLFS Source Code
  http://sourceforge.net/projects/plfs/
  http://sourceforge.net/projects/plfs/files/

plfs-2.1.tar.gz: 2011-10-13
README.installから 2行：
1) To install PLFS as a FUSE filesystem.
2) To install PLFS as a ROMIO ADIO layer for MPI-IO.

README.posix冒頭：
"The primary goal of PLFS is to support very high bandwidth parallel
 IO for HPC large scale parallel applications especially for N-1
 write workloads."

"A secondary goal is to be as POSIX compliant as possible without
 sacrificing the primary goal.  There are several known cases where
 PLFS is intentionally not POSIX compliant in order to satisfy the
 primary goal:"
(snip)
==================================================
2011/12/17
PLFSは N-1書き込みバンド幅を増加するために、並列ファイルシステムの上で動く
ミドルウェア、元データを PLFSで Log Structuredに再配置して、並列ファイルシステム
への書き出しバンド幅をかせぐ。書き込みバンド幅に対して読み出しは弱いと思われる
ので、例えばチェックポイントファイルとか

"PLFS: A Fast Checkpoint Filesystem"
 John Bent, Garth Gibson, Gary Grider, Ben McClelland, Paul Nowoczynski,
 James Nunez, Milo Polte, Meghan Wingate
   http://www.pdsi-scidac.org/events/PDSW10/resources/posters/plfs-r10.pdf
※Lustreでの比較グラフあり

"PLFS and HDFS: Enabling Parallel Filesystem Semantics In The Cloud"
 Milo Polte, Esteban Molina-Estolan, John Bent, Garth Gibson,
 Carlos Maltzahn, Maya B. Gokhale, Scott Brandt
   http://www.pdsi-scidac.org/events/PDSW10/resources/posters/PLFS-HDFS-svd10.pdf

POSTER SESSION PARTICIPANTS
  http://www.pdsi-scidac.org/events/PDSW10/postersession.html
5th Petascale Data Storage Workshop (PDSW)
Supercomputing '10, November 15, 2010
  http://www.pdsi-scidac.org/events/PDSW10/index.html