APPLICATIONS

Metadata Storage

The performance of many applications is dependent on rapidly finding, opening and saving many files or objects. These activities require many metadata operations. Where metadata is centralized, the performance of the metadata storage can be the performance limiter of the whole system.

Filesystems that use metadata servers include: Lustre, PVFS and many vendor-specific systems. Other uses of metadata servers include databases and data warehousing application such as ObjectivityDB.

Violin memory appliances deliver high performance Metadata storage. Low latency and high IOPS enable high performance metadata processing and can increase the throughput and scalability of the entire system.

Metadata servers that use disk storage are limited by the 5+ms access delays and 250 IOPS limits of highest performing drives. Where the metadata is stored in memory, much larger and more expensive servers are required. The cost and complexity of making these systems reliable is also very high.

Violin provides a third alternative with either low latency DRAM or higher capacity Flash storage. Violin’s low latency and high random READ and WRITE IOPS provides a significant competitive advantage over other solid state solutions.

 

Are your file access rates limited by metadata access?

Large file systems are structured to enable simpler administration. This structuring requires many layers of folders and directories. This metadata, as well as information on each file, is large, critical and very latency sensitive. Two common problems with metadata storage include:

Read latency:
Reading a small file, may require many metadata accesses. Where this metadata is on disk, the access delays can be hundreds of milliseconds.

Write IOPS:
File systems may require records of file accesses and changes. These changes must be made in a way which allows a server to fail and hence requires Writes to a trusted storage system.