Scality & WEKA partner on tiered AI storage design

Scality and WEKA have announced a jointly validated storage configuration for AI and high-performance computing that pairs WEKA's NeuralMesh with a Scality RING object storage tier.

The companies are positioning the combination as a way to keep active data on flash storage while moving larger, less active datasets to object storage. They argue this reduces infrastructure costs by avoiding all-flash systems for data that does not require it.

Tiered architecture

The design keeps new and active data on a NeuralMesh flash tier and tiers other data to Scality RING, which is aimed at large-scale capacity and long-term retention.

NeuralMesh is WEKA's storage system for AI data pipelines. It targets environments that need high throughput and low latency for training and inference, as well as high-performance computing workloads.

Scality RING is an object storage platform positioned as a cost-focused layer for large datasets where capacity, durability, and longevity matter more than flash-level performance.

Connector focus

Central to the announcement is an object connector that links NeuralMesh to Scality RING. Scality describes the connector as lightweight, and WEKA has validated it at scale.

Object storage integration often relies on S3-compatible interfaces. Scality says internal testing showed its connector with RING outperforming conventional S3 interfaces, reporting up to 10x faster performance on similar hardware and up to 20% lower infrastructure costs.

Those figures are based on Scality testing, not an independent benchmark. The companies did not disclose the test methodology, the configuration used, or which S3 implementations were used for comparison.

The vendors also say the connector can reduce operational overhead when building tiered storage by avoiding engineering changes in customer environments.

AI pipeline economics

The partnership targets organisations running large AI pipelines that want to manage costs as datasets grow. Training data, model artefacts, and derived datasets can expand quickly, while GPU clusters, networking, and storage systems add further cost pressure.

A key driver for tiered storage is the mismatch between the performance needs of active workloads and the cost of keeping all data on high-speed media. The companies frame the integration as a way to keep performance where needed while expanding capacity elsewhere.

WEKA also highlighted GPU utilisation and AI user experience, referencing time to first token-a metric in generative AI that measures the delay between a request and the first output token produced by a model.

"WEKA's NeuralMesh storage system delivers the high-performance software foundation modern AI pipelines require to run optimally. Enterprises that leverage the Scality RING lightweight connector together with NeuralMesh can achieve additional economic benefits, leveraging a cost-efficient object tier," said Nilesh Patel, WEKA Chief Strategy Officer.

"This enables our mutual customers to achieve their AI project outcomes faster and more efficiently," Patel added.

Durability and scale

Scality says RING scales to exabytes and offers "up to 14 nines durability". In object storage, durability claims typically describe the probability of data loss over time, based on redundancy and erasure coding schemes.

The companies also positioned the approach against Ceph, an open-source storage platform widely used for object and block storage, describing their integration as an alternative to "community-driven object stores" and pointing to support and manageability.

Scality Chief Product Officer Erwan Girard described the partnership as a balance between performance and capacity. "Our partnership with WEKA gives enterprises a smarter way to extend the economics of their AI pipelines," Girard said.

"WEKA drives the performance; Scality provides the scale. Together, we help customers reduce infrastructure costs and management complexity while keeping their AI and HPC environments running at peak efficiency," Girard said.

Availability

The Scality RING with NeuralMesh Connector for WEKA is available now. The companies are pitching the combination to enterprises building AI and HPC platforms that need flash performance and object storage capacity across the data lifecycle.