Versity, a leader in mass storage, joined the recent IT Press Tour in California a few days ago, and its CEO, Bruce Gilpin, reserved a surprise to the team with the announcement of their S3 gateway.
It is not new for Versity as they used, like a few others such as Qumulo in the past, the MinIO S3 gateway. But MinIO informed their partners and users that this solution flavor will be deprecated around mid 2022. The even pre-announced this in 2020 to help them find an alternative.
Does it exist? Not sure, especially at scale for workloads managed by Versity.
So Versity has decided to develop its own service layer and this is what they announced. It represents a serious answer to facilitate data integration within its archiving platform in addition to other methods.
So what is it?
The Versity Gateway, without the S3 word, is a development from scratch made by Versity with the support of Los Alamos National Laboratory and Pawsey Supercomputing Research Center.
The engineering team chose the Apache 2 open source license model to simplify adoption and integration. The source code is available on GitHub at https://github.com/versity/versitygw/. Written in Go, and recognized as a distributed oriented language, the tool has been designed to scale for high ingestion rate based on a stateless architecture, the best way so far to reach new scalability levels, leveraging the Go Fiber HTTP server framework.
It is layered on top of ScoutFS, the scalable cluster file system developed also by Versity under a GPLv2 license. But it works with any POSIX compliant file storage. So the idea is to ingest with a S3 mode on top of a file system, in order words a file is ingested via S3 then converted back as file on ScoutFS then stored on the archive media target orchestrated by ScoutAM.
Even if the gateway uses a multipart upload feature, it is not clear how it handles the file system layer. I understand so far that a file is entirely ingested with 1 gateway that writes to one ScoutFS node. So my question not answered today is the following: does the gateway send file chunks to different ScoutFS nodes, or only 1, and then a reconciliation is triggered to “rebuild” the file entity as it was before ingestion.
Released and announced June 13, the product is in early release mode and not for production. The company invites to test it, stress it and provides feedback.
This is a great news for the industry as they contribute to solve that need and offer the code. We wait now some case studies illustrating the adoption of this service layer.
0 commentaires:
Post a Comment