And the answer is yes. You can read a very short blog post I wrote about them in April 2015. It seems now that they changed a few things to be more visible with the help of the Norway government to financially support the project.
The company promotes 3 values with an erasure coding technique based on systematic mode: superior data integrity, very fast data recovery and super efficient storage capacity utilization thanks to 3 products:
- a C-library for Erasure Coding to be embedded with any storage engine. The team claims that that this library delivers 3x faster that classic Reed Solomon.
- a plugin for Ceph that combines erasure coding with error detection and correction. This plugin can replace Jerasure and ISA.
- a data integrity for RAID solutions.
Data Protection method | Storage overhead |
3x Replication | 3x |
"Classic" Erasure Coding | 1.5x |
MemoScale | 1.2x |
Recovery Speed | MB/s |
Erasure Coding | 13 |
MemoScale | 34 |
Data Fragments | Redundancy fragments | Extra Storage Overhead | Reed Solomon Erasure Coding recovery traffic* | MemoScale recovery traffic | MemoScale improvement factor in recovery traffic** |
6 | 3 | 50% | 6x | 2.67x | 2.25 |
8 | 3 | 37.5% | 8x | 3.67x | 2.18 |
10 | 4 | 40% | 10x | 3.85x | 2.60 |
16 | 4 | 25% | 16x | 5.96x | 2.68 |
20 | 4 | 20% | 20x | 7.54x | 2.65 |
** Ratio between the amount of recovery traffic of RS vs. MemoScale. Higher improvement factor means better performance.
It's important to understand that erasure coding introduces some impact, it explains why for demanding platform, the presence of such techniques is limited versus its appearance in secondary storage. Today erasure coding is a must, having the feature doesn't create any advantage but not providing it is a clear drawback. Having said that, there is real differences across different erasure coding techniques.
MemoScale is a super interesting solution with lot of promises and it's good to see some innovation in a sector when during a long time we didn't find real ones meaning that Reed Solomon was the de-facto standard.
As we try to classify erasure coding techniques, it seems that the Mojette Transform invented in France at the university of Nantes is still the best approach followed by MemoScale and ISA-L and then Reed Solomon. The Mojette Transform is available in one of the fastest Scale-Out NAS of the market. Trying to use erasure coding technique for primary storage allows to find immediately the performance impact and the divergence with service commitments.
I invite you to try each of these to realize benefits and how they operate in your environment.
0 commentaires:
Post a Comment