Last week VDC was host to Compuverde, a Swedish company with a compelling set of software products designed for the distributed storage market. In the briefing provided to VDC by CEO Stefan Bernbo and Executive Chairman Mikael Blomqvist, they highlighted several features of the products and claimed that the performance of the Compuverde solution was significantly better than at least two market competitors. As we are actively covering the Embedded Cloudas well as Rich Media/Big Data markets, we were very enthusiastic about the presentation.
One of the most interesting claims is that the Compuverde solution could reduce client storage system energy use by up to 50%. As this is a true differentiating “Green” factor, the company’s name is well chosen. In order to support their performance claims, Compuverde cited a research paper by Sogand Shirinbab, Lars Lundberg, and David Erman from the School of Computing at the Blekinge Institute of Technology (BIT) in Sweden. As part of the research, extensive testing was done to compare the Compuverde distributed storage system solution with Gluster as well as OpenStack’s Swift solutions.
Compuverde’s solution consists of Gateway and Object Store software products. These can be installed on lower-cost off-the-shelf computing and storage hardware platforms and these platforms can be regionally distributed to increase global performance and reliability. By embedding some high speed SSD storage cache at the edge gateways of the network Compuverde increases the throughput of the storage system. The testing done by the (BIT) team included opening new files, as well as read, write, and delete operations using randomly generated files of sizes from 10KB to 10MB, and creating various loads simulating from 2 up to 256 simultaneous clients. One last test looked at the time needed to replicate a failed storage node. The test structure was designed as follows:
The BIT research team found that Compuverde often had 2X to 3X the throughput as compared to the competing solutions. One of the most interesting things was that the replication time for Compuverde was from 19-22 minutes, as compared to 447 minutes for the unstructured data test using OpenStack and 1109 minutes for the structured data test comparison with Gluster.
Why this is Significant? Reliability and uptime are important as companies develop private clouds or use a public cloud and as service providers attempt to compete with Amazon and others to provide their own cloud services. Even with multiple layers of redundancy and highly reliable equipment, there can be problems related to a single site as shown by the recent Amazon outage that affected Netflix and Pinterest. Therefore, a regionally distributed solution can provide the needed redundancy and even increase performance if it automatically moves files closest to where they are used most. If a node goes down, the data is already duplicated on other nodes and the repaired node can be brought back. This is why VDC believes that the Compuverde claims are really significant.
Do the Compuverde test results reflect reality? VDC certainly hopes so, but we did have a few concerns. Because the test files were randomly generated and uniform in size for each test set, the tests may have shortchanged some of the advantages that Compuverde achieves by utilizing the SSD gateway cache. On the other hand, all of the tests were done on a laboratory LAN and there would have been minimal network latency. This would not represent the network conditions that could be seen on a regionally distributed configuration. One last concern that the VDC Embedded team had regarding the replication test is that it was done with 5 million 1MB files in the cluster being replicated, and the system was not under load while it rebuilt. This, to us did not seem like a realistic test of conditions that might be seen in a typical private or public cloud service.
Despite our concerns with some of the tests, the VDC Embedded hardware team is very positive about the Compuverde distributed storage solution. We look forward to hearing about successful use cases and news about them in the future.