In this second to last post in this series, I’ll review my thoughts on the near term future of the storage industry, technology adoption, and challenges faced by the incumbent storage array manufacturers. The next and final post will lightly touch on End User Computing and evolution of application development and delivery. If you have thoughts, comments, or would like to see more on any of what I have published in this series, I welcome all comments / questions / requests. Thanks for reading! – Brett
Storage Forecast:
As they have over the last 3-5 years, enterprise storage requirements will continue to grow at accelerated rates over the next 3-5 years. Information and intelligent access/use of that information is already a critical differentiator for companies in most industry verticals. As the both the amount of and the value of storage requirements grow, the enterprise will be forced to adopt storage management strategies to ensure that data is placed on the correct tier with the correct levels of availability and access if for no other reason that pure cost.
Virtualization of the storage layer will become prevalent in most organizations. Storage array refresh and data migrations are typically the most expensive, most disruptive, and take longer to complete than any other datacenter infrastructure operations. Virtualization of storage and storage array sub-systems will provide three major benefits to organizations
- Non-disruptive storage infrastructure refresh through virtualization enabled migrations
- The ability through either storage array virtualization or through hypervisor storage virtualization to non-disruptively move virtualized storage between both arrays and tiers, placing data on the most cost effective tier in relation to its organizational value.
- The ability to virtualize storage across physical sites or datacenter locations – when combined with network L2 adjacency will allow mobility of active running workload non-disruptively between physical datacenters.
Continued adoption of Flash (SSD/EFD) and dense SAS drives as well as SATA/NL-SAS will drive down cost, power requirements, and floor space requirements for storage. The shift from high-performing Fibre Channel or SAS drives to the combination of mixing EFD and SATA (or Near-Line SAS), it will be critical for storage subsystems or virtual infrastructures to automatically place data on appropriate tiers in order to meet data accessibility and performance requirements.
This previous point will become a problem for traditional storage vendors. Continued enhancements in server virtualization and the overall virtualization of more tier 1 workloads will lead to a blur of technologies between server and storage array. Many vendors (both hypervisor and storage) will look at methods by which inexpensive RAM on servers can be used as an extension of storage system cache to drive down latency between storage IO operations and access to data response time
Additionally, Hypervisor vendors will continue to push for pure commoditization of storage by attempting to move storage access, tiering, and performance management from the storage controller to the hypervisor CPU. The hypervisor vendors will claim more inherent knowledge of Virtual Machine storage requirements at a granular level (performance, backup requirements, replication requirements, clone/snap requirements, etc) and therefore, the hypervisor should control all of those types of functions at the storage level. Expect the storage vendors to provide access to the storage subsystems to allow this granular access and control by the hypervisor in order to stay competitive in the storage enabled space even though this will diminish the value and differentiators of the respective storage array manufacturers.
Even with a drive towards use of EFD and SATA, data growth will still be prohibitive to ensuring the ability to protect and backup all active data on a regular basis. Even with rapid and standard adoption of data deduplication based backup technologies, data growth will continue to outpace the capability to replicate, manage, and backup that data within appropriate time frames.
Organizations which have not adopted an archive strategy or a business-value approach to classification and long term archive of not-frequently-accessed data will see significant increases in both storage and backup expenses. By leaving all data on primary storage, the cost of disk will increase. Additionally, the cost of backup/replication and replicated data protection will also continue to increase from a pure cost perspective as well as the perspective of being able to deliver service level guarantees for access and recovery. Data availability requirements continue to increase, but as the amount of storage increases, most organizations will find that they are no longer able to provide backups in a timely manner to meet availability or recoverability SLAs
In order to reduce primary storage and backup system costs and to streamline operational backup capabilities, organizations will continue to invest in and deploy long term archive strategies for data that is no longer frequently accessed, must be retained permanently for compliance, or can more effectively be served by deployment on an archive tier.
Lastly, there will continue to be a focus by the enterprise storage vendors to bring application intelligence into the array. As data quantities grow, the ability for organizations to “mine that data” and make use of it for both traditional and non-traditional database management systems will be critical to unlocking the power of that data. Over time the ability of the traditional model of cache-based storage arrays will no longer be sufficient to support data analysis and response times required to support real-time, on-line data access requirements. Because of this, continued focus and adoption of “purpose built” storage array sub-systems, many with key integration directly into server platform technology, will become critical and will shift focus away from traditional approach of providing shared storage capacity.