Giorgio Regni, CTO at Scality shares his predictions for the data storage and management landscape in 2024.
Growing data volumes, tightening budgets, skills shortages, complex IT environments, and surging ransomware threats. These data storage and management challenges will be familiar to everyone reading this article. And as we move into 2024, these perpetual issues will continue to be top of mind.
In this context, the past year has seen thought-provoking discussions on topics such as the demise of the hard disk drive (HDD), best practices to protect data from ransomware, the complexity of hybrid and multi-cloud environments, and what skyrocketing use of generative AI means for underlying storage infrastructure. Exploring these themes further, here are our predictions for the data storage and management landscape in 2024.
The untapped potential of unstructured data for AI will emerge
Large language models (LLMs) have grown significantly throughout 2023 as organisations across numerous industries explore their potential. LLMs primarily rely on structured, text-based training data. However, many organisations hold vast volumes of unstructured data, such as images, videos, etc.
In 2024, we expect that this unstructured data will emerge as a valuable resource for AI/ML tooling and in particular for image recognition applications in sectors such as healthcare, transportation, or video surveillance. To make use of this unstructured data and gain insights quickly, organisations will store their unstructured data in ‘lakehouses’ that can scale to petabytes and can feed data to AI/ML tools in the public cloud as well as core and edge locations.
HDDs will survive despite claims of its death
Declarations that HDDs will soon become obsolete is reminiscent of the debates two decades ago around the death of tape storage. Tape media has survived the decades and remains a significant part of today’s data centre. In a similar way, HDDs are likely to live well into the foreseeable future.
Substantial latency benefits, as well as density advancements and a reducing cost per GB, make solid-state drives (SSDs) an attractive proposition. However, it is unlikely that they will replace HDDs. The promise of cost parity for flash SSDs is yet to arrive, and it is not feasible to eliminate this price disadvantage through data reduction techniques for backup and other unstructured data workloads.
As a result, SSDs are an ideal option for read-intensive, latency-sensitive workloads which justify the higher price point, but they are not the best fit in all cases. HDDs are well suited to numerous petabyte-scale unstructured data workloads and will continue to be so over the coming years, in particular for deployments where cost per TB and price/performance play an important role. A combination of flash and HDDs that takes advantage of the benefits of both media for specific workloads will provide the right balance of capacity, performance, durability and cost.
The challenge of hybrid cloud complexity will be met by managed services
Today, enterprises often embrace a multi-cloud approach, incorporating various SaaS and IaaS solutions from different providers. This brings agility and reduces costs, among other benefits – however, it can also result in a high-level of complexity which offsets these advantages. Integrating on-premises and public cloud within one application or workload can result in extremely complex environments, with diverse application deployment models and multiple vendor APIs and orchestration frameworks.
In the coming year, we expect organisations will increasingly rely on managed service providers (MSPs) to help them navigate these challenges. Leveraging their expertise and experience, MSPs will help enterprises simplify their multi-cloud infrastructures and ensure their IT systems deliver business value and ROI.
Data protection solutions will incorporate ransomware detection
Recent years have seen cybersecurity threats and ransomware attacks skyrocket and become increasingly sophisticated, and the IT industry has invested heavily in data protection solutions and approaches. One example is immutability, which has become a cornerstone of data protection and storage solutions. Immutability prevents data from being changed or deleted, ensuring organisations can recover their data. Immutability is especially important for backup data, which is targeted in over 90% of ransomware attacks.
In 2024, we expect to see ransomware protection and detection become a standard part of application and storage deployments. In particular, new features that can scan and detect malicious behaviour will be introduced to data protection and storage solutions.
