Data warehouses and modern cloud architectures go together better than chocolate and peanut butter. The “abilities” (scalability, flexibility, reliability, etc.) of the cloud allows one to quickly and efficiently create and grow an enterprise data warehouse for today’s information-hungry organizations. In this post, I will dive into how to leverage the cloud for your data warehouse.
Looking back just 10 years, before the cloud services started to take off, one had to procure all of the technology needed to house their applications and databases. As a former VP of IT, it was my team’s responsibility to research, purchase and set up the storage devices, servers, network devices and software necessary for the applications and databases to run the organization’s tactical and strategic systems. This process often took months. Once we had everything up and running, there was the constant maintenance necessary to keep the infrastructure running and meet our ever-increasing resource demand. All of this does not include the need to address the continuing operations requirements. There was a need to create disaster recovery options and high availability options like active/active processing of applications in remote data centers throughout the country and sometimes the world. Someone had to set up this infrastructure adding to the cost.
As you can see this on-premises (or at a local data center cage) infrastructure took a lot of time and resources to implement and configure. Using today’s cloud infrastructures that which tool weeks and months to complete can be spun up in days or even in hours. The reliability of cloud storage has been calculated at 99.999999999% (eleven 9’s). The reliability of cloud servers is 99.95% and increasing each year. If my server is being taxed by an increase in transactional volume, I can quickly horizontally or vertically scale my servers by clicking a few buttons or by configuring the servers to auto-scale when certain thresholds are crossed. The savings over the on-premises infrastructure is immeasurable, not only in time and cost (labor), but because I no longer impact my customer with a degradation in performance.
As the cloud technology continues to improve I recommend you consider moving your current data warehouses and operational data stores to the cloud and reap the benefits of this mature technology today.