Cloud Storage; An Introduction

 

Alongside compute and network, storage is one of the fundamental resources required in today's technological systems and software development. Regardless if your systems are on-premises, in the cloud, or both, you always rely on storage components to persists your data. However, if you are used to the world of data centers, like Google Cloud Storage - with SAN, NAS, and local hard drives - and are starting to venture into public cloud, the cloud concept of cloud-based storage and the different services available for it might prove tricky.

Google Cloud provides three main services for different types of storage: Persistent Disks: for block storage, Firestore for network file storage, and Cloud Storage for object storage. These services are at the core of the platform and act as building blocks for the majority of the Google Cloud services and, by extension, to the systems you build on top of it.

Since NetApp Cloud Volumes ONTAP is now available in the Google Cloud, NetApp users can easily take advantage of this grown cloud platform. Let's take a closer look at each of these Google Cloud storage services, what they were designed for, and what use cases they are each best suited to handle.

Google Cloud Persistent Disks (Block Storage): Block storage is the traditional storage type, both in the cloud and in on-premises systems. A Google Cloud Persistent Disk provides block storage and it is used by all virtual machines in Google Cloud (Google Cloud Compute Engine). The easiest way to understand it is by imaging those Persistent Disks as mere USB drives. They can be attached or detached from virtual machines and enable you to build, as the name suggests, data persistence for your services whenever virtual machines are started, stopped, or terminated.

Very much like a virtual disk in your local machine, a Google Cloud Persistent Disk can either be HDD or SSD, the latter for high I/O performance. In addition, there is also the ability to choose where they are located and what type of availability is needed: they can be Regional, Zonal, or Local. While local disks are not, in theory, part of the Google Cloud Persistent Disk service, it is important to mention them. These local disks are only available in the hardware where the virtual machine is running and, while providing the best I/O performance, they are not often recommended due to the low availability and redundancy. On the other hand, if you require high availability, Regional disks will offer you that out-of-the-box, with your disks being replicated behind the scenes in different zones within a region. A more moderate (and less-expensive) approach are Zonal disks, which are storage disks that are also highly available, but only within a single zone.

Other lesser known, yet great features of Google Cloud Persistent Disks are automatic encryption, flexibility to resize while-in-use, and a snapshot capability which can be used for both backup and virtual machine image creation.

What are the different type of data we need store?

1. Structured Data [Name, Phone number etc which can be categorized and related]

2. Un-structured Data[Photo, Video, Static Files like pdf, docx etc]

For Structured Data there are different databases available:

1. Google Cloud SQL: This is fully managed service and offer MySQL, Postgresql and MS SQL Server. In short its a relational database service.

2. Cloud Spanner: Horizontal scaling capability with and this is fully managed, relational database service for regional and global application data.

3. Cloud Bigtable: NoSQL wide-column database for storing big data with low latency.

4. Firestore: NoSQL document database for mobile and web application data.

5. Firestore Realtime Database: NoSQL cloud database for storing and syncing data in real time.

6. Memory Store: In-memory data store service for Redis & Memcache for fast data processing.

7. BigQuery: A fully managed serverless Enterprise grade data warehouse. It a petabyte scale data warehouse. It has built in capabilities of Machine Learning and GIS.