“Data is the new oil,” an expression coined around a decade ago by British data science mathematician and entrepreneur, Clive Humby, seems to become more relevant as we usher in an era marked by rapid global digitization and digital services. How do we deal with this new currency of the modern digital economy? One essential part of this is data storage. Google Cloud is leading the way with cloud service offerings, data storage services and customer use cases to help you develop a sound data storage management strategy. This resource outlines the various Google Cloud storage services your organization can leverage.
Google Cloud Firestore1
Google Cloud Firestore is a serverless, real-time, highly scalable, NoSQL document-based database that is adept at syncing, storing and querying data and can be used for web-based, mobile or IoT apps. It provides built-in security, auto scaling and multi-region replication. In addition, it gives software development kits (SDKs) for mobile apps, and has server-side components with robust client libraries that directly connect your app to data, accompanied with built-in offline support.
Data is organized in collections and documents and can be accessed via performant queries and by using a particular database entry path. Every document needs to belong to a collection and can have sub-collections that belong to a document, along with a generous billing structure where you pay based on reads and writes.
Google Cloud Bigtable2
Google Cloud Bigtable is a fully managed NoSQL big data service that is highly scalable and best suited for analyzing huge workloads. Bigtable powers substantial services like Google Maps and Gmail behind the scenes and distributes data to drive performance on massive datasets. It is also designed as a sparsely populated database3 that can scale to thousands of columns and billions of rows, making it ideal to be used while dealing with terabytes or petabytes of data. As a big data database service, it also integrates well with existing big data tools like Google Cloud Hadoop. This data service can also easily accommodate an increase in requests and is best suited for containing structured or semi-structured data like financial, reporting, marketing data, or data needed to run machine learning models. Google Cloud Bigtable can be set up using Google Cloud, the web console, or the API.
Google Cloud Storage4
Google Cloud Storage (GCS) is the object storage service that can be used for large scale data processing and provides reliable, scalable and consistent data storage. Contrary to a traditional file storage system, this is an object storage service that stores data in the form of an arbitrary sequence of bytes addressed by a unique key for object-based storage services in the form of an URL and can easily be used with other web-based technologies. Object storage utilizes grouping data in unique namespaces, popularly called “buckets.” Enterprises can primarily use GCS to store large binary objects, data to be served to websites, content data, historical data for compliance and as a data archive.
Storage Classes in Google Cloud
Google Cloud buckets have various storage classes:
- Standard: This is the most common choice and corresponds to the bucket being in a specific Google Cloud region or stored across multiple regions. Typically, multi-region is opted when the data is very frequently accessed and needs to be geo-redundant. In contrast, regional is opted for frequent data access in a specific region and is relatively less redundant. This storage class is best suited for data that needs to be highly available and quite performant.
- Nearline: This choice is typically used if data must be accessed less than once per month. This is a low-cost option but is a highly durable option for monthly reports or similar scenarios.
- Coldline: This is like Nearline storage but is used for data typically accessed once per year or less frequently. This is a very low-cost, yet highly durable service commonly used for archival, backup and storing data for compliance purposes with a minimum 90-day storage duration and relatively higher costs per operation.
Google Cloud SQL5
Google Cloud SQL is a fully managed, easy to use relational database management system (RDBMS) that offers MySQL, Postgres and SQL server as a service on the cloud. Suppose you are in the starting phase of building your company and don’t want to worry about the integrities of applying patches, configuring for replication, backup and updates. Google Cloud SQL can seamlessly integrate with other Google Cloud offerings like App Engine, Compute Engine or Kubernetes Engine.
Cloud SQL provides vertical and horizontal scaling and can be configured either using the cloud console or the Google Cloud command-line interface. This is the ideal RDBMS if you are looking for frequent queries, and fast response time.
Google Cloud Spanner6
Google Cloud Spanner can be leveraged when you need a SQL database for massive scale, to the tune of 1000s of writes per second and 100,000s of reads per second, globally. This is a fully managed, unlimited scale, string consistency RDBMS system on the cloud that supports secondary indexes and provides vital data consistency by employing hardware-assisted time synchronization. It is a globally replicated database that encrypts data at rest and transit and has low maintenance, and a high availability of 99.999 percent offering import and export of data. This database can be set up by the cloud console or the Google Cloud command-line interface and is ideal for large scale projects in domains like healthcare, retail, finance, etc.
Google Cloud BigQuery7
BigQuery is a column comprehensive data warehousing service and an analytical database that indexes data by the column. It is a serverless data warehouse designed to ingest, store and query large amounts of data. BigQuery provides ways to aggregate data from discrete sources and make it available for business processing and can integrate with other third-party tools for data analysis and visualization with ease.
It works with standard SQL and provides client libraries for interacting with it in multiple programming languages and can be set up using the web console, command-line tool, or API calls in corresponding client libraries. Fully managed by Google Cloud, it only charges you for storing, querying and streaming the data.
Choosing the right storage option
|Requirements to consider||Service to consider|
|Do you need a robust and scalable NoSQL database for cloud-native applications?Do you prefer a database that seamlessly integrates with serverless architecture?Do you need a database where you pay as you go?||Google Cloud Firestore|
|Are you dealing with data that is at least 1 TB in size?Do you need a key value pair NoSQL DB that needs to deal with mass storage of data rather than application state data?||Google Cloud Bigtable|
|Do you require high performing, scalable data storage with simple administrative overhead?Do you require an effective solution for storing large volumes of data, but not limited to backup, content storage, archival, compliance, disaster recovery incidents?Do you require encryption for the data at rest as well as in transit?||Google Cloud Storage|
|Do you need a fully managed generic SQL system that encrypts data automatically in rest as well as in transit?Are you looking for a DB that encrypts external connections as well?Do you need an RDBMS on the cloud that scales horizontally and vertically?8||Google Cloud SQL|
|Do you need a highly available RDBMS for massive, large-scale data, which allows ACID updates?Do you need a system that is encrypted to not worry about data corruption?Do you need a database that auto replicates and facilitates online schema changes?||Google Cloud Spanner|
|Do you need a fast, highly scalable and reliable data warehouse for data analytics?Do you need a secure environment, where data is encrypted and protected with IAM support?Are you looking for a disaster proof solution where you can easily revert to a previous state?||Google Cloud BigQuery|
Google Cloud’s dynamic, scalable storage services for various applications, database or business requirements can help your business ensure that your data is properly protected and retrievable for necessary performance.