Mind the key replication axioms
tl;dr Data consistency is in many cases the key requirement for database migration or replication: if consistency cannot be guaranteed, the (contents of the) target database might be inconsistent compared to the (contents of the) source database. This blog discusses data migration and replication conceptually from the viewpoint of data consistency independent of any specific database migration or replication system or technology and independent of any non-functional aspects like throughput or latency.
Definition of database replication consistency
Database replication in its simplest form is continuously replicating all data from a source database to a target database:
For clarity of the discussion this blog focuses on…
tl;dr Cloud Spanner introduced point-in-time-recovery (PITR) as a supported database feature. This blog demonstrates how to recover a dropped table using PITR using gcloud commands.
Point-in-time-recovery
PITR allows you to recover data at a point in time in the past. Cloud Spanner enables you to configure the data (and schema) retention period — up to 7 days.
If you enable 7 days of retention you can query up to 7 days in the past using stale reads. This enables you to recover part of the database or the whole database for up to 7 days. …
tl;dr For the longest time I managed gcloud commands in documents or text files, especially during prototyping or initial research. From there I copy and paste them one-by-one or in blocks to a terminal or to the Cloud Shell for execution. This is tedious, error prone and very inefficient — so I changed my approach: in this blog I briefly show you how you can combine both in JupyterLab. You can manage as well as execute gcloud commands in a notebook. I list a few interesting use cases as well in addition to some installation caveats that I came across.
Example: Cloud Spanner — from zero to query in 5 commands
…
By Christoph Bussler and Anand Jain
This blog discusses Cloud Spanner table interleaving from a developer’s perspective: what are interleaving tables, when to use this schema design concept, some edge cases and gotchas.
tl;dr: interleaving tables are a query performance optimization tool for specific join query patterns. Interleaving tables are not a data modeling concept even though they appear as such at first glance during schema design. Interleaved indexes fall into the category of performance optimization as well.
Cloud Spanner’s use of distributed persistent data management
Cloud Spanner is Google Cloud’s “fully managed relational database with unlimited scale, strong consistency, and up to 99.999% availability.”
One underlying fundamental design decision to achieve unlimited scale…
Selecting Databases for Multi-cloud Use Cases
Note: this blog has the same content as the series with the same name. With medium.com series being deprecated, I post the same content unchanged as regular blog here.
This series discusses various aspects of multi-cloud database management:
- What is multi-cloud database management?
- What are use cases for managing databases within and across different public and private clouds?
- What types of database management systems are available that satisfy multi-cloud use cases?
- Which deployment architectures support multi-cloud use cases?
… and many more.
☁️ If you are interested, please follow the series as it develops.
A small request …
by Christoph Bussler and Sireesha Pulipati
Introduction
Multi-tenancy is a software architecture pattern in which a single or few instances of an application serve multiple tenants or customers, often hundreds or thousands. This approach is fundamental to cloud computing platforms where the underlying infrastructure is shared among multiple organizations. Basically, multi-tenancy can be thought of as a form of partitioning based on shared computing resources like databases. An analogy is tenants in an apartment building: shared infrastructure, but dedicated tenant space. Multi-tenancy is also the hallmark of most, if not all, software as a service (SaaS) applications.
An example could be an HR SaaS provider implementing its…
A tutorial for setting up a multi-cloud VPN
Introduction
This blog is a tutorial on how to set up a VPN between Google Cloud and AWS, create multi-zone subnetworks in each cloud and test any-to-any connectivity. For part of the setup this blog follows the community tutorial Google Cloud HA VPN interoperability guide for AWS and cites verbatim from it. This blog goes beyond the content of the community tutorial and includes setting up subnetworks as well as perform any-to-any testing of the network setup.
Please note upfront that this is a step-by-step manual setup all the way in case you want to understand and to execute all the…
by Christoph Bussler, Szabolcs Rozsnyai
What is zero downtime database migration and replication?
Zero downtime database migration and replication (Database Migration — Concepts and Principles (Part 1), Database Migration — Concepts and Principles (Part 2)) refers to migrating or replicating data from a source database to a target database without impacting the client’s access of the source database in terms of availability or scalability. A client can continue to operate on the source database while the database migration or replication progresses.
In the general case, database migration migrates all data from the source database to a target database with the goal to retire the source database and make the target database the primary…
(to be more precise: for almost no-one)
tl;dr
Online database migration is an important — if not the most important — approach when migrating between schemas, between databases, or between data centers or clouds. There are many variations to online database migration (aka, zero downtime database migration): a popular one is dual-write and in my opinion a — if not the most — “dangerous” one when data consistency is paramount. …
As the COVID-19 outbreak continues (Visualizing the History of Pandemics) several application development projects and consortia are underway to develop an Immunity passport on mobile devices (“immunity apps”, “immunity certificates”) where the bearer can demonstrate having undergone a successful medical test for antibodies of a virus.
However, in context of COVID-19, there is a rub: having antibodies might not create immunity against COVID-19.
And, most important, there is a whole list of ethical issues that might arise that counter the benefit of an immunity app significantly.
The following resources provide you an overview of various aspects and might help you…
