As long as we are dealing with a system where our database lives on a server which never fails and the data that database stores remains the same, we don’t need to think about replication. But as soon as we face either of the below problems:
- Server failure due to power outage or network issues
- High traffic of incoming requests from clients
We need to start thinking about replicating our databases. Above mentioned problems can lead to either a degraded performance of our application or in worse case scenarios loss of data.
Consider a scenario where we have our database stored on a single server. With increase in traffic for read/write operations, performance of our database will get affected. We will soon start seeing high latency or timeouts for database requests.
Also a scenario where there is just one node for our database, a node failure can result in losing all the data. Yes, we can have a mechanism to store the data on a log file on a separate node which can be reused to bring up the database node. But this is a time consuming task in which the amount of time to bring up the database node with all the data is going to increase along with the amount of data that we have stored in the logs. During this team we fail to serve client traffic and end up in application outage.
To tackle above issues we rely on replication. Please note that we can rely on replication approach to solve the above problems as long as we can ensure that our data can be stored on one instance of database. Once we are unable to store our data on a single instance, we need to start thinking about partitioning our dataset. Many of the large scale data systems use a combination of partitioning and replication to build a robust data infrastructure.
Most of the mainstream databases provide replication functionality. Cloud service providers even provide sophisticated replication mechanisms for most of the database flavors.
When thinking about replication we need to make a decision on two different categories of configurations:
- Algorithm for performing replication
- Leader-Follower Replication
- Multi Leader Replication
- Leaderless Replication
- Process in which replication is performed
- Synchronous Replication
- Asynchronous Replication
- Combination of both synchronous & asynchronous replication
As part of this blog-series, I am planning to cover all the above categorized algorithms and replication processes in details. I will also present examples of real-life database applications where these algorithms are used and explain why it makes sense for their use case to follow that approach. Stay tuned.