Mastering VM Replication in Proxmox VE with ZFS

Proxmox VE is an open-source server management platform that allows users to manage virtual machines, containers, storage, virtualized networks, and high availability clusters from a single web-based interface. It’s a powerful tool for managing complex server environments.

Data replication is a process that involves copying data from one location to another to ensure that all users share the same level of information. The goal of replication is to maintain data availability and consistency across a distributed system.

Prerequisites

On the Proxmox VE platform, a replication feature is provided that can be used by both virtual machines and containers. It can be scheduled to periodically replicate the latest state of virtual machines and containers to another node. If the original node fails, there will be a replica available on another node.

To use the replication feature provided by PVE, the following conditions must be met:

• You need to have at least two Proxmox nodes

• The source and target nodes must be in the same cluster

• The disk of the virtual machine or container must be stored on ZFS storage

• The source and target nodes need to have ZFS storage with the same name

• The target node needs to have enough storage space

ZFS and storage replication in Proxmox

Proxmox VE uses ZFS for its replication functionality. ZFS is an advanced filesystem that includes features like snapshot management and data integrity verification. Proxmox VE can take ZFS snapshots of virtual machine and container storage volumes and replicate them to another node in the cluster. 

The command-line tool, pvesr, is utilized for managing the Proxmox VE storage replication framework. Storage replication enhances the redundancy of guest machines utilizing local storage while diminishing migration times.

This tool facilitates the replication of guest virtual disks to different nodes, thereby enabling access to guest data on other nodes without the necessity for shared storage. Storage replication employs snapshot technology to reduce the volume of network-transferred data. Thus, after the first complete synchronization, it’s only necessary to transmit the new incremental data. In the event of node failure, your guest can initiate operation on the replicated node.

Each guest can simultaneously replicate to numerous target nodes, however, replication to the same target node cannot occur simultaneously more than once.

To configure storage replication, it is imperative to establish storage on ZFS locally for the virtual machine.

Steps of setting up replication feature in Proxmox

1. Create a ZFS pool of storage

You need to prepare a separate disk and format it.

Access the Graphical User Interface of the Proxmox VE, select the node where you want to create the storage pool, and click on ZFS.

Thereupon, click on Create ZFS. A subsequent pop-up window will surface, presenting the already-formatted disk partitions. Select the partition you wish to utilize, and enter a name.

On other service nodes, perform the same steps as above. Note that when creating ZFS on other nodes, check Add Storage. And the name created must be the same as the name of the first ZFS storage pool created before. Otherwise, an error will be reported during storage copying.

Subsequently in your Datacenter > Storage, select the curated ZFS storage, edit its node to that of the desired ZFS storage node.

2. Incorporating a replication scheme

Select the virtual machine or container intended for replication. On its right, select Replication, then click Add. A pop-up window will appear where you’ll designate the replication target, define the frequency of replication in terms of time interval, as well as the rate before clicking Create.

Consequently, this results in the virtual machine data synchronously replicating within other nodes.

Additionally, if the source virtual machine has taken a snapshot, it will also be replicated during the replication process.

During the replication process, logs will be displayed, but there is a special point to note: the task of replication is not displayed in the log window at the bottom of the main system interface. You need to click into the replication tab of the node or virtual machine to see it.

To check the status of the ongoing replication task or the result of the last replication task, you can select the task and then press the Log button.

3. In conjunction with High Availability

In instances where virtual machine server nodes fail, high availability facilitates the migration of the virtual machine to other operational nodes.

Navigate to Datacenter > HA, and below, incorporate the virtual machine or container you wish to manage with HA. Select the destination node and then click Create. 

To test: You may manually power off the server nodes, monitor whether the HA-managed virtual machine migrates to other nodes.

Comprehensive Data Protection for Proxmox VE

While replication is a powerful tool for maintaining data availability, data backup should also not be ignored. Replication protects against hardware failure and catastrophic events, but it doesn't protect against data corruption or accidental deletion. Therefore, it's important to also have a separate backup system in place.

Vinchin Backup & Recovery is a robust Proxmox VE environment protection solution, which provides advanced backup features, including automatic VM backup, agentless backup, LAN/LAN-Free backup, offsite copy, instant recovery, effective data reduction, cloud archive and etc., strictly following 3-2-1 golden backup architecture to comprehensively secure your data security and integrity in Proxmox VE beyond replication.

Besides, data encryption and anti-ransomware protection offer you dual insurance to protect your Proxmox VE VM backups. You can also simply migrate data from a Proxmox host to another virtual platform and vice versa.

It only takes 4 steps for you to backup Proxmox VE VMs:

1. Select the backup object.

2. Select backup destination.

3. Select backup strategies.

4. Review and submit the job.

Vinchin Backup & Recovery has been selected by thousands of companies and you can also start to use this powerful system with a 60-day full-featured trial to protect your Proxmox environment! Also, contact us and leave your needs, and then you will receive a solution according to your IT environment.

FAQs

1. Q: Can a virtual machine replicate to multiple target nodes simultaneously?

A: Yes, each virtual machine can replicate to numerous target nodes simultaneously, enhancing redundancy. However, simultaneous replication to the same target node is restricted to avoid conflicts.

2. Q: What are the key differences between ZFS and Ceph in Proxmox VE’s storage ecosystem?

A: ZFS is a file system and volume manager, offering snapshot management and data integrity verification. Ceph, on the other hand, is a distributed object storage system, providing scalable and fault-tolerant storage. The choice depends on specific storage needs, with ZFS often used for local storage and Ceph for distributed scenarios.

3. Q: Is Proxmox VE replication a substitute for a comprehensive backup strategy?

A: No, replication in Proxmox VE safeguards against hardware failures, but it doesn't address issues like data corruption or accidental deletion. A separate backup system, such as Vinchin Backup & Recovery, is crucial for comprehensive data protection.

Conclusion

Proxmox VE’s replication feature, driven by ZFS, ensures data consistency and availability across nodes in a cluster. By establishing ZFS storage, configuring replication schemes, and integrating with High Availability, users can fortify their server environments against failures, minimize downtime, and seamlessly manage virtual machines.

You can choose Vinchin Backup & Recovery to easily protect VMs in Proxmox VE. Don’t miss the free trial!