Backup of vmware virtual machines. Backup VMWare ESXi virtual machines using MKSBackup. Creating backup copies of virtual machines on the Microsoft Virtual Server platform

If you want to understand in more detail the processes of setting up and ensuring comprehensive security of local and network infrastructure built on the Linux OS, I recommend that you get acquainted with OTUS. The course is not for beginners; you must pass it to enroll.

If you have not yet configured the free hyper-v hypervisor, I recommend that you read my material on this topic -. If you already have it configured and the question arises about how to quickly, conveniently and freely back it up, then I recommend using free utility HV Backup.

HVBackup utility description for hyper-v

The main goal of the HVBackup project was to create a free, simple and effective backup solution virtual machines hyper-v, working both in single mode and in a cluster.

The utility works on all versions of the operating system, starting with Windows Server 2008 onwards, as in graphic mode, and Server Core installations. The utility also supports the free edition of the Hyper-V Server hypervisor.

HVBackup supports app consistent and crash consistent hyper-v backups using VSS built into the system. There are many commercial solutions based on the same method of backing up virtual machines, but among the free ones I know of only this product.

Company codeplex, the manufacturer of the program, claims that it regularly uses HVBackup in its infrastructure, which guarantees high quality work and timely corrections or changes.

The program can be launched from command line, called from Powershell scripts, or be built into any .Net application via the appropriate library.

The operating principle of this utility is quite simple. The program creates a separate zip archive for each virtual machine in the specified directory, which includes all the information related to it.

System requirements:
.Net Framework 3.5

Installing .Net Framework 3.5 on the command line

To install .Net Framework 3.5 you need to run the following commands in Powershell.

Windows Server or Hyper-V Server 2012:

Install-WindowsFeature NET-Framework-Core

Windows Server 2008 or 2008 R2 (not core):

Ocsetup NetFx3

Windows Server (core) or Hyper-V Server 2008 / 2008 R2:

Ocsetup NetFx3-ServerCore

HVBackup usage example

Perform a full VSS backup of all host virtual machines:

HVBackup -a -o d:\vm-backup

Perform a full VSS backup of the list of virtual machines (you must use quotes if the machine names contain spaces). The backup is performed on a remote server:

HVBackup -l "VM1,VM2" -o \\backup-server\wm-backup

Make a full backup of virtual machines, the list of which is contained in a text file (each VM on a separate line):

HVBackup -f list.txt -o d:\wm-backup

Script for automatic backup

If you want to schedule automatic execution backup, then it is most convenient to make a simple bat file for these purposes and schedule its execution. Let's create backup-all.cmd the following content:

Set BCKPATH="\\backup-server\wm-backup" rem net use %BCKPATH% /user: Pushd %BCKPATH% && forfiles.exe -m *.zip -d -7 -c "cmd /c del @path" popd HVBackup.exe -a -o %BCKPATH% 1> log_out.txt 2> log_err.txt

This script will delete all virtual machines before performing a backup zip files older than 7 days in the specified directory. You can edit the paths and parameters to suit your needs.

Now let’s create a scheduled task on the server to periodically perform backups using the script we created earlier:

Schtasks.exe /create /tn HVBackup /tr c:\backup-all.cmd /sc DAILY /ru /rp/st 01:00:00

If you need to delete a task:

Schtasks.exe /delete /tn HVBackup /f

Immediate execution of the task:

Schtasks.exe /run /tn HVBackup

HVBackup Command Line Options

Usage: HVBackup.exe

-b, --backup
Run backup (default).
-r, --restore
Perform recovery.
-f, --file
Text file, containing a list of virtual machines, one on each line.
-l, --list
List of virtual machines for backup.
-a, --all
Backup all virtual machines on the server.
-n, --name
If specified, only the listed virtual machines are backed up.
-g, guide
If specified, a backup of the listed virtual machines is made by GUID.
-o, --output
Required parameter. The final folder for archives. Can be in the form of a local or network path.
--outputformat
Virtual machine archive name format. (0) machine name, (1) machine GUID, and (2) The current date and time. Default: "(0)_(2:yyyyMMddHHmmss).zip"
-s, --singlevss
Make one snapshot for all virtual machines.
--help
Display help for commands:

Conclusion

That's all. The only note is that I did not test the performance drop during backup. I didn't notice any slowdowns in the operation of the virtual machines. But they were all unloaded. The time it takes to perform a backup on each machine is significant. But, of course, it depends on many factors.

Eat good opportunity combine HVBackup and . For example, make archives locally on a server with hyper-v, mount to linux server folder and use rsync to pick up zip files from virtual machines. Or do it right away backup on Linux samba, for example. The link above has an example on this topic.

Online course Linux Security

If you want to understand in more detail the processes of setting up and ensuring the security of local and network infrastructure built on the Linux OS, I recommend that you get acquainted with online course "Linux Security" in OTUS. The training lasts 3 months, after which successful course graduates will be able to undergo interviews with partners. The course program is developed as a series of practical workshops and is aimed at trained students who already have knowledge and experience in Linux administration. What you'll learn:
  • Apply the best global practices and information security standards (Debian, RedHat, MitRE);
  • Use tools to detect and exploit vulnerabilities in the Linux OS;
  • Install and configure network systems detecting and preventing attacks;
  • Work with monitoring and containerization systems based on Docker;
Test yourself on the entrance test and see the program for more details.

Many businesses, as well as home users, are increasingly using virtual machines to perform various tasks and increase the efficiency of their activities. If previously virtual machines were used mainly by enthusiasts, now the quality of desktop and server virtualization platforms has allowed them to be used by professionals on a large scale. The ability to run multiple virtual systems on one physical computer has many advantages, including savings on hardware, simplified maintenance, and reduced energy costs in large data centers. In addition, an important advantage of virtual machines is their easy portability to another physical platform and simple procedure their backup. But just like regular operating systems, virtual environments require high attention to creation backup copies critical data. When running virtual machines in an enterprise production environment, many companies plan entire strategies for archiving and recovering virtual infrastructure after failures, which are called Disaster Recovery.

Many commercial virtualization platform providers offer enterprise users built-in virtual machine backup tools, such as VMware Consolidated Backup (VCB) for the ESX Server platform. However, in the SMB (Small and Medium Business) sector, where the number of virtual machines used is small, there are practically no backup tools provided by the platform manufacturer. As a result, small companies have to involve system administrators to write various scripts, as well as the use of standard operating system utilities that provide archiving and recovery of files and folders with vital data.

Understanding data backup

Along with the virtual infrastructure planning process, it is also necessary to initiate the process of developing a Disaster Recovery Plan. First of all, it is necessary to highlight the most critical elements of the IT infrastructure, which are potentially susceptible to damage from internal and external sources, such as power outages, malfunctions hard drives, virus threat and others. After this, it is necessary to consider the frequency of backup of virtual machines of various categories depending on the degree of criticality. The company's virtual production servers, which operate in full public availability mode, must be archived quite often and regularly and have the ability to be quickly restored in the event of a failure. Internal servers of the organization that do not require such high attention and quick recovery, can be archived less frequently, with longer recovery times. Then you need to determine which storage devices will be used for archiving (IDE or SCSI drives of other servers, SAN devices, etc.).

When choosing the frequency of archiving, it is necessary to take into account that some types of backup have the property of creating quick backups, but slower recovery. Conversely, it is possible to perform longer archiving with shorter recovery times. The following are the main types of data backup that can also be applied to virtual machines:

  • Regular (full) archiving (full backup)
    This type of archiving creates a complete copy of all stored data. The process of creating such a backup copy is quite lengthy, but does not require much recovery time, since it does not require multiple recovery tasks. Full backup resets file and folder archive markers, which are used to determine which files should be backed up. These markers are used to check file states during incremental and differential backups.
  • Incremental backup
    This type of backup involves copying files and folders that have changed since the last backup. Therefore, if you perform two incremental backups in succession and do not change the file between them, it will not be added to the recovery image.
  • Differential backup
    This backup includes all changes that have occurred in files and folders since the last full backup. Accordingly, with two consecutive differential archiving, a file that has not changed between them, but has changed since the last full archiving, will be archived both times.

In order to explain how these types of archiving differ, we will give an example of combining types of backup. When using full and incremental archiving, backup time is significantly reduced, but recovery time is increased. For example, if we did a full backup on Monday and rolled out additional backups every day, and on Friday the system was damaged, we will need to restore the full backup copy of Monday and sequentially all additional copies until Friday, which will take a very long time. Combining full and differential activation, on the contrary, requires more time for archiving, but less time for recovery, since you only need to restore the full archive copy of Monday’s data and roll Friday’s differential archive onto it.

These, of course, are not all types of archiving that can be used when backing up data, but the types listed are some of the most frequently used. Obviously, for servers with high recovery time criticality, it is more advisable to use differential archiving in combination with full archiving rather than incremental archiving. The first is suitable for external servers of the organization, the second - for internal ones, which can tolerate more downtime.

Since, basically, a virtual machine is a folder with files, you can use the built-in backup tools of the host operating system if you are using a virtualization platform on top of the host system, such as, for example, Microsoft Virtual Server or VMware Server. IN Microsoft Windows For these purposes you can use the utility ntbackup. When using bare-metal platforms such as ESX Server or Virtual Iron, you must use the virtualization system vendor's tools or third-party products.

Additionally, virtual machine backup can be done by creating guest images using software such as Acronis True Image. It is also worth noting that there are situations when it is necessary to archive not the entire virtual machine, but some data in the guest system. In this case, when writing batch archiving scripts, you can use mounting utilities virtual disks to the host system. For VMware platforms, this utility is the VMware Disk Mount application.

Backup and recovery of virtual machines on the VMware ESX Server platform

VMware's pilot product, the ESX Server platform, is a key element of virtual infrastructure in an enterprise production environment. Virtual infrastructure VI3 (Virtual Infrastructure 3) is a set of products and tools that allow a fleet of virtual machines to effectively perform their tasks and function smoothly under various conditions, including force majeure factors such as sudden load surges, power outages and equipment damage. One of the most important tools for implementing a Disaster Recovery strategy is VMware Consolidated Backup (VCB). VCB can be used to create backup copies of individual ESX servers, as well as in conjunction with the VMware Virtual Center virtual server fleet management tool. VCB provides the following capabilities:

  • creating backup copies of virtual machines with different types archiving through a special proxy server VCB Proxy Host, which removes the burden of creating backup copies from the company’s production server where the virtual machines are running
  • does not require installation of additional agents on ESX servers
  • provides ample opportunities for integration with third-party backup products; support for various packages is already built into VCB
  • supports file-level archiving for guests Windows systems(you can create archival copies separate files and folders inside the guest system), as well as archiving at the level of virtual machine images for any guest OS

Backing up virtual machines using VCB takes place by creating snapshots of virtual machines without stopping their operation. VCB also supports SANs. If the virtual machines are located on a SAN storage device, the backup procedure is as follows:

Snapshots of virtual machine states created during operation using an agent located on the VCB proxy server are saved on backup media, from where they can then be restored in the event of a failure of a running guest system or equipment damage. In this case, the backup agent has direct access to the LUNs (Logical Unit Number) in the SAN devices. For SANs, VCB supports the Fiber Channel protocol, as well as tape media for storing backup copies. VCB closely leverages the capabilities of VMware Tools running inside the guest system to create backup copies of guest OS data.

VMware Consolidated Backup on this moment supports the following packages (the list includes only officially tested software with the ESX Server product):

  • Symantec Backup Exec 10.0
  • Symantec Backup Exec 10d
  • Veritas Netbackup 5.0
  • Veritas Netbackup 5.0 MP4
  • Veritas Netbackup 5.1
  • Veritas Netbackup 5.1 MP2
  • Veritas Netbackup 5.1 MP3
  • Veritas Netbackup 6.0
  • Tivoli Storage Manager v5.2.1
  • Tivoli Storage Manager v5.2.3
  • Tivoli Storage Manager v5.3
  • EMC Networker v 7.0
  • EMC Networker v 7.1.x
  • EMC Networker v 7.2
  • EMC Networker v 7.3
  • CA BrightStor ARCServe r11
  • CA BrightStor ARCServe r11.1
  • CA BrightStor ARCServe r11.5
  • Commvault Galaxy v5.9
  • Commvault Galaxy v6.1

In general, the work of creating archive copies of virtual machines can be represented as follows:

  1. The backup software runs a pre-archive script that performs the following tasks:
    • makes sure that no read-write operations occur within the guest system into saved folders and files (for Windows guest OSes only)
    • switches the virtual machine to snapshot mode, creates a snapshot of the state of the virtual machine and makes it available to the application using the VCB
    • mounts a snapshot of the virtual machine from the SAN to the proxy server
  2. A backup copy of a snapshot of the virtual machine is created at the image level, or at the level of files and folders of the guest system (full, differential or incremental copy).
  3. The backup software calls a post-backup script that completes the backup (unmounts the virtual machine snapshots from the proxy server and takes the virtual machine out of snapshot mode).

During the backup process, VCB tools use the following virtual infrastructure components:

To summarize, VMware Consolidated Backup is a powerful tool for creating backup copies of virtual machines and allows you to use standard backup software used in your organization to create archival copies of your data.

Backup with Vizioncore esxRanger

Vizioncore's esxRanger product, now controlled by Quest Software, is currently one of the most popular solutions for creating backup copies of virtual machines on the ESX Server platform. esxRanger does not require installation of any additional agents on ESX servers and creates backup copies of virtual machines from a single server or group of servers through integration with the Virtual Center product. The process of creating backup copies takes place on one Windows server, from where archived images of virtual systems can be stored on various devices storage in the organization's production environment.

esxRanger has both a GUI and a command line interface, allowing you to use a regular scheduler Windows tasks to run backup jobs on a schedule, which eliminates the need to write additional scripts. The main window of the esxRanger product is shown below:

By connecting to VMware Virtual Center, if you have the appropriate permissions, you can select individual virtual machines of data center servers for backup. Copied images are automatically compressed during archiving and decompressed during recovery, saving system administrators time.

esxRanger integrates with VMware Consolidated Backup when used in SANs and allows you to create full or differential copies of virtual machines, as well as individual files and folders in Windows guests. Additionally, during the backup process, esxRanger collects various information about backup metrics (such as time spent on backup and recovery), stores it in a database, and allows you to use it to trend your Disaster Recovery strategy. In addition to this, esxRanger has a policy engine that allows you to build a data archiving strategy based on templates and integrate it with other components of the organization's IT infrastructure, minimizing the workload of system administrators.

The backup procedure using esxRanger looks like this:

  1. A virtual machine savepoint is created and saved to the database.
  2. Using the VMware API, virtual disk files are “unlocked” for reading (they are locked by default) and the creation of .REDO files that will store changes to the virtual disks since the save point.
  3. Virtual disk files are compressed.
  4. The compressed files are backed up and the .REDO files are applied to the VMDK files of the virtual machines.
  5. After the changes are applied, the VMDK files are returned to their original locked state.
  6. The system administrator adds comments to the backup copies of the virtual machines, containing instructions in case the virtual machines fail.

Overall, esxRanger is a convenient, reliable, and easy-to-use virtual machine backup tool for Virtual Infrastructure 3 that integrates with VMware Consolidated Backup for use across SANs for companies of any size.

Creating backup copies of virtual machines on the Microsoft Virtual Server platform

Unfortunately, Microsoft, the owner and developer of the server virtualization product Virtual Server 2005, does not provide users with such a powerful backup and recovery tool as VMware Consolidated Backup. For now, Microsoft is focused primarily on developing hypervisor-based virtualization support built into the next version of the Windows Server platform, codenamed Viridian. However, Microsoft is constantly delaying the final release date of Windows Virtualization, and is also cutting back on its announced features, so it is now difficult to say anything definitive about the backup capabilities of the upcoming virtualization platform. With a fairly high probability we can say that there will be built-in support for “live” archiving, but it is not yet clear in what form. Today, backups of virtual machines in Virtual Server can be created “two and a half ways,” including:

  • usage standard means backup operating system images that can be created by agents running inside guest systems, such as Symantec Backup Exec.
  • writing specialized scripts that save the state of the virtual machine, copy its data to backup media and start the virtual machine again
  • the use of volume shadow copy services (Volume Shadow Service, VSS), support for which in Virtual Server appeared quite recently and is not yet supported by manufacturers of data backup systems

In order to archive running virtual machines on the Virtual Server platform, you can use its COM interface by writing a script, for example, using Visual Basic Scripting (vbs). When you back up a virtual machine, you must first put it in Saved State, then copy its files to a specified location, and then run it again. Below is an example of a vbs script that does these necessary steps to copy a single virtual machine. It can be run on a schedule using the standard Windows task scheduler. " backupvm.vbs " author: John Savill " usage: backupvm.vbs Option Explicit On Error Resume Next Dim objFSO, objVirtualServer, objVM, objSaveTask, objVHD " Connecting to a file system object set objFSO=CreateObject("Scripting.FileSystemObject") " Connecting to Virtual Server set objVirtualServer = CreateObject("VirtualServer.Application") " Search for a virtual machine set objVM = objVirtualServer.FindVirtualMachine(WScript.Arguments(0)) " Saving virtual machine state set objSaveTask = objVM.Save " Pause to perform a save operation while not objSaveTask.isComplete WScript.Sleep 1000 wend " Copying virtual disks and UNDO disks for each objVHD in objVM.HardDiskConnections If objFSO.FileExists(objVHD.HardDisk.file) Then "Wscript.Echo objVHD.HardDisk.file & " " & WScript.Arguments(1) objFSO.CopyFile objVHD.HardDisk.file, WScript.Arguments (1) End If If objFSO.FileExists(objVHD.undoHardDisk.file) Then "Wscript.Echo objVHD.undoHardDisk.file & " " & WScript.Arguments(1) objFSO.CopyFile objVHD.undoHardDisk.file, WScript.Arguments(1 ) End If Next " Copying vsv and vmc files objFSO.CopyFile objVM.File, WScript.Arguments(1) objFSO.CopyFile objVM.SavedStateFilePath, WScript.Arguments(1) " Starting a virtual machine objVM.Startup

This script must be used as follows:

C: emp>cscript backupvm.vbs

It should be noted that Microsoft does not officially support this backup process, since the integrity of a virtual machine copied in a saved state may be compromised due to the fact that part of its memory is not saved in the vsv and vhd files in this case.

Using the Volume Shadow Service

VSS support was introduced in the recently released Virtual Server 2005 R2 SP1. Using shadow copy services in Virtual Server involves creating backup copies of running virtual machines by creating images, which should significantly simplify and speed up the backup and recovery procedure. However, it is not enough to software for backup to support VSS, it is also necessary that it also supports the new Virtual Server VSS Writer Service (VS Writer), support for which, at the moment, has not been found in any of the archiving systems. According to Microsoft, backup tools can use VS Writer to back up and restore virtual machines in the following way: they notify Virtual Server that the backup process has begun, Virtual Server responds by creating a snapshot of the virtual machine, and then the copy process begins. At the moment, the NTBackup utility does not support this mechanism either.

Backing up Xen virtual machines

XenSource, which supports the Open-Source Xen project and also distributes the commercial virtualization platform XenEnterprise, does not offer many options for archiving virtual machines on the Xen platform. One of them is shown below using NFS (Network File System) storage devices.

Background information:

  • XenServer host (in the example backup procedure its IP is 192.168.1.10)
  • The computer used as a server for storing archive copies (in the example, its IP is 192.168.1.1)
  • XenVM virtual machine (in the example its IP is 192.168.1.12)

Backup procedure:

  1. Install the NFS server by adding the following line to the /etc/exports file:
    / *(rw,sync,no_root_squash)
  2. On the XenServer host, add the following to the /etc/xen/xmexample1 file:
    kernel /boot/xenkernel
    name = "ExampleDomain"

    Root = /dev/nfs

    Nfs_server = "192.168.1.1"
    nfs_root = "/ip=192.168.1.10:192.168.1.1:192.168.1.1:255.255.255.0:::"

  3. Save a copy of the /etc/fstab file and add the following lines to it:
    192.168.1.1:rootdevice/nfs rw,hard,intr 1 1
    192.168.1.1:swapdevice swap swap defaults 0 0
    192.168.1.1:usrpartition /usr nfs rw,hard,intr 1 1
    192.168.1.1:varpartition /var nfs rw,hard,intr 1 1
    none /dev/pts devpts gid=5,mode=620 0 0
    none /proc proc defaults 0 0
  4. Copy /lib/modules/2.6.16.29-xen from the XenServer host to the backup device
  5. Run the following command on the archive copy server:
    #scp 192.168.1.10:/lib/modules/2.2.16.29-xen /lib/modules/
  6. To activate the console using udev, run the following commands on the backup server:
    mkdir /tmp/dev
    mount --move /dev /tmp/dev
    sbin/MAKEDEV null console zero
    mount --move /tmp/dev /dev
  7. Run the following command to mount the backup device on the Xen host:
    #xm create -c xmexample1
  8. Back up the xenstore-ls file and copy the contents of the file system (excluding the /proc and /sys directories) to another folder:
    #rsync -a -e ssh --exclude="/proc/*" --exclude="/sys/*" 192.168.1.10:/ /backupdir

Conclusion

Drawing up and implementing a backup and disaster recovery plan (Disaster Recover Plan) for the most important servers and workstations of an organization is a necessary component of its activities. Virtual machines, even more than physical ones, require high attention to data archiving, since usually several virtual systems are consolidated on one physical host. Leading virtualization platform vendors are striving to provide powerful and convenient backup capabilities, but so far only VMware has achieved this. The backup strategy can be carried out in two ways: one of the simplest ways is to do this as part of the standard strategy for archiving data in the company’s IT infrastructure, by installing backup agents in guest systems and creating images. Another, more convenient and faster way is to use built-in platform tools such as VMware Consolidated Backup or writing scripts system administrators. In any case, we must never forget that equipment failure or other force majeure circumstances should not significantly affect the company's critical activities.

At the moment, there are several manufacturers of backup storage programs, both paid and free.
We are what free programs either inconvenient to use (complicated installation, constant threat of failure, lack of native interfaces), or they lack the most important backup options.
In this case it is worth purchasing paid program, which, unlike the free one, will be fully functional with all the basic backup functions.
Below is a list of the best backup solutions, according to experts:

    Data Recovery with VCenter Server support

    Veeam BackUp & Replication

These programs are the main backup programs used by most users:

    Data Recovery With support vCenter Server

As has been written in the past, this is the most the right way creating a backup of the machine if you purchased VCenter Server and no longer have the desire or means to deal with this issue. This technology is quite easy to configure, complete guide can be found at the following link:

This solution works both with and without VCenterServer, but there will be no ability to configure backup over time. We'll look at all the main features below when we compare all the products.

    Veeam BackUp & Replication

This product is now quite popular, since the type of licensing of this product (licensed per socket) for server rooms with low-power servers will be extremely beneficial. Below we will look at several server configurations and consider price characteristics. This product also supports the option of instant data recovery after a failure thanks to its vPower technology.

    Also recently it has been releasing tools for backup in virtual environments. In addition, Symantec is the only backup solution that uses V2P technology (converting a virtual environment to physical servers). True, Vcenter has such technology, but not within the framework of backup technology

    • Despite the fact that Acronis tools are widely used in virtual systems, Acronis was initially created as a backup of physical machines, and the creation of special archives developed by the company itself in order to minimize the backup volume. Acronis includes mechanisms for converting machines in environments of all types (V2V, V2P, P2V and P2P).

      Detailed comparison of backup technologies. VMware vs Veeam vs Symantec vs Acronis

      So, we have listed the main backup solutions, now let’s compare them. We will compare by capabilities, licensing, options and estimated cost of products:

      We will consider 2 types of servers:

      We will present comparisons for one and fifty servers (ESX hosts).

      Let's consider the types of licensing for our technologies:

      1. Veeam Backup & Replication is licensed by number physical processors (sockets) host of VMware ESX/ESXi servers;

        Acronis is licensed by number server host VMware ESX/ESXi;

        Symantec is licensed by number server host VMware ESX/ESXi;

      Products selected for comparison:

        Vmware Data Recovery + Vcenter Server;

        Veeam Backup & Replication Enterprise Edition;

        Symantec Backup Exec System Recovery Virtual Edition;

        Acronis Backup & Recovery 10 Advanced Server Virtual Edition;

      Features and FeaturesData Recovery+VCenterVeeamSymantecAcronis
      Data backup + + + +
      Taking Snapshots + + + +
      Backup by time + + + +
      Sending logs by e-mail - + + +
      Rolling back machines to a previous state + + + +
      Centralized management interface + + + +
      Fully compatible with Vmware solutions + + + +
      Deduplication mode 1 + + - 2 - 3
      Incremental backup 4 + + + +
      Configurable parameters for multiple Vcenters in LinkedMode + + + +
      Recovering individual data + + + +
      Volume Shadow Copy Service (VSS) + + + +
      Policy management + - 5 + +
      Combination with vMotion, HA, DRS services + + + +
      Support for storage types (Local, NFS, Share, iSCSI, Fiber Channel, NAS)Local, NFS, Share, iSCSI, Fiber Channel, NASLocal, NFS, Share, iSCSI, Fiber Channel, NAS, SANLocal, NFS, Share, iSCSI, Fiber Channel, NAS, SAN, USB, DASLocal, NFS, Share, iSCSI, Fiber Channel, NAS, SAN, DAS, cloud services
      VCenter requirement + - - -
      Possibility of recovery on another hardware platform 6 - - + +
      Working with SQL databases - + - 7 -
      Working with the Exchange Server - + - 8 -
      Working with Active Directory - + - 9 -
      Ability to convert virtual environments to physical (V2P) - - + +
      Ability to convert physical to virtual (P2V) environments + - + +
      Vcenter availability recommendation + + + +
      Instant recovery after a failure - + + +
      Bare metal recovery function 10 - - + +
      Protecting Template Files - + + -
      Data replication - + - -
      Recovery check 11 - + - -
      Working with multiple versions of ESXThe section goes by the first digit of the version + + +
      OS support Copies the entire machine, no matter what OS it is runningWindows, LinuxSupport most OS
      Platform supportVMware onlyVMware onlyVMware, Microsoft Hyper-V, Citrix Xen, physicalVMware, Microsoft Hyper-V, Citrix Xen, Parallels, physical
      Estimated cost for 1 server, rub.
      2 processors with 4 cores 50 000 60 000 100 000 70 000
      4 processors with 12 cores 50 000 180 000 100 000 70 000
      Estimated cost for 50 servers, rub.
      2 processors with 4 cores 180 000 3 000 000 5 000 000 3 500 000
      4 processors with 12 cores 180 000 9 000 000 5 000 000 3 500 000

        The deduplication mode allows you to save a backup of not the entire machine, but only the data that has been changed since the last backup. This gives us 2 significant advantages:

        • Significant space saving backup storage data;

          Traffic savings when servers are located at long distances from each other (geographical component);

        The function is available with the additional Deduplication Option;

        The function is available with the additional Deduplication option;

        Incremental backup allows you to first back up the entire source directory and then “append” to it those files that have changed since the last backup. This function allows you to backup the machine without putting it into maintenance mode;

        Feature available with additional program Veeam Monitor;

        Symantec Restore Anyware technology allows users to move the system to another computer without having to re-install;

        If the machine’s files are lost, it allows you to create a new VM with the same characteristics and restore the old one to it;

        After creating a backup, this technology checks whether it can pick up the machine immediately after it crashes;

      Data Recovery with vCenter Server support

      This package is very convenient if we do not want to buy third-party products, and can be put into operation at any level of company development. Among the disadvantages, it is worth highlighting its small functionality compared to other backup systems.

      Veeam BackUp& Replication

      The most popular way to create backups in the VSphere environment. It is multifunctional, can perform most functions, although a fair number of additional parameters are options (VeeamOne, VeeamReporter, VeeamMonitor, etc.), which will increase its cost when purchasing the full package. But, nevertheless, the Veeam BackUp & Replication 5 program itself is a complete product used in many companies, both small and large. This program includes 2 modules: creating backup copies and their replication. This product has new technology, which is used as a test one in many backup programs. VeeamBackUp & Replication5 allows you to launch a VM directly from a backup. This technology Veeam calls it vPower. Thanks to this technology, the user receives the following benefits:

        Instant recovery of virtual machines

        Universal Object Recovery for Any Application (U-AIR)

        SureBackup Recoverability Confirmation

      The replication function allows you to create modified blocks every few minutes, which makes it possible, if a machine fails, to immediately switch to a separate replica and restore the machine's functionality. This feature eliminates the need for expensive hardware and products, and creates an alternative to traditional continuous data protection.

        Instant recovery after a failure

        Starting a VM directly from a backup

        Using replication technology, it is possible to create backups every few minutes without reducing performance

        Possibility to choose a path in favor of speed or reliability (RTO & RPO)

      Symantec Backup Exec System Recovery Virtual Edition

      Symantec, unlike Acronis, offers not only the consolidation of servers into a virtual infrastructure, but also the reverse transfer of virtual machines to a physical base.

      Currently, Symantec is releasing a product for VM backup - Symantec BackupExec SystemRecovery VirtualEdition. This product includes the Symantec Management Solution, Standalone Client and Recovery Disk. For file server backup, we only need an SSR license, but for advanced functions when using SQL databases, Exchange servers, etc. we need to purchase agents for these servers. As practice shows, for most companies it is not enough to create one backup data storage, so Symantec SystemRecovery provides the function of creating an external backup copy on FTP server or additional disk storage for enhanced disaster recovery.

      Symantec has a number of its own unique features:

        Possibility of recovery on another hardware platform;

        Ability to convert virtual environments into physical ones (V2P);

        At using USB, as a storage for backup machines, Symantec recognizes it, determines its type and intelligently acts on it, continuing to perform backup jobs;

      Symantec also uses a data compression function when using P2V technology, which allows you to save on traffic when converting machines at a distance (at the end of the conversion, the space occupied by the VM on the disk will be equal to the volume of the physical machine.)

      Symantec agents are created for certain types of servers (SQL, Exchange, DB2, ActiveDirectory, etc.), including operating systems, which allows them to recognize all the features of such servers and not only create a backup of the entire machine, but also support a number of individual characteristics for each of them (the Exchange agent works separately with its mail databases, and the SQL agent restores the database structure almost instantly after a failure)

      Acronis Backup & Recovery 10 Advanced Server Virtual Edition

      Acronis has a product for VM backup - Acronis Backup & Recovery 10 Advanced Server Virtual Edition. The solution uses technology similar to vPower from Veeam - AcronisInstantRestore, which allows you to instantly restore a machine after a failure. Acronis Backup & Recovery 10 AdvancedServer VirtualEdition allows organizations of any size to maximize the economic impact of virtualization by protecting all the virtual machines running on each individual physical server at an affordable, fixed price. Acronis Backup & Recovery 10 AdvancedServer VirtualEdition not only supports VMware, Microsoft Hyper-V ® , Citrix XenServer and Parallels platforms, but also allows unlimited migrations between these platforms. Acronis offers to calculate savings from its program using a budget savings calculator using this program: http://www.acronis.ru/backup-recovery/roi-calculator.html.

      But Acronis’s capabilities don’t stop there. Acronis has included another feature in the Acronis Backup & Recovery 10 AdvancedServer Virtual Edition package: server consolidation to transfer systems from physical to virtual platforms, with a built-in task scheduler. As a result, we have that this program performs 2 main functions:

        Emergency system recovery

        Server Consolidation

      Main advantages compared to other technologies:

        The ability to work both with physical servers and in a virtual environment, which allows you to combine consolidation with high reliability ratings at the initial stages of company development

        Wide range of supported backup storage devices (even optical devices and magnetic tapes)

        Creating an Acronis Secure Zone partition on the same VM server, which allows you to restore the machine in a short time, and this partition will be protected by deduplication mode on another server

        The bare metal recovery function will allow you, in the event of a complete loss of machine files, to create the same machine and restore a snapshot of the previous one on it.

        Supports most virtual platforms.

        Support for most operating systems when installing a backup agent

Handy Backup can use two methods to create VMware backups: internal and external.

Internal method

A copy of Handy Backup is installed on a VMware virtual machine running Windows or Linux. Operating Handy Backup on a virtual machine is no different in principle from using a similar solution on “physical” computers.

External method

Handy Backup runs on a VMware virtual machine server to copy images of specific VMware instances as regular files. Handy Backup uses a special plugin to back up VMware machines and arrays, running in “hot” mode (without stopping the VMware machine).

How to save a VMware virtual machine image

The VMware backup image is copied using a specialized tool. Using the VMware plugin settings, it is also possible to stop the copied VMware machine and then restart it for a “cold” copy.

  1. Open Handy Backup and create a new task by pressing Ctrl+N or selecting a menu item. Select a backup task.
  2. In Step 2, select the plugin " VMware Workstation".

  1. Double-click on the “New Configuration” line to select the VMware access configuration.
  2. In the dialog that opens, make a choice between the modes " Hot" (backup without stopping the machine) and " Enable suspend" (with stopping the virtual machine to obtain its exact image).

  1. Next, select in the dialog the specific machine image to which this configuration will be applied.

  1. Click "OK" and continue creating the task as usual.

The above sequence will stop and then restart VMware virtual machines without any additional intervention.

1. Backing up VMware ESXi virtual machines

Introduction

This document presents various ways and VMware ESXi backup strategies using vSphere and Bacula Enterprise Edition versions 8.0, 8.2 and 8.4. The Bacula Enterprise Edition plugin for VMware virtual machine backup with vSphere gives you the ability to restore a virtual machine to its original state, while file backup at the guest VM level makes it easier to protect mission-critical application data. VMware backup uses a technology called Changed Block Tracking (CBT) to ensure that, in order to create more efficient backups and reduce network load, only those blocks that have changed since the initial full and/or last are sent to the current incremental or differential backup stream. incremental and/or differential backup.

Key Features of VMware Backup

  • Online backup via VADP
  • Creating VSS snapshots inside guest OSes to pause applications
  • Full, differential and incremental VM backup at the image level
  • Restoring a full VM image
  • Restoring vmdk files to an alternative directory
  • Access to VMware storage, both via TCP/IP and via SAN (FC/ISCSI)

VMware Backup Overview

The current version of the plugin for VMware vSphere supports vSphere versions 6.0, 5.5, 5.1, 5.0, 4.1 (at least version 7 of the virtual hardware). This document presents software solutions Bacula Enterprise Edition 8.0 and subsequent versions, which do not apply to earlier versions of the software.

VMware Backup Glossary

This document uses the following terms related to how to make a VMware backup:

  • CBT– technology for tracking changed blocks.
  • Datastore– the name used by VMware to refer to data warehouses.
  • vSphere is a VMware technology for OS virtualization and cloud computing.
  • VDDK is a set of C/C++ libraries that allows you to create and access VMware virtual disks. The VDDK is used in parallel with the vSphere API to write backup and recovery software or similar applications.
  • When using a VMware ESXi server, the virtual machine files are placed in external memory large volume.
  • NBD– network block device. vSphere allows you to access files hosted in the Datastore using direct file access, NBD access, NBD over SSL or SAN access. In case of accessing files via NBD as network protocol TCP/IP protocol is used.
  • SAN. vSphere allows you to access files in your data store using direct access technology. SAN can use Fiber Chanel network (no-boot backup technology local network Lan free backup) or ISCSI over TCP/IP technology.
  • VMware ESX and VMware ESXi are a hypervisor architecture installed on a server without an operating system. The smaller ESXi codebase means a smaller attack surface and smaller size code for the patch, which improves the reliability and security of the system.
  • VCB– Consolidated VM Backup Method An older VMware API that is generally no longer used. The VMware plugin does not use VCB technology.
  • VADP– The next generation of VMware data protection infrastructure, implemented in vSphere 4.0, allowing backup software to create centralized, efficient VMware backups off host machines and without loading the local network.
  • .vmdk -file format, used for virtual appliances designed for VMware products.
  • .bvmdk – internal file format used by the Bacula Enterprise plugin to process sparse blocks and differential/incremental VMware binary backups. Once converted using the vddk tool, the file becomes a raw image of the original disk, which can be converted to vmdk format using the qemu-img utility.
  • ESX 3.x uses virtual hardware version 4, vSphere 4.x uses version 7, and vSphere 5 uses version 8.
  • A fingerprint can be generated from the ESXi host
    openssl x509 -sha1 -in /etc/vmware/ssl/rui.crt \-noout -fingerprint | cut -d ‘=’ -f 2
  • guestfish – A shell and command line tool for viewing and modifying the VM file system.
  • VM (or VM) – an abbreviation for the term "virtual machine".
  • vSphere is a platform for server virtualization with the ability to coordinate virtual data centers.
  • SELinux - Security-Enhanced Linux (SELinux) is a security module in the Linux kernel that provides a mechanism to support access control security policies, including authoritative access control (MAC).

1.1 How to make a VMware backup in a guest OS

1.1.1 Installing the Bacula Client on each guest OS

The first strategy does not involve using a plugin Bacula Enterprise Edition for vSphere. Instead, Bacula Enterprise File Daemon is installed on each VM as if the VMs were regular physical servers. To optimize I/O flows on VMware ESX/ESXi servers, tasks are used Schedule, Priority And Maximum Concurrent Jobs to distribute backup tasks in the backup window. Because all servers share the same set of disks, performing all backup tasks at the same time, it is possible to create bottlenecks in the disk/network subsystem.

Figure 1: Installing bacula-fd on each guest VM

Installing Bacula Enterprise File Daemon on each VM allows you to manage virtual servers, as if they were physical servers, and also use all the features of Bacula Enterprise software, such as:

  • Fast recovery of individual files
  • Checksumming individual files to detect viruses and spyware
  • Checking the task
  • Excluding file/directories (such as swap files and temporary files)
  • File-level compression, etc.

1.1.2 VMware backup using the Bacula Enterprise Edition plugin for vSphere

In the case of a strategy for creating a backup image of a VMware virtual machine, the plugin Bacula Enterprise Edition for vSphere, stores Client disks as raw images in the VMware/vSphere context. In order to implement this strategy, you do not need to install the Bacula File daemon on each guest machine.

The Bacula plugin for vSphere will communicate with the VMware ESXi host to read and store the contents of the VM disks over the NBD or SAN. With direct access to the image vmdk, saved in data store Bacula software will not have to run through the Client's file system to open/read/close files. Accordingly, the software will consume less ESXi infrastructure resources than if a VMware backup was created on each guest machine. At the same time, Bacula software will also read and store useless data such as page files and temporary internet files.

Figure 2: Creating a TCP Backup Using NBD

If the vSphere backup plugin uses the NBD data transport method, the data is streamed to the backup server through the ESXi system's VMkernel port.

The Bacula Enterprise plugin for vSphere can also leverage SAN infrastructure to reduce the load on ESXi servers. However, despite consuming fewer resources on the ESXi server, data will still need to be read from your disks, which may result in conflict when trying to send/receive data at the same time.

When using block differential methods, such as those used by the vSphere plugin, it is necessary to ensure that all incremental backups are available for recovery. If at least one backup task is missing at the time of recovery, the Bacula plugin will not be able to recreate the correct image. The use of differential backups allows you to reduce the number of tasks required for recovery, thereby reducing the risks of possible data loss. To prevent the loss of important tasks for creating incremental backups, storage periods Volume retention must be large enough to recover all data.

1.1.3 Comparison of VMware backup strategies

Table 1. Comparison of backup strategies

The procedure for restoring individual files from a VMware machine backup created using the vSphere plugin is described in Section 2 on page 27.

1.2 Installation

Documentation detailing the installation process is available upon request.

1.2.1 Configuration

The Plugin Directory parameter of the File Daemon utility, stored in /opt/bacula/etc/bacula-fd.conf, should point to where the plugin is installed vsphere-fd.so. Typically, by default the Bacula plugin is installed in the directory: /opt/bacula/plugins

The File daemon utility must have direct access to the vSphere network or access through a SAN. You can check the connection using the telnet program. vSphere network access to ESX or vCenter Server must be configured in /opt/bacula/etc/vsphere_global.conf.

Figure 3. Backup via SAN

Parameter Required Default value Description
Chapter general settings global
keep_generation No 100 Max. number of backups between two full backups.
profile_all_vm No vsphere_all_vm.profile Name internal file, used to store VM profile information.
root_directory No /opt/bacula/working/vsphere vSphere plugin root directory.
vddk_path No /opt/bacula/bin/vddk
Settings section vsphere
username Yes The name of the user who is allowed to connect to vSphere.
password Yes Password for the username that is allowed to connect to vSphere.
hpassword No Hidden password for the username that is allowed to connect to vSphere.
timeout No 60 Timeout to connect to the vSphere server in seconds.
thumbprint Yes SSL fingerprint of the vSphere server certificate.
server Yes vSphere ESXi server used to create backups.
url Yes The address of the vSphere ESXi or vCenter server used for the purpose of making a call using SOAP.
Default_datastore No datastore1 Default recovery data storage.
default_restore_host No ESX server used by default for recovery if multiple servers are available in vCenter.
default_ovf No The default OVF description used in case the current OVF description cannot be loaded into VMWare.
root_directory No /opt/bacula/working/vsphere The directory used to store the plugin's internal data.
datastore_minimum_space No Minimum size for storing data in a data store. For example, 5GB.
datastore_allow_overprovisioning No Yes Allows you to restore VMs using the Over Provisioning function. If the parameter is set to " No", when restoring, you must ensure that all disks are sized to match the Datastore size.
datastore_refresh_interval No 600 The interval used to update data storage statistics in the Datastore.

Table 2. Configuring a vSphere connection using the vsphere_global.conf file

The fingerprint can be obtained using the console screen by pressing F2 and then logging in. The Thumbprint will appear in the window View Support Information under SSL Thumbprint (SHA1). Or you can connect via ssh:

Using multiple vSphere servers

You can specify multiple vsphere servers in the vsphere_global.conf file. When using this function you need to set the server=xxx parameter in command line plugin. Also be sure to specify an alternate directory in case your VM has the same MoRef value.

Please note that the default section is required in the vsphere_global.conf file.

Parameter Required Default value Description Example
host No Guest VM name host=srv1
host_include No The guest VM image that needs to be enabled host_include=srv3
host_exclude No Guest VM image to exclude host_exclude=srv
disk_exclude No List of drives to exclude disk_exclude=0,2,4
keep_cbt No Don't try to activate CBT keep_cbt
quiesce_host Yes Stop the guest VM before taking a snapshot (try, yes, no) quiz_host=no
server No Specify vsphere server server=vsrv2
Debug No Allow Debugging debug
abort_on_error No Stop executing a task after encountering an error
update_timeout No Change the initial update timeout

Table 3. vSphere plugin command parameters

Please take into account the fact that the teams host_include And host_exclude are regular expression Java.

Hide vSphere password

Starting from plugin version 8.0.3 you can hide the vSphere password in a file vsphere_global.conf. Field hidden password called hpassword. To generate a hidden password, you can use the command @encode. Please note that if the string you want to encrypt contains the expression “=”, when writing the command you must use the format string= keyword.

Testing vSphere Configuration

To test the plugin for vSphere, you can use the following command (as root user):

When using the update command vsphere-ctl A list of all VMs that are configured on the ESXi server should appear. If this does not happen, please check that your credentials are configured correctly in the file vsphere_global.conf.

Team list allows you to display information found on ESX hosts and data stores.

Example of using the Job function

When running tasks to create incremental/differential backups, it is necessary to specify the parameter Accurate.

Examples of using the FileSet function

This section presents various options for using the function FileSet. Please note that the vsphere plugin is not compatible with the FileSet function for sparse files.

Figure 4. Backup of the VMware guest1 virtual machine on an ESXi server

Testing the FileSet function

You can use the command estimate to test the FileSet function.

Implementing incremental VMware block-level backups

Please note that the CBT utility is not supported on virtual hardware versions 6 and earlier, or when the virtual disk is connected to a shared virtual SCSI bus.

So that CBT can identify changed disk sectors since last change ID, the following conditions must be met:

  • Host version ESX/ESXi 4.0 and higher.
  • Version 7 (and higher) of the VM hardware that owns the disks whose changes should be monitored.
  • Data I/O operations must be performed through the ESX/ESXi memory element block. NFS is supported as are RDM disks in virtual compatibility mode, but not RDM disks in physical compatibility mode. Also used is the VMFS file system supported by SAN, iSCSI, or local disk.
  • The CBT utility must be activated for the VM (see description below).
  • The VM storage should not (permanently or non-persistently) be represented by an independent disk, that is, one that will not be affected by snapshots.

In order for the CBT utility to be able to determine disk sectors using a full backup, the following conditions must be met:

  • The virtual disk must be located on a VMFS volume supported by a SAN, iSCSI, or local disk.
  • The VM must have a zero number of snapshots (0) when activating CBT to implement the so-called. clean start.

When using “Thick Provisioned Eager Zeroed” drives, VMWare CBT will display all blocks as used during a full backup. For VMs that do not support CBT, the vSphere plugin will always perform a full backup of the virtual disks. To check if the CBT utility has been activated for virtual disk, open the vSphere client, select the command powered-offvirtual machine without snapshots(shut down the VM without creating snapshots).

  • Right-click on the VM and select edit settings Edit Settings.
  • Go to the tab Options.
  • Click on the tab General under the tab Advanced, and then by item Configuration Parameters. The settings configuration dialog box opens.
  • Click on the item Add Row.
  • Add a parameter ctkEnabled and give it a value true.
  • Click on Add Row, add parameter scsi0:0.ctkEnabled and give it a value true.

Attention: line scsi0:0 in the parameter scsi0:0.ctkEnabled points to the SCSI device assigned to hard drive added to the VM. Every HDD, added to the VM, gets its own SCSI device, designated scsi0:0, scsi0:1, or scsi1:1. During the first full VMware backup, the vSphere plugin will attempt to automatically activate the CBT utility when the VM is shut down. To disable this function enter the command keep_cbt on the plugin command line.

Problems when using CBT

If you are reverting to a snapshot older than the last incremental backup, you must create a full VM backup before using incremental backups again. This problem was resolved in vSphere 4.1 and the third update of vSphere 4.0. Instead of potentially providing incomplete data, a change ID number received before reverting to a previous snapshot is now correctly treated as invalid (http://kb.vmware.com/kb/1021607).

Compressing backup size by resetting CBT

Once a block is marked as "used" by the VMWare CBT utility, the system will continually create backup of this specific block when performing a full backup, even if that block will be marked as “free” by the guest OS. After some time, a situation may arise in which a large full VMware backup will be created with a small amount of disk space used.

By recreating the disk using VMotion, the CBT table can be reset to mark only the blocks that are actually used. To perform this operation, you must first clear the guest VM's disk by writing "zero" blocks to cover all free space. Please be aware that the operation will consume resources and must be performed outside of business hours.

On Windows OS, the procedure can be performed using the utility Microsoft delete, available at http://technet.microsoft.com/en-us/sysinternals/bb897443.aspx

On Linux OS you can use the built-in tool dd. Take into account the fact that you may want to limit dd to avoid filling up the entire disk.

Once the operation is complete, you must stop the guest VM. This can be done through the ESXi shell interface as follows:

Information about the location of the disk and configuration file can be found as follows:

After this, the zero blocks of the VMDK files must be cleared through the ESXi shell interface as follows:

Once the operation is complete, you need to deactivate CBT for the guest disks you want to shrink. You can also edit them through the vSphere Management Console or IN AND.

Then you need to power on/off the guest VM to apply the changes to the CBT utility. You can wait until the host is fully up and running.

Now you should not see files like “*-ctk.vmdk” and can re-enable CBT in the host configuration file and start your guest VM.

Files of type “*ctk.vmdk” will be created again. Team estimate bacula plugin should display the files bvmdk smaller size.

Since this procedure is quite complex, we recommend that you try it out through the sandbox first. If the ESXi SSH interface is activated, you can create a script for anything.

Definition of CBT unavailability

If the CBT (Changed Block Tracking) utility is not available for the disk, the file vsphere-ctl*log may contain the following error:

When this error occurs, the vSphere plugin will automatically create a full backup of the disk image. To enable CBT for a specific drive, see section 1.2.1 on page 14.

Activating access via SAN

You may have difficulty setting up SAN access on the host. The VixDiskLib VMWare library is compiled for the Redhat 5 64bit version. On later OSs such as Ubuntu or Redhat 6, you need to compile and install the 1.95.7 library. Please note that the Bacula Enterprise plugin for vSphere contains this library in the package bacula-enterprise-vixdisk.

To use SAN data movement technology, the backup server on which the vsphere plugin is installed must have access to all LUNs exported to the ESX server. Packages like multipathd, will not have problems with devices with different connections. If your drives are visible as /dev/sda, /dev/sdb, ... the vSphere plugin will open each drive to get the UUID and compare it to the one provided by the ESX server. For example, when using iSCSI, the lsscsi command will map the disks as follows:

You can verify that the SAN transfer method is being used by using the debug function debug in the plugin command line and make sure that the file vddk trace contained in the following location:

If the SAN transfer mode is not available, the vSphere plugin will automatically switch to the nbd transfer mode.

Deleting old snapshots

If the VMware system contains snapshots that were not automatically deleted by the vSphere plugin, using the vSphere Plugin version 6.6.3 and higher, you can clean the system using the following commands.

  • Deleting old snapshots and previous unsuccessfully generated snapshots

vsphere-ctl clean-snapshot —snapshot myhost

  • Deleting old snapshots with a name starting with a string

vsphere-ctl clean-snapshot --snapshot-base pluginTest myhost

  • Deleting all snapshots with all derivatives; possibly faster)

vsphere-ctl clean-snapshot --snapshot --snapshot-delete-child myhost

When starting a new backup task, the vSphere plugin will automatically check to see if there were any problems with the previous task and delete any old snapshots if necessary.

Debug trace

The vSphere plugin uses various technologies and third-party libraries. As a result, the system makes extensive use of the trace function. The user will be able to take advantage of the following files:

Table 4. Tracing methods used by the vSphere plugin

To extract a bvmdk file without converting it using vddk during the restore procedure, you need to set the FileDaemon debug level to 1000. During the restore, Bacula may generate incorrect file size reports.

Working files

The vSphere plugin creates special files in working directory. These files are required for the CBT VMWare utility to work. To clear the plugin working directory for vSphere, you can use the command vsphere-ctl:

This will remove 30 days worth of files and directories. This period should correspond to at least the period for creating a full backup, plus a few days for security. When creating a backup, if the plugin cannot find working files when creating the last backup, the vSphere plugin will create a full backup of all disks.

Disk exclusion

To exclude a specific disk from the procedure, you can activate independent mode through the vSphere console, or use the function disk_exclude(see table 1.2.1 on page 11). To find diskid in order to use it in a function disk_exclude, you can use the command estimate listing. 0.bvmdk is the diskid 0 image.

1.3 VMware vSphere Backup and Recovery Procedures

1.3.1 Backup

Figure 5. Excluding a disk from backup


1.3.2 Recovery

Bacula Enterprise software allows you to recover any file (bvmdk, ovf, ...) on local disks. After this you can mount the image locally using the VMWare tool vmware-mount tool or qemu-nbd and perform file-level recovery. When using the parameter where=/path/to/dir In the recovery function, the plugin will automatically restore the selected files to the specified location.

It is also possible to copy the raw image to any device or mount it and restore the files directly.

Recovering to a new guest VM

If you run your VM recovery procedure using the where=/ parameter, and select all files in the directory vm, the plugin for vSphere will try to restore your disks on a new VM created during recovery with the existing attributes (disks, controller, CPU type, ...).

Enhanced SAN mode is not currently supported for recovery. The vSphere plugin uses data transfer via NBD.

The ESX host and datastore that will be used to restore the guest VM will be determined automatically. However, you can change the default destination by changing the plugin's recovery options via the bconsole menu:

Or you can use the BWeb interface (see Figure 6)

Figure 6: Selecting a datastore, ESXi server, or hostname at the time of recovery

Please note that you need to configure at least one VM on your ESX server to automatically recover the VM using Bacula. In the future, we plan to eliminate this limitation.

Starting with Bacula Enterprise 6.2.4, the vSphere plugin supports automatic network topology creation. So, if your ESX host does not provide the correct vSwitch configuration for the VM, the Bacula plugin will have to recreate all network settings during recovery.

Beginning with Bacula Enterprise 8.2.1, the plugin for vSphere can check for available memory in the Datastore during recovery. The user can prevent the reserve area from growing and reserve a minimum amount of memory in the storage. These two parameters can be configured in the file vsphere_global.conf and can be overwritten from the recovery menu.

server = 192.168.0.68

url = https://192.168.0.68/sdk

datastore_minimum_space = 64MB

datastore_refresh_interval = 10

datastore_allow_overprovisioning = false

The “unallocated” amount of memory returned by the vSphere server is not always accurate. The refresh rate can be changed using the method described in the manual at http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2008367

Sometimes Bacula PM fails to load the OVF file describing the guest VM to the vSphere or vCenter server. In particular, this is due to certain VMware limitations such as “you cannot use an OVF that contains references to a mounted CDROM”... The vSphere plugin uses workarounds to resolve issues like this, but it does not solve all problems. If you have similar difficulties, you can use the option default_ovf in file vsphere_global.conf. Typically, you need to configure the parameter default_ovf such that it references an existing simple OVF template. During the recovery process, this template will be used automatically, and you will need to configure the VM later by specifying values ​​such as CPU number, RAM size, etc.

On Windows, in some cases you may need to perform additional tasks after the recovery process has actually completed. For example, if the restored system will not boot, you may need to use the Windows recovery to debug the system. For servers with installed service Active Directory may need to study Microsoft guidelines in order to keep AD databases consistent and in sync with other AD servers. If the installation involves dynamic disks, you must import them into the newly restored system after rebooting. You can import using Disk Manager or using the “diskpart” function by selecting one of the dynamic disks and entering the “import” command.

Recovery without vSphere plugin

If you are trying to recover disks in a File Daemon that does not have the Bacula Enterprise plugin for vSphere installed, you will have to convert the bvmdk files to raw files using the vddk command from the command line:

Format bvmdk used by the vSphere Plugin to ensure data integrity and efficient processing of sparse information by the CBT utility.

1.4 Suspending a guest VM

To properly suspend a guest VM, you must install and update VMware Tools on the Linux/Windows Virtual Machine.

Plugin command quiesce_host=Try/yes/no allows you to control the procedure for stopping guest VMs using vSphere before capturing a snapshot. The default value is try. IN this mode the plugin will try to stop the guest VM while creating the snapshot, and if the snapshot creation fails, the plugin will try to re-create the snapshot without stopping the guest VM. The first attempt will be logged as an error in the task log.

More detailed information You will find the specific error message in the vSphere console log.

Warning message from ESXi: the guest OS has reported an error during quescing. Error code was: 2 the error message was: custom quiesce script failed. (Error message from ESXi: The guest OS reported an error when stopping. Error code 2: Stop script error)

An error occurred while saving the snapshot: Failed to quiesce the virtual Machine (An error occurred while saving the snapshot: The VM cannot be stopped)

1.4.1 Linux

By creating a special script in /usr/sbin/pre-freeze-script, you can stop your system automatically when you create a snapshot using vSphere. vSphere will try to execute the script /usr/sbin/post-thaw-script in case it will be present in the guest OS.

1.4.2 Windows VSS

The plugin enhances Windows protection, creating VSS-based snapshots before backups to stop VSS-enabled applications.

Pre-freeze and post-thaw scripts for VSS. Starting with ESX/ESXi 3.5 U2 and higher, VMware Tools first searches for scripts alphabetically in C:/Program Files/VMware/VMware Tools/backupScripts.d, calling them with an argument freeze, and then in reverse alphabetical order calls with argument thaw(or freezeFail in case of unsuccessful stop).

1.5 Supported platforms

The VSphere plugin supports the following products on the VMware platform:

  • ESX/ESXi versions: 6.0, 5.5, 5.1, 5.0, 4.1

We are currently testing the correct operation of the plugin for VSphere with the following VMware platform products:

  • vCenter Server versions 6.0, 5.5, 5.1, 5.0, 4.1 managing ESX/ESXi 4.1 and later versions
  • VirtualCenter versions 2.5, managing ESX/ESXi 4.1

To manipulate files and snapshots, the VSphere plugin uses the vStorage API. This extension requires a valid non-free VMWare license.

  • The VSphere plugin has been tested (and supported) on the following Linux-based platforms: RHEL 6, 7 (Red Hat Enterprise Linux) 64bitSLES 11 (SUSE Linux Enterprise Server) 64bit

1.6 Limitations

Plugins may not be compatible with default VirtualFull tasks. Please contact Bacula Systems support to ensure you are using optimal settings.

2 Overview of VMware Single File Recovery Procedure

This section provides information on how to use the Single File Recovery feature. VMware by using Bacula Enterprise Edition and a plugin for vSphere.

Brief description of functions


Single File Recovery Tool Bacula Enterprise Edition allows you to use the following functions:

  • Console interface
  • Bweb Management Suite Interface
  • Support for creating full/differential/incremental backups
  • Support Windows 2003 to 2012
  • Linux support (ext3, ext4, btrfs, lvm, xfs)
  • ESX 5.x and 6 support

2.1 Installation

Documentation detailing the installation procedure is available upon request.

2.2 Recovery scripts

This feature allows you to quickly locate and restore specific files from a directory in a VMware environment.

2.2.1 Via the text console interface

Plugin for recovering one file ( VMware single file restore) allows you to use a simple software console that provides access to files inside the VM. The process of restoring one file begins with installing VM backups:

Select the right client first

Then, select the task you want to restore.

Then select the desired VM.

Now select the location of the guest file system (locally or via SMB)

At this stage, the VM file system is mounted locally (in the example above, the files are available at /opt/bacula/working/vmware/5. As is the case with standard file system, you can find directories and copy files (via cp, scp, ftp) from another terminal session using Unix “root” and “bacula” accounts. If you need to use a different Unix account to work with files, use the function “-o allow_other” when running the script mount-vmware.

To clear the session, simply press “Enter” in the terminal session in which the script was launched mount-vmware.

Starting with Bacula Enterprise 8.4.8, you can limit the Job list using the following command lines:

  • -s= limit the list of tasks to the last XXX days
  • -l= limit the list of tasks to the latest entered numbers
  • -f= specify an advanced filter based on the task name and/or FileSet name

2.2.2 Restoring VMware from the interface Bweb Management Suite

Single file recovery function VMware single file restore can be implemented using Bweb Management Suite. This utility is a recovery wizard that allows you to easily and easily recover files from a guest VM. First, you need to select the client on which the task of creating a backup using vSphere was performed (see Figure 7).

Figure 7. Client Selection

After the Client is selected, the administrator must select a Job (Restore Point) to restore. (See Figure 8 on another page). If the selected Job is a valid vSphere Job, i.e. can be executed, the third step will display a list of virtual machines included in the FileSet (see Figure 9 on the next page).

At this point, Bacula software should create virtual image selected VM. You need to restore a couple of small files from each Job that make up the selected restore point Restore Point. After completing the Bacula software procedure, you need to mount the disk of the selected VM on the system. These steps are generally quick, but the time taken depends largely on the configuration used. Indexes are created and saved during this phase to speed up subsequent recovery requests.

After the disk is mounted, the files of the selected VM will be displayed in the file manager. In it you can select files or directories to restore. (See Figure 10 on page 31). The administrator can then create a ZIP or TAR archive. The archive will be created automatically and saved to / opt/bacula/working. A link will be created to securely download the archive via HTTP. The administrator will be able to provide this link to the end user.

Each time the administrator selects files, he will be able to choose the method of restoring the file in compressed form in tar or zip format. (See Figure 11 on page 32). After recovery, it is important to end the session in order to free up resources used for recovery.

Figure 8. Selecting a restore point

Figure 9. Selecting a VM

Figure 10. Selecting files

Figure 11. File access

2.3 Notes

2.3.1 Cache directory

To speed up subsequent single file recovery procedures, some files created during the recovery session are stored in a cache directory.

After some time, you can delete cache files. They will be recreated if necessary.

2.4 Limitations

  • VMware's single file recovery feature uses the Bacula BVFS interface to display a list of files and directories. In the case of MySQL; Despite MySQL's limitations with indexes on TEXT columns, the procedure does not have a significant impact on MySQL performance. However, for best results, we recommend using PostgreSQL.