Summary
For Linux nodes (case of study RH5.11), the / backup completes as BASE, but always fails on INCR run, with the following sample errors. On Linux servers, it is possible to set an individual snapshot size for each mount point being backed up on the server.
Symptoms
A Linux node's root system block backup fails with Buffer I/O error (NDMP Error 542113793):
………. job log
X.X.X.X 11/12/2015 7:57:46 am SNBNCJ_103E Task 4 NDMP_LOG: id(2), type(NDMP_LOG_ERROR), text(process was aborted)
X.X.X.X 11/12/2015 7:57:46 am SNBNCJ_100N Task 4 NDMP_LOG: id(0), type(), text(NDMP_SVS_NOTIFY_TRANSFER_BACKUP operation failed. Error(542113793,
NDMP_SV_RPL_GENERIC); Rseq(11))
X.X.X.X 11/12/2015 7:57:46 am SNBSVH_969E Transfer backup(/vol/AR_nctirlms1/[AR_nctirlms1b]nctirlms1@{B98768D0}) failed with exception:
NDMPSessionException(0, transferBackup exception: <EXCEPTION CATEGORY="BEXEXCEPTION"><CODE>501</CODE></EXCEPTION>)
………. nibbler.log
Nov 12 07:57:36 2015:23331:ERROR:Failed reading raw disk: read_block(8192) at disk offset[0] failed: Last system error:[strerror(5):Input/output error]
Nov 12 07:57:36 2015:23331:ERROR:Failed sending allocated block[0,8192]. Exiting backup.:
………..snap
X.X.X.X 11/12/2015 7:57:36 am SNBSNP3072J 47734886389504: Snapshot for /data is
created successfully and mounted to /var/BackEx/mount/snap.wgvWbC
X.X.X.X 11/12/2015 7:57:36 am SNBSNP3069E 47734886389504: Command
"/usr/sbin/lvcreate --snapshot --size 4160M --permission r --name
bex_snapshot.2098702574 /dev/local/dedigate" failed, exit code = 5: (Volume group
"local" has insufficient free space (102 extents): 130 required.)
………. system messages
Nov 12 07:57:36 nctirlms1 kernel: Buffer I/O error on device dm-4, logical block 0
Nov 12 07:57:36 nctirlms1 kernel: Buffer I/O error on device dm-4, logical block 1
Nov 12 07:57:36 nctirlms1 kernel: Buffer I/O error on device dm-4, logical block 2
Nov 12 07:57:36 nctirlms1 kernel: Buffer I/O error on device dm-4, logical block 3
Nov 12 07:57:36 nctirlms1 kernel: Buffer I/O error on device dm-4, logical block 0
Resolution
The issue is caused by / backup exceeding the snapshot size threshold set in DPX. The default snapshot size on a logical volume is 10% of the source volume. There are two options available on the client machine to change this value.
Option 1
The BEX_SNAPSHOT_SIZE environment variable on the client node can be set to change the default value. This setting applies globally to all mount points on the node.
Option 2
A new DPX feature allows for a separate snapshot size configuration for each mount point.
Create the following file: $SSPRODIR/misc/snapshot.conf. You can specify the size for each volume snapshot with this configuration file.
The following is an example of the $SSPRODIR/misc/snapshot.conf file:
#sample config file
/ 5%
/opt 20%
/usr 10%
If the mount point is not listed above, then the snapshot size specified in the environment variable BEX_SNAPSHOT_SIZE will be used. If the environment variable BEX_SNAPSHOT_SIZE is not used, then by default,10% of the source value on the logical volume will be used.