Summary
Snapvault backup failed with error return code rc=501. The snapmirror log on the filer showed a read socket error.
Symptoms
When running a Snapvault backup of a Windows node the job log showed the following error:
Transfer backup(/vol/OSSV_Vol/[OSSVnoncrit]<NODE>) failed with exception: <EXCEPTION CATEGORY="BEXEXCEPTION"><CODE>501</CODE></EXCEPTION>
The corresponding nibbler log showed the following errors:
Mar 26 19:21:54 2007:5316:WARNING:SVP:496:Failed writing to [FILER]. System call send(496,2101248) failed: [GetLastError(10055):An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full. ].
Mar 26 19:21:54 2007:5316:ERROR:Failed sending raw disk data to secondary. OSSVEncodeFileData() failed:[OSSV_SEND_DATA_ERR]:
Mar 26 19:21:54 2007:5316:ERROR:Failed sending allocated block[2993930240,15036416]. Exiting backup.:
Mar 26 19:21:54 2007:5316:ERROR:Failed backup. OSSVEncodeEndStreamHeader() failed:[OSSV_ILLEGAL_STATE_ERR]
Mar 26 19:21:54 2007:5316:ERROR:Failed backup. OSSVEncodeEndAck() failed:[OSSV_ILLEGAL_STATE_ERR]
Mar 26 19:21:54 2007:5316:INFO:SVP:496:Closed connection to [FILER]:[N]
Resolution
This problem results from a specific timing or communications issue between the client and the filer that triggered this event. This problem won't occur as you open your host socket in blocking mode. In this case, the send() operation returned error 10055 rather than blocking. This is not directly a Backup Express (BEX) problem and is described further in Microsoft's knowledge base article 201213.
support.microsoft.com/kb/201213/en-us
To help alleviate this problem if it reoccurs, we recommend increasing the size of the TCP send buffers for the nibbler by implementing the following setting. This setting is implemented on the Windows source node. To implement this setting, open regedit on the client node and go to HKLM\Software\Syncsort\<server-name>\0. Create the following:
String Value Value Data
NIB_SO_SNDBUF 2048
After making this change, you should stop and restart the Backup Express Advanced Protection Manager service on the client.