Summary
This article describes the solutions to various network interface card issues.
Symptoms
Backups terminate or hang with return codes of 12111 and/or 10054.
Resolution
A) With some network cards, particularly those optimized for high bandwith and large packet sizes, the reliability when sending small packet sizes is significantly lower. You can identify this particular problem using the DPX command-line utility testperf with both small and large packet sizes. This utility is similar in function to the GNU utility ttcp. To use this utility, the DPX cmagent must be running on both the source and target machines. To send large packets from the current server to a target server, simply execute testperf from the tools (or bin) directory of DPX with the following options:
testperfsend 10000 32768 -p -v -r -u -s <TARGETSERVER>
Where TARGETSERVER is the target machine (destination) to send 10000 packets that are 32 KB (32768 bytes) in size. Then, you can run an additional test with a small packet size and see if there are any errors:
testperfsend 10000000 32 -p -v -r -u -s <TARGETSERVER>
Where TARGETSERVER is the target machine (destination) to send 1000000 packets that are 32 bytes in size. Run this test in both directions. In addition, this is a lot of data for a small packet size, you may wish to try a lower number of packets first so that you will see a reasonable completion time. If sending the data with large packets succeeds whereas with small packets this test gives an error, i.e. 12111 or 10054, then there is likely a problem with the network interface card. If one or both cases fail, there is a problem communicating between these machines. Try the following suggestions.
WARNING: Using Registry Editor incorrectly can cause serious problems that may require you to reinstall your operating system. Microsoft cannot guarantee that problems resulting from the incorrect use of Registry Editorcan be solved. Use Registry Editor at your own risk. For information about how to edit the registry, view the "Changing Keys And Values" Help topic in Registry Editor (Regedit.exe) or the "Add and Delete Information in the Registry" and "Edit Registry Data" Help topics in Regedt32.exe. Note that you should back up the registry before you edit it. If you are running WindowsNT, you should also update your Emergency Repair Disk (ERD).
B) If this is a FastE NIC, it should be 100 fixed/full-duplex, not autodetect/sense. If this is a GigE or fiber NIC, the DPX communication parameters for the NIC should be investigated on the client, device, and master server.
- For Windows - Open regedit and navigate to, HKLM\Software\Syncsort\Agent\SSICMAPI. Remove the option -tnd from this string value if it is present. Then, restart DPX.
- For UNIX - Edit the file SSICMAPI in the misc directory of DPX andremove the option -tnd from this file if it is present. Then, restart DPX.
C) For Windows 2000+ servers, upgrade to the latest service pack and install the latest NIC device drivers.
D) For Windows NT, install the latest NIC drivers, install or reinstall service pack 6a, change the temp and tmp directory environment variable to point to the fastest drive (these variables are often user-specific so make sure that cmagent is using the fastest drive for temp space), and make sure that there is ample available space in the tmp and temp directories.
E) A suggestion found on Microsoft's web is to adjust the following TCP/IP setting by adding a subkey in the registry that reduces the number of timeouts by allowing more time for the connection to complete. See MS KBarticles 170359 and 235387 for more detail. The setting is not present in the registry by default.
Start Registry Editor (Regedt32.exe) and go to the following subkey:
HKLM\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters
On the Edit menu, click Add Value, and then add the following information:
Value Name: TcpMaxDataRetransmissions
Value Type: REG_DWORD - Number
Valid Range: 0 - 0xFFFFFFFF
Default Value: 5 Decimal
New Value: 10 Decimal
Click OK and then exit the Registry Editor, restart the server.
F) If you have a Compaq NC6136, NC6134, or NC7131 NIC in the client node, remove the value "TransmitMode"="3" from the Registry. If the removal of "TransmitMode"="3" doesn't help, try switching to a Microsoft distributed driver for the Compaq Gig NIC cards.
G) Nodes with a Compaq NC3163 Embedded Fast Ethernet Controllers:
- Verify that you are using the latest Compaq driver.
- Change the Performance tuning on the NIC:
- Go to Computer Management
- Select the Compaq NC3163 NIC.
- Open Properties.
- Open Advanced tab.
- Select Adaptive Performance Tuning.
- Set it to 0.
H) If you have an Intel 8255 NIC, please update to the latest driver.
I) If you have a Broadcom NetXtreme NIC with teaming mode enabled. We have seen errors running this NIC in teaming mode. If possible reconfigured the card, removing the teaming mode.
J) It is possible that the Local System account settings are different than the Administrator account. One troubleshooting step would be to change the logon user that the CMAGENT service is running as from the Local System account to the Administrator account.
K) Apply Network Recovery modules to the servers that are exhibiting problems. Support for Network Recovery on some operating systems is available in DPX. NetworkRecovery is enabled when the file cmcbklib.* (extension varies by operating system) is present in the bin directory of DPX. If this module is not in the DPX bin folder, contact DPX technical support to request this feature for your system.
If you continue to experience issues after applying the above steps, the following suggestions may be helpful. Perform these steps only after request and direction from DPX technical support.
L) update DMBS params to apply "--perfthresh nnnnn" which will send performance info every nnnnn KB rather than the default of 32768 KB (32 MB). The effect is that fewer (but larger) performance updates are sent to JH by DMBS. For DPX, the performance threshold for the updates is 32 MB by default.
M) Slow down the Backup Sender. In the registry on the client add the following switches to DMBS:
--delaysend
--delaytime 10
If this setting improves the situation but does not resolve it, increase the delaytime in increments of 10.