edit · history · print

Tuning the Solaris TCP stack for long distance fast communication

The usual settings of both OpenSolaris as well as Solaris seem to be optimal for sending data across that link as long as you remain in the 1Gbit/s region, but are not optimal for receiving. The quick tests showed with default settings ca 900 Mbit/s upload and 250Mbit/s download speeds.

To check your current settings have a look at the following variables (the last one seems not to be defined in OpenSolaris):

 ndd /dev/tcp tcp_recv_hiwat
 ndd /dev/tcp tcp_xmit_hiwat 
 ndd /dev/tcp tcp_max_buf
 ndd /dev/tcp tcp_cwnd_max
 ndd /dev/ip ip_soft_rings_cnt

Increasing the TCP buffer sizes (the system has to have plenty of memory though) can be achieved with:

 ndd -set /dev/tcp tcp_recv_hiwat 524288
 ndd -set /dev/tcp tcp_xmit_hiwat 524288
 ndd -set /dev/tcp tcp_max_buf 16777216
 ndd -set /dev/tcp tcp_cwnd_max 16777216
 ndd -set /dev/ip ip_soft_rings_cnt 16

This sets the default buffer sizes to 512KB and the maximum buffer size to 16MB. It should be easily handled by a decent system and should give a decent performance even at 300ms distances and multi Gbit/s speeds. In case memory issues appear you can reduce the usage by halving the numbers.

edit · history · print
Page last modified on October 27, 2009, at 05:42 PM