linux网卡收包调优

网卡监控

#网卡统计
ethtool -S eth0

#sysfs统计
cat /sys/class/net/eth0/statistics/rx_dropped

#proc统计
cat /proc/net/dev

网卡调优

队列

#查看网卡队列
ethtool -l eth0
#调整网卡队列
ethtool -L eth0 rx 8
##combined queue，这种模式下，RX queue 和 TX queue 是一 对一绑定
ethtool -L eth0 combined 8

#查看队列大小
ethtool -g eth0
#调整队列大小
ethtool -G eth0 rx 4096

#队列权重
ethtool -x eth0
#调整队列权重
ethtool -X eth0 equal 2

#收包哈希
ethtool -n eth0 rx-flow-hash udp4
ethtool -N eth0 rx-flow-hash udp4 sdfn

#ntuple filtering
ethtool -k eth0
ethtool -K eth0 ntuple on
ethtool -u eth0
ethtool -U eth0 flow-type tcp4 dst-port 80 action 2

硬件中断

cat /proc/interrupts

软中断

#查看软中信息
cat /proc/softirqs

echo 0 > /proc/irq/IRQ_NUMBER/smp_affinity_list

监控数据处理

cat /proc/net/softnet_stat
sysctl -w net.core.netdev_budget=3600
sysctl -w net.core.netdev_budget_usecs = 10000

RPS

/sys/class/net/DEVICE_NAME/queues/QUEUE/rps_cpus

RFS

将属于相同 flow 的包送到相同的 CPU 进行处理 sysctl -w net.core.rps_sock_flow_entries=32768 echo 2048 > /sys/class/net/eth0/queues/rx-0/rps_flow_cnt

收包打时间戳

sysctl -w net.core.netdev_tstamp_prequeue=0

backlog

在 Linux 网络收包过程中，CPU 的 backlog queue（积压队列）是指内核在软中断阶段处理网络包时，如果当前 CPU 处理不过来，会将未处理的包暂存到 backlog 队列中。这个队列是 per-CPU 的，主要用于缓冲那些暂时无法及时处理的网络包，防止丢包。

backlog queue 的作用：

缓解瞬时流量高峰，避免因处理速度跟不上而丢包。 由 softirq 处理（如 net_rx_action），当 backlog 队列满时，内核会丢弃新到的包。 相关参数和监控：

/proc/net/softnet_stat 可以查看每个 CPU 的 backlog 队列状态和丢包统计。 net.core.netdev_max_backlog 用于设置 backlog 队列的最大长度。 简而言之，backlog queue 是 CPU 级别的网络包缓冲队列，用于提升网络收包的鲁棒性和性能。

sysctl -w net.core.netdev_max_backlog=3000 sysctl -w net.core.dev_weight=600 sysctl -w net.core.flow_limit_table_len=8192

cat /proc/net/snmp InReceives: The total number of IP packets that reached ip_rcv before any data integrity checks. InHdrErrors: Total number of IP packets with corrupted headers. The header was too short, too long, non-existent, had the wrong IP protocol version number, etc. InAddrErrors: Total number of IP packets where the host was unreachable. ForwDatagrams: Total number of IP packets that have been forwarded. InUnknownProtos: Total number of IP packets with unknown or unsupported protocol specified in the header. InDiscards: Total number of IP packets discarded due to memory allocation failure or checksum failure when packets are trimmed. InDelivers: Total number of IP packets successfully delivered to higher protocol layers. Keep in mind that those protocol layers may drop data even if the IP layer does not. InCsumErrors: Total number of IP Packets with checksum errors.

sysctl -w net.core.netdev_max_backlog=3000 sysctl -w net.core.dev_weight=600 sysctl -w net.core.flow_limit_table_len=8192 AMD_pstat=disable idle=poll nohz=off iommu=pt

cpupower monitor