安装 DPDK（CentOS 7.x）

运行环境

  • 操作系统：CentOS-7-x86_64-Minimal-1908.iso

  • RT Kernel 版本：3.10.0-1127.rt56.1093.el7

  • DPDK 版本：18.08

  • Redis 版本：3.2.12-2.el7

  • hiredis 版本：0.14.0

基础环境部署

安装 RT kernel

  • 操作系统更新

$ yum update -y && yum upgrade -y

  • RT repo

sudo tee /etc/yum.repos.d/CentOS-rt.repo >/dev/null <<EOF
# CentOS-rt.repo
 
[rt]
name=CentOS-7-rt
baseurl=http://mirror.centos.org/centos/\$releasever/rt/\$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
EOF

  • 软件依赖

$ yum install tuna tuned tuned-profiles-realtime rt-setup rt-tests -y

  • RT kernel

$ yum install kernel-rt.x86_64 kernel-rt-devel.x86_64 kernel-headers.x86_64 -y
$ grubby --set-default /boot/vmlinuz-3.10.0-1127.rt56.1093.el7.x86_64

  • 卸载无用 kernel

$ uname -a ; rpm -qa kernel\* | sort
Linux nes-base.novalocal 3.10.0-1127.8.2.el7.x86_64 #1 SMP Tue May 12 16:57:42 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
kernel-3.10.0-1127.10.1.el7.x86_64
kernel-3.10.0-1127.8.2.el7.x86_64
kernel-headers-3.10.0-1127.10.1.el7.x86_64
kernel-rt-3.10.0-1127.rt56.1093.el7.x86_64
kernel-rt-devel-3.10.0-1127.rt56.1093.el7.x86_64
kernel-tools-3.10.0-1127.10.1.el7.x86_64
kernel-tools-libs-3.10.0-1127.10.1.el7.x86_64

$ rpm -e kernel-3.10.0-1127.8.2.el7.x86_64
$ rpm -e kernel-3.10.0-1127.10.1.el7.x86_64

$ uname -a ; rpm -qa kernel\* | sort
Linux nes-base.novalocal 3.10.0-1127.8.2.el7.x86_64 #1 SMP Tue May 12 16:57:42 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
kernel-headers-3.10.0-1127.10.1.el7.x86_64
kernel-rt-3.10.0-1127.rt56.1093.el7.x86_64
kernel-rt-devel-3.10.0-1127.rt56.1093.el7.x86_64
kernel-tools-3.10.0-1127.10.1.el7.x86_64
kernel-tools-libs-3.10.0-1127.10.1.el7.x86_64

  • 重启虚拟机并检查

$ uname -r
3.10.0-1127.rt56.1093.el7.x86_64
$ ls /usr/src/kernels/
3.10.0-1127.rt56.1093.el7.x86_64

开启大页内存

  • 查看 Linux 操作系统是否启动了大页内存，如果 HugePages_Total 为 0，意味着 Linux 没有设置或没有启用 Huge pages。

$ grep -i HugePages_Total /proc/meminfo
HugePages_Total:       0

  • 查看是否挂载了 hugetlbfs。

$ mount | grep hugetlbfs
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)

  • （可选）如果没有挂载则手动挂载。

$ mkdir /mnt/huge_1GB
$ mount -t hugetlbfs nodev /mnt/huge_1GB
$ vim /etc/fstab
nodev /mnt/huge_1GB hugetlbfs pagesize=1GB 0 0

  • 修改 grub2，为系统配置 12 个 1GB 的大页面。

$ vi /etc/grub2.cfg
# 定位到 linux16 /boot/vmlinuz-3.10.0-1127.rt56.1093.el7.x86_64 在行末追加
default_hugepagesz=1G hugepagesz=1G hugepages=12

  • 重启并验证

$ cat /proc/meminfo | grep Huge
HugePages_Total:      12
HugePages_Free:       12
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:    1048576 kB
$ cat /sys/devices/system/node/node0/hugepages/hugepages-1048576kB/nr_hugepages
12

安装 DPDK

  • 软件依赖

$ yum install -y libpcap.x86_64 libpcap-devel.x86_64
$ yum install numactl numactl-devel.x86_64 numactl-libs.x86_64 -y
$ yum install pciutils -y
$ yum install net-tools.x86_64 -y
$ yum install gcc gcc-c++ -y

  • 设置 DPDK 环境变量

# vi dpdk.rc
export RTE_SDK=/opt/dpdk-18.08
export RTE_TARGET=x86_64-native-linuxapp-gcc
export DPDK_BUILD=${DPDK_DIR}/${RTE_TARGET}
export LD_LIBRARY_PATH=${RTE_SDK}/${RTE_TARGET}/lib:/usr/local/lib:/usr/lib:

  • 获取 DPDK 源码

$ cd /opt
$ wget http://fast.dpdk.org/rel/dpdk-18.08.tar.gz
$ tar -zxvf dpdk-18.08.tar.gz
$ cd /opt/dpdk-18.08/

  • 编译代码

$ source dpdk.rc
$ cd ${RTE_SDK}
$ make config T=${RTE_TARGET}
$ make clean && make
$ make install T=${RTE_TARGET}

  • 配置开启 PDUMP 抓包

$ cd ${RTE_SDK}/${RTE_TARGET}/
$ vi .config
CONFIG_RTE_LIBRTE_PMD_PCAP=y
CONFIG_RTE_LIBRTE_PDUMP=y
$ make

$ cd ${RTE_SDK}/app/pdump
$ make && make install