Linux: Neighbour Table Overflow Error and Solution

I setup a CentOS Linux based Linux server running as a gateway and firewall server. However, I'm getting the following messages in the /var/log/messages log file:

Dec 20 00:41:01 fw01 kernel: Neighbour table overflow.
Dec 20 00:41:01 fw01 last message repeated 20 times

OR

Dec 20 00:41:01 fw03 kernel: [ 8987.821184] Neighbour table overflow.
Dec 20 00:41:01 fw03 kernel: [ 8987.860465] printk: 100 messages suppressed.

Why does kernel throw "Neighbour table overflow" messages in syslog? How do I fix this problem under Debian / CentOS / RHEL / Fedora / Ubuntu Linux?

For busy networks (or gateway / firewall Linux server) it is mandatory to increase the kernel's internal ARP cache size. The following kernel variables are used:

net.ipv4.neigh.default.gc_thresh1
net.ipv4.neigh.default.gc_thresh2
net.ipv4.neigh.default.gc_thresh3

To see current values, type:
# sysctl net.ipv4.neigh.default.gc_thresh1
Sample outputs:

net.ipv4.neigh.default.gc_thresh1 = 128

Type the following command:
# sysctl net.ipv4.neigh.default.gc_thresh2
Sample outputs:

net.ipv4.neigh.default.gc_thresh2 = 512

Type the following command:
# sysctl net.ipv4.neigh.default.gc_thresh3
Sample outputs:

net.ipv4.neigh.default.gc_thresh3 = 1024

So you need to make sure that the arp table to become bigger than the above defaults. The above limitations are good for small network or a single server. This will also affect your DNS traffic.

How Do I Fix "Neighbour Table Overflow" Error?

Edit /etc/sysctl.conf file, enter:
# vi /etc/sysctl.conf
Append the following values (this is taken from server that protects over 200 desktops running MS-Windows, Linux, and Apple OS X):

 ## works best with <= 500 client computers ##
# Force gc to clean-up quickly
net.ipv4.neigh.default.gc_interval = 3600
 
# Set ARP cache entry timeout
net.ipv4.neigh.default.gc_stale_time = 3600
 
# Setup DNS threshold for arp
net.ipv4.neigh.default.gc_thresh3 = 4096
net.ipv4.neigh.default.gc_thresh2 = 2048
net.ipv4.neigh.default.gc_thresh1 = 1024
 

To load new changes type the following command:
# sysctl -p