libnetfilter_queue is a userspace library providing an API to packets that have been queued by the kernel packet filter. It is is part of a system that replaces the old ip_queue / libipq mechanism (withdrawn in kernel 3.5).

libnetfilter_queue homepage is: https://netfilter.org/projects/libnetfilter_queue/

Dependencies

libnetfilter_queue requires libmnl, libnfnetlink and a kernel that includes the Netfilter NFQUEUE over NFNETLINK interface (i.e. 2.6.14 or later).

Main Features

receiving queued packets from the kernel nfnetlink_queue subsystem
issuing verdicts and possibly reinjecting altered packets to the kernel nfnetlink_queue subsystem

The cinematic is the following: When an nft rule with action queue matches, the kernel terminates the current nft chain and enqueues the packet in a chained list. It then formats and sends an nfnetlink message containing the packet id and whatever information the userspace program configured to receive (packet data and/or metadata) via a socket to the userspace program.

The userspace program must issue a verdict advising the kernel to accept or drop the packet. Either verdict takes the packet off the queue: drop discards the packet while accept passes it on to the next chain. Userspace can also alter packet contents or metadata (e.g. packet mark, contrack mark). Verdict can be done in asynchronous manner, as the only needed information is the packet id.

When a queue is full, packets that should have been enqueued are dropped by kernel instead of being enqueued.

Git Tree

The current development version of libnetfilter_queue can be accessed at https://git.netfilter.org/libnetfilter_queue.

Privileges

You need the CAP_NET_ADMIN capability in order to allow your application to receive from and to send packets to kernel-space.

Using libnetfilter_queue

To write your own program using libnetfilter_queue, you should start by reading (or, if feasible, compiling and stepping through with gdb) nf-queue.c source file. Simple compile line:

gcc -g3 -ggdb -Wall -lmnl -lnetfilter_queue -o nf-queue nf-queue.c

The doxygen documentation LibrarySetup is Deprecated and incompatible with non-deprecated functions. It is hoped to produce a corresponding non-deprecated (Current) topic soon.

Somewhat outdated but possibly providing some insight into libnetfilter_queue usage is the following article: https://home.regit.org/netfilter-en/using-nfqueue-and-libnetfilter_queue/

ENOBUFS errors in recv()

recv() may return -1 and errno is set to ENOBUFS in case that your application is not fast enough to retrieve the packets from the kernel. In that case, you can increase the socket buffer size by means of nfnl_rcvbufsiz(). Although this delays the appearance of ENOBUFS errors, you may hit it again sooner or later. The next section provides some hints on how to obtain the best performance for your application.

Performance

To improve your libnetfilter_queue application in terms of performance, you may consider the following tweaks:

increase the default socket buffer size by means of nfnl_rcvbufsiz().
set nice value of your process to -20 (maximum priority).
set the CPU affinity of your process to a spare core that is not used to handle NIC interruptions.
set NETLINK_NO_ENOBUFS socket option to avoid receiving ENOBUFS errors (requires Linux kernel >= 2.6.30).
see –queue-balance option in NFQUEUE target for multi-threaded apps (it requires Linux kernel >= 2.6.31).
consider using fail-open option see nfq_set_queue_flags() (it requires Linux kernel >= 3.6)
increase queue max length with nfq_set_queue_maxlen() to resist to packets burst