ixgbevf XDP shared queues#5
Open
walking-machine wants to merge 28 commits into
Open
Conversation
Similarly as in commit 5384467 ("iavf: kill "legacy-rx" for good"), drop skb construction logic in favor of only using napi_build_skb() as a superior option that reduces the need to allocate and copy memory. As IXGBEVF_PRIV_FLAGS_LEGACY_RX is the only private flag in ixgbevf, entirely remove private flags support from the driver. When compared to iavf changes, ixgbevf has a single complication: MAC type 82599 cannot finely limit the DMA write size with RXDCTL.RLPML, only 1024 increments through SRRCTL are available, see commit fe68195 ("ixgbevf: Require large buffers for build_skb on 82599VF") and commit 2bafa8f ("ixgbe: don't set RXDCTL.RLPML for 82599"). Therefore, this is a special case requiring legacy RX unless large buffers are used. For now, solve this by always using large buffers for this MAC type. Suggested-by: Alexander Lobakin <aleksander.lobakin@intel.com> Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com> Reviewed-by: Alexander Lobakin <aleksander.lobakin@intel.com> Signed-off-by: Larysa Zaremba <larysa.zaremba@intel.com>
Again, same as in the related iavf commit 920d86f ("iavf: drop page splitting and recycling"), as an intermediate step, drop the page sharing and recycling logic in a preparation to offload it to page_pool. Instead of the previous sharing and recycling, just allocate a new page every time. Suggested-by: Alexander Lobakin <aleksander.lobakin@intel.com> Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com> Reviewed-by: Alexander Lobakin <aleksander.lobakin@intel.com> Signed-off-by: Larysa Zaremba <larysa.zaremba@intel.com>
Use page_pool buffers by the means of libeth in the Rx queues, this significantly reduces code complexity of the driver itself. Suggested-by: Alexander Lobakin <aleksander.lobakin@intel.com> Reviewed-by: Alexander Lobakin <aleksander.lobakin@intel.com> Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com> Signed-off-by: Larysa Zaremba <larysa.zaremba@intel.com>
Implement XDP support for received fragmented packets, this requires using some helpers from libeth_xdp. Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com> Signed-off-by: Larysa Zaremba <larysa.zaremba@intel.com>
Use libeth to support XDP_TX action for segmented packets. Reviewed-by: Alexander Lobakin <aleksander.lobakin@intel.com> Signed-off-by: Larysa Zaremba <larysa.zaremba@intel.com>
To fully support XDP_REDIRECT, utilize more libeth helpers in XDP Rx path, hence save cached_ntu in the ring structure instead of stack. ixgbevf-supported VFs usually have few queues, so use libeth_xdpsq_lock functionality for XDP queue sharing. Adjust filling-in of XDP Tx descriptors to use data from xdp frame. Otherwise, simply use libeth helpers to implement .ndo_xdp_xmit(). While at it, fix a typo in libeth docs. Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com> Signed-off-by: Larysa Zaremba <larysa.zaremba@intel.com>
Introduce pseudo header split support in the ixgbevf driver, specifically targeting ixgbe_mac_82599_vf. On older hardware (e.g. ixgbe_mac_82599_vf), RX DMA write size can only be limited in 1K increments. This causes issues when attempting to fit multiple packets per page, as a DMA write may overwrite the headroom of the next packet. To address this, introduce pseudo header split support, where the hardware copies the full L2 header into a dedicated header buffer. This avoids the need for HR/TR alignment and allows safe skb construction from the header buffer without risking overwrites. Given that once packet is too big to fit into a single page, the behaviour is the same for all supported HW, use pseudo header split only for smaller packets. Signed-off-by: Natalia Wochtman <natalia.wochtman@intel.com> Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com> Co-developed-by: Larysa Zaremba <larysa.zaremba@intel.com> Signed-off-by: Larysa Zaremba <larysa.zaremba@intel.com>
Currently, when MTU is changed, page pool is not reconfigured, which leads to usage of suboptimal buffer sizes. Always destroy page pool when cleaning the ring up and create it anew when we first allocate Rx buffers. Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com> Signed-off-by: Larysa Zaremba <larysa.zaremba@intel.com>
xskxceiver attempts to change MTU after attaching XDP program, ixgbevf rejects the request leading to test being failed. Support MTU change operation even when XDP program is already attached, perform the same frame size check as when attaching a program. Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com> Signed-off-by: Larysa Zaremba <larysa.zaremba@intel.com>
AF_XDP ZC Rx path is also required to implement skb creation. Move all common functions to a header file as inlines. Signed-off-by: Larysa Zaremba <larysa.zaremba@intel.com>
Implement xsk_buff_pool configuration and supporting functionality, such as a single queue pair reconfiguration. Also, properly initialize Rx buffers. Signed-off-by: Larysa Zaremba <larysa.zaremba@intel.com>
Add code that handles Tx ZC queues inside of napi_poll(), utilize libeth. As NIC's multiple buffer conventions do not play nicely with AF_XDP's, leave handling of segments for later. Signed-off-by: Larysa Zaremba <larysa.zaremba@intel.com>
Add code that handles AF_XDP ZC Rx queues inside of napi_poll(), utilize libeth helpers. Signed-off-by: Larysa Zaremba <larysa.zaremba@intel.com>
To finalize basic AF_XDP implementation, set features and add .ndo_xsk_wakeup() handler. Signed-off-by: Larysa Zaremba <larysa.zaremba@intel.com>
Transmitting multi-buffer AF_XDP packets is not very straightforward given HW limitations in ixgbevf, namely that the first data descriptor must contain the length of the whole packet. Use private data of an sqe to store the length of an unfinished packet so far and the first descriptor index. Once EoP zero-copy descriptor is processed, write the accumulated length into the saved first descriptor. Signed-off-by: Larysa Zaremba <larysa.zaremba@intel.com>
num_online_cpus() is not a good goal for number of VF queues. We only have 3 interrupts of which 1 is used for mbx, so effectively we have 2 channels max, only 2 CPUs are used. More queue pairs are still useful for AF_XDP/TCs. Good idea to default to 2 queue pairs by default. Another option is to limit Rx queue number to num_online_cpus(), and hence always have possible XDP without TCs. Handling of queue number when dealing with TCs has to bee updated too: * how does VF know, if the configuration is a result of DCB, and hence incompatible with XDP vs GDV TCs * we should not limit number of Tx queues to 1 when there are TCs, Rx queue number also doesn't have to be equal to TC, as LKV allows 2 queues per TC, and this makes RSS sense Signed-off-by: Larysa Zaremba <larysa.zaremba@intel.com>
Currently, XDP Tx bulk flush implicitely unlocks the used XDP Tx queue, which is assumed to be locked inside of the prep() callback. This does not permit the usage of any other locking mechanism, e.g. locking netdev queues as igc does to circumvent resource limitations. Given that locking is assumed to be done by the prep() callback, add an unprep() callback, which is meant to unconditionally undo the effects of prep(). Add a default unprep() which unlocks XDP TxQ in order to not disturb the existing driver code. Signed-off-by: Larysa Zaremba <larysa.zaremba@intel.com>
While having a platform-dependent priv size improves 32-bit performance, it complicates usage of priv as a container. Make priv a 64-bit int, so structures can be packed there more conveniently. Signed-off-by: Larysa Zaremba <larysa.zaremba@intel.com>
Currently, XDP queue sharing, and hence modulo operation in the bulk initialization, are dependent on initialized libeth XDP SQ locks. But when using Tx queues for XDP, we do not use libeth locking. So add a static variant of a buff initializeation function, which will perform modulo operation unconditionally. Signed-off-by: Larysa Zaremba <larysa.zaremba@intel.com>
libeth_tx provides structures and helpers for Tx buffer management. Using it in ixgbevf will allow seemless integration with libeth_xdp and to reduce code volume. Use libeth_tx in Tx buffer management code. Signed-off-by: Larysa Zaremba <larysa.zaremba@intel.com>
VFs supported by ixgbevf driver can have pretty limited queue number, e.g. only 4. User may wish to use all 4 netdev queues and XDP simultaneously. Such feature is already provided by igc and igb drivers. Based on igb and igc approach, add a shared (between XDP and skb Tx) queue mode, which is used, if current configured number of channels is more than half of number of available queue pairs. Signed-off-by: Larysa Zaremba <larysa.zaremba@intel.com>
When AF_XDP ZC socket utilizes Rx much more than Tx, libeth_xsk_xmit_do_bulk() will often be called without any frames to xmit. This leads to unneccessary calls to the prep() callback, which in case of shared queues includes locking. Furthermore, ixgbevf HW requires proper context descriptors for transmission. This is not a big deal, when the ring is XDP-only, since one descriptor at the start is enough. But when a single ring is shared between XDP and regular skb processing, context needs to be reset before each bulk, so from the prep() callback. This leads to multiple unused context descriptors on the ring, which triggers an MDD event for this VF. Check whether we actually have any buffers to transmit before flushing bulk, and hence prepping Tx queue. Signed-off-by: Larysa Zaremba <larysa.zaremba@intel.com>
Signed-off-by: Larysa Zaremba <larysa.zaremba@intel.com>
walking-machine
force-pushed
the
ixgbevf-xdp-public
branch
from
July 6, 2026 06:32
3176c41 to
d9c60d3
Compare
walking-machine
force-pushed
the
ixgbevf-xdp-public
branch
from
July 27, 2026 12:17
d9c60d3 to
0a7a647
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.