Let's Build a Tiny Firewall with XDP

We saw what an XDP program was in part one: it can be a way to filter a network stream.

In this part, I will take you on a brief tour of the different headers and show you how filtering can be done. We will see how to redevelop xdp-filter with the Rust Aya framework.

Follow the guide!

I assume you are already in an environment for developing with Aya. If not, you can use the Killercoda lab:

Let’s build an XDP hello world program

#

Let’s recreate the development environment

#

As we saw in the previous section, I suggest installing namespaces:

git clone https://github.com/littlejo/eunomia.dev
cd eunomia.dev/docs/tutorials/42-xdp-loadbalancer/
./setup.sh

Let’s recall what we created:

Let’s create the “hello world” program

#

As we did with xdp-filter, we will install the XDP program on veth0.

Let’s generate the Aya program:

cargo generate --name browser-xdp \
               -d program_type=xdp \
               -d default_iface=veth0 \
               https://github.com/aya-rs/aya-template

Let’s build and install the “hello world” program:

cd browser-xdp/
cargo run

On another terminal, let’s test the program:

ip netns exec lb ping -c 2 10.0.0.1

On the cargo run terminal, we can clearly see two packets:

[INFO  browser_xdp] received a packet
[INFO  browser_xdp] received a packet

So every time the interface receives a packet, the XDP program is launched.

Now we’ll see how to stop a packet and what type of packet we’re dealing with.

The basics of creating an XDP program

#

XDP actions

#

Let’s first look at the code generated by cargo.

The most important file is this one: browser-xdp-ebpf/src/main.rs, i.e., the kernel-side code.

Overall, the code looks like the code generated for other types of eBPF programs such as Tracepoints:

#[xdp]
pub fn browser_xdp(ctx: XdpContext) -> u32 {
    match try_browser_xdp(ctx) {
        Ok(ret) => ret,
        Err(_) => xdp_action::XDP_ABORTED,
    }
}

fn try_browser_xdp(ctx: XdpContext) -> Result<u32, u32> {
    info!(&ctx, "received a packet");
    Ok(xdp_action::XDP_PASS)
}

However, there is one small difference: xdp_action::*.

This determines what the packet does once it has finished traversing the eBPF program.

There are five different actions:

• XDP_ABORTED: Stops the packet with an error
• XDP_DROP: Stops the packet silently
• XDP_PASS: Lets the packet continue through the network stack.
• XDP_REDIRECT: Redirects the packet to another interface or to an AF_XDP socket attached to an interface.
• XDP_TX: Redirects the packet to the same interface

In this article, we will only use XDP_PASS and XDP_DROP.

The XDP context and its methods

#

As with all other eBPF programs, to go beyond hello world, you need to understand how to work with the context.

Let’s take a look at the documentation:

Unlike previous types of programs where only one method was used, there are four different methods for XDP:

• data() returns the memory address of the start of the network packet.
• data_end() returns the memory address of the end of the network packet.
• metadata() returns the memory address of the start of the XDP metadata linked to the Linux kernel or driver.
• metadata_end() returns the memory address of the end of the metadata. In the same way as data_end(),

In this article, we will only use the data() and data_end() methods to retrieve the different headers.

How to retrieve the headers?

#

Let’s recall the following diagram:

• data() is located at the beginning of the ethHdr header
• data_end() is located at the end of the payload

How do we retrieve the Ethernet header (ethHdr)? It is located at the very beginning of the packet.

let ethhdr = ctx.data();

To retrieve the next header, simply add the length of the Ethernet header. In Rust pseudo-code, this would look like:

let ipv4hdr = ctx.data() + sizeof(ethhdr);

Etc.

Let’s focus on the Ethernet header. How can we retrieve its elements? ctx.data() is of type usize. This is a natural integer that represents a memory address. We therefore need to retrieve its content. How can we do this?

In the same way that *const u8 represents a pointer to a character string, we must do the same for the Ethernet header.

We must therefore convert it into a structure that represents the Ethernet frame header. There are several ways to do this, such as looking at the C code and converting it to Rust, which is a good exercise but a little tedious. Fortunately, since Rust is now well established in the eBPF ecosystem, there is already a Rust crate that has done the work for us: network-types.

To retrieve this crate, we will modify the browser-xdp-ebpf/Cargo.toml file and add the following to the dependencies section:

network-types = "0.1.0"

We will then be able to include it in the main code (browser-xdp-ebpf/src/main.rs):

use network_types::eth::EthHdr;

[...]

let ethhdr = ctx.data() as *const EthHdr;

Now that we have retrieved the Ethernet header structure, we can retrieve the various elements the header consists of:

• DST MAC: the destination MAC address
• SRC MAC: the source MAC address
• ETHERTYPE: determines the upper-layer protocol (such as IPv4, IPv6, or Arp, to name the most well-known).

How can this be done with Aya? Let’s review the documentation:

So to retrieve the destination address, simply code:

let dst_addr = unsafe { (*ethhdr).dst_addr};

info!(&ctx, "dst_addr {:mac} ", dst_addr);

We have to use unsafe because we are dereferencing a raw pointer.

Now let’s try it. Here is the modified main code:

use network_types::eth::EthHdr;

fn try_browser_xdp(ctx: XdpContext) -> Result<u32, u32> {
    let ethhdr = ctx.data() as *const EthHdr;

    let dst_addr = unsafe { (*ethhdr).dst_addr};

    info!(&ctx, "dst_addr {:mac} ", dst_addr);

    info!(&ctx, "received a packet");
    Ok(xdp_action::XDP_PASS)
}

Let’s build the program:

cargo build

The compilation works.

Let’s install it in the kernel:

cargo run

We then get the following error:

Error: the BPF_PROG_LOAD syscall failed. Verifier output: 0: R1=ctx() R10=fp0
; unsafe { (*self.ctx).data as usize } @ xdp.rs:16
0: (61) r1 = *(u32 *)(r1 +0)          ; R1_w=pkt(r=0)
; let dst_addr = unsafe { (*ethhdr).dst_addr}; @ main.rs:20
1: (71) r0 = *(u8 *)(r1 +1)
invalid access to packet, off=1 size=1, R1(id=0,off=1,r=0)
R1 offset is outside of the packet
verification time 65 usec
stack depth 0
processed 2 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0


Caused by:
    Permission denied (os error 13)

The verifier doesn’t really like this part of the code:

let ethhdr = ctx.data() as *const EthHdr;

The important part of the error is:

invalid access to packet,
R1 offset is outside of the packet

Not very clear. Let’s try to explain it better. When we try let ethhdr = ctx.data() as *const EthHdr;, there is memory access. The verifier requires us to prove that this access is always within the packet (between data() and data_end()).

We therefore need to perform a check to reassure it before this access:

if ctx.data() + EthHdr::LEN > ctx.data_end() {
    return Ok(xdp_action::XDP_PASS);
}

This can be translated as: if the access is not at the packet level, the XDP program does not process the packet: it lets it continue on its way in the Linux kernel network stack.

Otherwise, we can convert it to EthHdr:

let ethhdr = ctx.data() as *const EthHdr;

Here is the complete code for the main function:

fn try_browser_xdp(ctx: XdpContext) -> Result<u32, u32> {
    let start = ctx.data() ;
    let end = ctx.data_end() ;

    if start + EthHdr::LEN > end { //Check for the verifier
        return Ok(xdp_action::XDP_PASS);
    }

    let ethhdr = start as *const EthHdr;

    let dst_addr = unsafe { (*ethhdr).dst_addr};

    info!(&ctx, "dst_addr {:mac} ", dst_addr);
    info!(&ctx, "received a packet");

    Ok(xdp_action::XDP_PASS)
}

Let’s check that it works:

cargo run

Let’s run some pings. We successfully retrieve the MAC address:

[INFO  browser_xdp] dst_addr de:ad:be:ef:00:01
[INFO  browser_xdp] received a packet
[INFO  browser_xdp] dst_addr de:ad:be:ef:00:01
[INFO  browser_xdp] received a packet

Now that we have successfully retrieved the ethHdr header, how do we retrieve the other headers? Simply add the lengths of the preceding headers.

Obviously, we will use very similar methods for all headers. It would be wise to create a function. Here it is:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

use core::mem::size_of;

#[inline(always)]
fn ptr_at<T>(ctx: &XdpContext, offset: usize) -> Result<*const T, u16> {
    let start = ctx.data();
    let end = ctx.data_end();
    let len = size_of::<T>();

    if start + offset + len > end {
        return Err(1);
    }

    Ok((start + offset) as *const T)
}

A few notes on creating this function:

• (L3) We inline the function for performance reasons
• (L4) <T>: allows us to define a generic function that doesn’t depend on a specific structure
• the offset lets us to navigate from header to header (offset = 0 for ethHdr)
• (L7) We had to change how the structure length is calculated, using size_of, to make it work properly.

Then the main code becomes:

fn try_browser_xdp(ctx: XdpContext) -> Result<u32, u32> {
    let ethhdr: *const EthHdr = ptr_at(&ctx, 0)?;

    let dst_addr = unsafe { (*ethhdr).dst_addr};

    info!(&ctx, "dst_addr {:mac} ", dst_addr);

    info!(&ctx, "received a packet");
    Ok(xdp_action::XDP_PASS)
}

Now that we have created the function that allows us to traverse the different headers, let’s see how we can block or allow packets based on network headers.

Guided tour of the different headers

#