Skip to content

feature: rate limit - #346

Open
SkohTV wants to merge 3 commits into
facebook:mainfrom
SkohTV:rate-limit
Open

SkohTV wants to merge 3 commits into
facebook:mainfrom
SkohTV:rate-limit

Conversation

@SkohTV

@SkohTV SkohTV commented Oct 8, 2025 •

Copy link
Copy Markdown
Contributor

Fix #215

Hi, sorry for the long hiatus,
A bunch of stuff came up, and I didn't have the time to do the big rebase+refactor.

Changes:

  • Did the big rebase.
  • Changed ratelimit from an action to a matcher, supports for eq with an uint32
  • Kept the map, because some persistence is required for rate limiting.
  • Also kept and slightly changed the elfstub.
  10 first All after
chain my_chain BF_HOOK_XDP{ifindex=2} DROP rule meta.ratelimit eq 10 ACCEPT ACCEPT DROP
chain my_chain BF_HOOK_XDP{ifindex=2} ACCEPT rule meta.ratelimit eq 10 DROP DROP ACCEPT
chain my_chain BF_HOOK_XDP{ifindex=2} DROP rule meta.ratelimit not eq 10 ACCEPT DROP ACCEPT
chain my_chain BF_HOOK_XDP{ifindex=2} ACCEPT rule meta.ratelimit not eq 10 DROP ACCEPT DROP

If that implementation seems good, I'll go ahead and add QoL / the documentation

I'd really like to get this PR over the finish line at some point :)

@meta-cla meta-cla Bot added the cla signed label Oct 8, 2025
@SkohTV
SkohTV force-pushed the rate-limit branch 2 times, most recently from 3d56c6e to 63e5923 Compare October 8, 2025 20:45
@SkohTV SkohTV changed the title cli: add ratelimit keyword feature: rate limit Oct 10, 2025

@qdeslandes qdeslandes left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went for an early review, mostly for the parsing part. On the BPF side, the map will contain the runtime values for a given rule's rate limit (current burst/allowance, limit, last update time...).

Comment thread src/bfcli/parser.y Outdated
Comment thread src/bfcli/lexer.l Outdated
Comment thread src/bfcli/parser.y Outdated
Comment thread src/bfcli/parser.y Outdated
Comment thread src/bpfilter/cgen/program.c Outdated
Comment thread src/bpfilter/cgen/program.c Outdated
@SkohTV
SkohTV force-pushed the rate-limit branch 4 times, most recently from df768e9 to fd116e7 Compare January 23, 2026 21:39
@SkohTV

This comment was marked as outdated.

@SkohTV
SkohTV marked this pull request as ready for review January 23, 2026 21:49
@SkohTV
SkohTV requested a review from qdeslandes January 31, 2026 01:08
Comment thread src/bfcli/parser.y Outdated
Comment thread src/bfcli/parser.y Outdated
Comment thread src/bfcli/parser.y
Comment thread src/bpfilter/bpf/ratelimit.bpf.c
Comment thread src/bpfilter/cgen/prog/map.c Outdated
Comment thread src/bpfilter/cgen/elfstub.h Outdated
Comment thread src/bpfilter/cgen/program.c Outdated
Comment thread src/bpfilter/cgen/program.c Outdated
@SkohTV
SkohTV force-pushed the rate-limit branch 4 times, most recently from 0ecf4d4 to 699137e Compare February 3, 2026 20:40
@SkohTV
SkohTV marked this pull request as draft April 29, 2026 10:38
@SkohTV
SkohTV marked this pull request as ready for review April 30, 2026 21:38
@SkohTV
SkohTV requested a review from qdeslandes April 30, 2026 21:40
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown

This PR has had no activity for 14 days. It will be closed in 14 more days unless it is updated. Comment or push to keep it open.

@github-actions github-actions Bot added the stale label Aug 5, 2026
@qdeslandes

Copy link
Copy Markdown
Contributor

This PR has had no activity for 14 days. It will be closed in 14 more days unless it is updated. Comment or push to keep it open.

That's on me, looking at this today :)

@github-actions github-actions Bot removed the stale label Aug 6, 2026

@qdeslandes qdeslandes left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing the rate limit to a matcher is the good call, it will provide more flexibility.

The implementation has been simplified a lot, so much that so of the functionalities we would deem necessary have been removed.

You will need to think about two things now:

  • How do you handle the unit? For now, supporting only s (e.g. meta.limit 10/s) would be enough, but allowing a unit to be passed will allow us to expand support for rate limiting later on.
  • How to handle access to the bf_ratelimit (or whatever the name is) structure from a rule? Rules have IDs, but nothing prevents a rule from having multiple rate-limiting matchers. An BPF array map is a good pick, but you need to ensure a meta.limit matcher is mapped to the correct bf_ratelimit structure in the map.

Also, you'll have to rebase on main :)

Comment thread src/bfcli/lexer.l
Comment thread src/bfcli/lexer.l Outdated
Comment thread src/libbpfilter/CMakeLists.txt Outdated
Comment thread src/libbpfilter/include/bpfilter/matcher.h Outdated
Comment thread src/libbpfilter/matcher.c Outdated
Comment thread src/libbpfilter/matcher.c Outdated
Comment thread src/libbpfilter/cgen/program.h Outdated
Comment thread src/libbpfilter/bpf/ratelimit.bpf.c Outdated
Comment thread src/libbpfilter/bpf/ratelimit.bpf.c Outdated
Comment thread src/libbpfilter/cgen/prog/map.c Outdated
@SkohTV
SkohTV marked this pull request as draft August 18, 2026 12:09
@SkohTV
SkohTV requested a review from qdeslandes September 8, 2026 12:18
Comment thread src/libbpfilter/bpf/limit.bpf.c Outdated
Comment thread src/libbpfilter/cgen/matcher/meta.c Outdated
Comment thread src/libbpfilter/cgen/matcher/meta.c Outdated
@SkohTV
SkohTV force-pushed the rate-limit branch 4 times, most recently from 26e37be to 1187fbb Compare September 24, 2026 10:16
@SkohTV

SkohTV commented Sep 24, 2026

Copy link
Copy Markdown
Contributor Author

I think I figured it out
I've kept it as 2 commits for now, to simplify the review, I'll squash them afterward

@SkohTV
SkohTV requested a review from qdeslandes September 24, 2026 10:21

@qdeslandes qdeslandes left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments, but it's on the right track.

Comment thread src/bfcli/parser.y Outdated
bf_parse_err("failed to create new limit");

bf_list_add_tail(&ruleset->limits, limit);
snprintf(payload, sizeof(payload), "%zu", (uint32_t)idx);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The payload of the meta.limit matcher should not be a string, but an actual value.

Comment thread src/bfcli/parser.y Outdated
Comment thread src/libbpfilter/cgen/matcher/meta.c Outdated
Comment thread src/libbpfilter/cgen/matcher/meta.c Outdated
Comment thread src/libbpfilter/cgen/program.c
Comment thread src/libbpfilter/limit.c Outdated
Comment thread src/libbpfilter/matcher.c Outdated
Comment thread src/libbpfilter/matcher.c Outdated
if (errno == 0) {
*(uint32_t *)payload = idx;
return 0;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to reject trailing characters (*endptr != \0\) and prevent integer overflow.

Comment thread src/libbpfilter/pack.c Outdated
Comment thread src/libbpfilter/matcher.c Outdated
assert(payload);

if (*(uint64_t *)payload >> 32 == BF_TIME_S)
(void)fprintf(stdout, "%d/s", *(uint32_t *)payload);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are you trying to do here?

@SkohTV

SkohTV commented Sep 29, 2026

Copy link
Copy Markdown
Contributor Author

Since I switched from bf_matcher_new_from_raw(...) to bf_matcher_new(...), _bf_parse_limit(...) and _bf_print_limit(...) are not used anymore.
There's no char * to uint32_t conversion anymore.

But that means BF_MATCHER_META_LIMIT in _bf_matcher_metas is just

[BF_MATCHER_META_LIMIT] =
	{
		.layer = BF_MATCHER_NO_LAYER,
	},

Which feels wrong.

@SkohTV
SkohTV requested a review from qdeslandes September 29, 2026 14:38
@qdeslandes

Copy link
Copy Markdown
Contributor

@claude review

@github-actions

github-actions Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

Claude review of PR #346 (20c5847)

Must fix

  • bf_limit_free is a no-op (memory leak) — src/libbpfilter/limit.c:72 — the function never frees the calloc'd struct, breaking cleanup-attribute and list-destructor paths
  • bf_chain_free missing limits cleanup — src/libbpfilter/chain.c:407 — cleans sets and rules but not the new limits list
  • Out-of-bounds read: uint64_t from uint32_t payload — src/libbpfilter/cgen/matcher/meta.c:124 — reads 8 bytes from a 4-byte matcher payload buffer
  • NULL dereference from bf_list_get_at — src/libbpfilter/cgen/matcher/meta.c:126 — return value is dereferenced without a NULL check
  • strtoul validation ineffective — src/libbpfilter/limit.c:50 — overflow undetected due to truncation to uint32_t, dead range checks, wrong errno (-ENOMEM instead of -EINVAL)

Suggestions

  • Unconditional limit map loading — src/libbpfilter/cgen/program.c:1270 — creates a BPF map for every chain even when no rules use meta.limit
  • Unused _raw_payload strdup — src/libbpfilter/limit.c:57 — allocated and never read, dead code from set-parsing copy
  • Unused pstr variable — src/libbpfilter/cgen/program.c:1062 — leftover from copy-paste of printer map loader

Nits

  • Misleading field comment — src/libbpfilter/cgen/runtime.h:194 — says "Current timestamp" but field is a packet counter
  • Unrelated comment regression — src/libbpfilter/matcher.c:51 — generic matcher docstring changed to "bf_matcher_limit" by accident
  • Unnecessary includes in limit.h — src/libbpfilter/include/bpfilter/limit.h:12 — hashset.h, dump.h, matcher.h, stdbool.h, stddef.h are unused

Workflow run

Comment thread src/libbpfilter/limit.c
return 0;
}

void bf_limit_free(struct bf_ratelimit **ratelimit)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude: must-fix: bf_limit_free is a no-op — it casts away the parameter and never frees the calloc'd struct. This means:

  1. Every _free_bf_limit_-annotated local that goes out of scope on an error path leaks.
  2. bf_list_clean on a chain's limits list (which uses bf_limit_free as the free callback) leaks every node's data.

Every other _free function in the codebase follows the established contract (see bf_set_free in set.c, bf_rule_free in rule.c). Fix:

void bf_limit_free(struct bf_ratelimit **ratelimit)
{
    assert(ratelimit);

    if (!*ratelimit)
        return;

    BF_FREEP(ratelimit);
}

assert(program);
assert(matcher);

uint32_t key = *(uint64_t *)bf_matcher_payload(matcher);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude: must-fix: Out-of-bounds read: the payload was stored as uint32_t (4 bytes) in the parser (sizeof(limit_id)), but here it is read through a uint64_t *, reading 8 bytes from a 4-byte buffer. This is undefined behavior.

Every other meta matcher in this file casts to the type that was stored (e.g., *(uint32_t *)bf_matcher_payload(matcher) for iface at line 40). Fix:

uint32_t key = *(uint32_t *)bf_matcher_payload(matcher);


uint32_t key = *(uint64_t *)bf_matcher_payload(matcher);
struct bf_ratelimit *node =
bf_list_get_at(&program->runtime.chain->limits, key);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude: must-fix: bf_list_get_at returns NULL when the index is out of bounds (see core/list.c:222), but node is dereferenced on line 128 without a NULL check. The existing pattern in chain.c:55-58 guards against this:

set = bf_list_get_at(&chain->sets, set_index);
if (!set)
    return bf_err_r(-EINVAL, "rule %u references non-existent set", ...);

Add a similar guard here and return an appropriate error.

Comment thread src/libbpfilter/limit.c
assert(ratelimit);
assert(raw_payload);

limit = strtoul(raw_payload, &endptr, BF_BASE_10);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude: must-fix: The strtoul result validation has several issues:

  1. limit is uint32_t but strtoul returns unsigned long (64-bit on LP64). The return value is silently truncated before the range check, so overflow is undetected.
  2. limit <= 0 on an unsigned type is equivalent to limit == 0 — the < part is dead.
  3. limit > UINT32_MAX is always false for a uint32_t.
  4. errno is never checked for ERANGE.
  5. No check for trailing characters after 's' (e.g., "10/sfoo" would pass).
  6. Line 54 returns -ENOMEM but this is a validation error — should be -EINVAL.

The existing pattern in matcher.c uses errno = 0 before the call and checks errno == ERANGE afterward. Use an unsigned long intermediate and validate the range before narrowing:

errno = 0;
unsigned long parsed = strtoul(raw_payload, &endptr, BF_BASE_10);
if (errno == ERANGE || parsed == 0 || parsed > UINT32_MAX ||
    endptr[0] != '/' || endptr[1] != 's' || endptr[2] != '\0')
    return bf_err_r(-EINVAL, "limit is not valid '%s'", raw_payload);
limit = (uint32_t)parsed;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check how we parse integers for matchers, that logic could be reused.

if (r)
return bf_err_r(r, "failed to load the state map");

r = _bf_program_load_limit_map(prog);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude: suggestion: _bf_program_load_limit_map is called unconditionally for every program, even when no rules use meta.limit. This creates an unnecessary BPF array map and wastes a file descriptor for every chain.

Other optional maps guard on chain state before creating (e.g., _bf_program_load_log_map returns early when BF_CHAIN_LOG is unset). Consider returning 0 early when the chain has no limits:

if (bf_list_is_empty(&program->runtime.chain->limits))
    return 0;

Comment thread src/libbpfilter/limit.c
return bf_err_r(-ENOMEM, "limit is not valid '%s'", raw_payload);
}

_raw_payload = strdup(raw_payload);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude: suggestion: _raw_payload is strdup'd but never read — the parsed limit value is used directly. The allocation is cleaned up by _cleanup_free_ so there is no leak, but the strdup and its error path are dead code. This appears copied from bf_set_new_from_raw where the duplicate is actually tokenized. Remove lines 40 and 57-60.


static int _bf_program_load_limit_map(struct bf_program *program)
{
_cleanup_free_ void *pstr = NULL;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude: suggestion: _cleanup_free_ void *pstr = NULL; is declared but never used. This was likely copied from _bf_program_load_printer_map where pstr holds the assembled printer output. Remove it.

*/
struct bf_ratelimit_data
{
/** Current timestamp (at the last call of the BPF) */

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude: nit: The Doxygen comment says "Current timestamp (at the last call of the BPF)" but the field is a packet counter (ratelimit->current++ in limit.bpf.c), not a timestamp. Consider: "Number of packets seen in the current rate-limit window."

Comment thread src/libbpfilter/matcher.c
* Matcher definition.
*
* Matchers are criterias to match the packet against. A set of matcher defines
* Matchers are criterias to match the packet against. A set of bf_matcher_limit defines

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude: nit: Unrelated comment change: "A set of matcher defines" was changed to "A set of bf_matcher_limit defines". This Doxygen block documents the general bf_matcher concept, not rate limiting. Revert to the original wording.

#include <stddef.h>
#include <stdint.h>

#include <bpfilter/core/hashset.h>

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude: nit: Several includes are unused in this header: <bpfilter/core/hashset.h>, <bpfilter/dump.h>, <bpfilter/matcher.h>, <stdbool.h>, and <stddef.h>. Only <stdint.h> (for uint32_t) and <bpfilter/pack.h> (for bf_rpack_node_t, bf_wpack_t) are needed.

@qdeslandes qdeslandes left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The approach here is sound, but there are some minor fixes to do:

  • Fix review comments and Claude comments
  • Create atomic commits
  • Add tests

Comment thread src/bfcli/parser.y
uint32_t limit_id = bf_list_size(&ruleset->limits);
int r;

struct bf_ratelimit *limit = NULL;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Group it with other variables.

r = bf_map_new(&program->handle->rmap, _BF_LIMIT_MAP_NAME,
BF_MAP_TYPE_LIMIT, sizeof(uint32_t),
sizeof(struct bf_ratelimit_data),
bf_max(1, program->runtime.chain->limits.len));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bf_max(1, program->runtime.chain->limits.len) I assume this it to prevent errors when the limits map is empty. If so, the map should not be created.

Comment thread src/libbpfilter/limit.c
Comment on lines +39 to +45
_free_bf_limit_ struct bf_ratelimit *_ratelimit = NULL;
_cleanup_free_ char *_raw_payload = NULL;

char *endptr;
uint32_t limit;
uint32_t duration;
int r;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Group variables definition.

Comment thread src/libbpfilter/limit.c
assert(ratelimit);
assert(raw_payload);

limit = strtoul(raw_payload, &endptr, BF_BASE_10);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check how we parse integers for matchers, that logic could be reused.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Traffic rate limiting

3 participants