Mekhanik evgenii/fix 1346 1 #2456
Open
+1,233
−737
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
EvgeniiMekhanik
force-pushed
the
MekhanikEvgenii/fix-1346-1
branch
from
a62787c to
c9bba36
Compare
EvgeniiMekhanik
commented
Jul 3, 2025
fw/ss_skb.c
Outdated
| __u8 pfmemalloc = skb->pfmemalloc; | ||
|
|
||
| WARN_ON_ONCE(skb->sk); | ||
| skb_orphan(skb); |
Contributor
Author
There was a problem hiding this comment.
Please pay attention on this place. Here we release skb owner and decrease client->mem. This function ss_skb_init_for_xmit is called before push skb to the socket write queue. So all skbs in socket write queue are not taken into account for client memory calculation. We release skb owner here, because if don't do it we need a rather big kernel patch to adjust skb memory before it will be passed to socket write queue. @krizhanovsky @const-t what do you think about it?
Contributor
There was a problem hiding this comment.
Why don't we make a pointer to a client accounting in skb->cb instead of to play with skb_orphan()? I'd prefer to avoid this since we can get plenty of crashes in this patch or in later kernel version migrations due to breaking kernel logic about orphaned skbs.
EvgeniiMekhanik
force-pushed
the
MekhanikEvgenii/fix-1346-1
branch
from
c9bba36 to
26e3525
Compare
const-t
reviewed
Jul 8, 2025
EvgeniiMekhanik
force-pushed
the
MekhanikEvgenii/fix-1346-1
branch
16 times, most recently
from
40654f8 to
e2de424
Compare
EvgeniiMekhanik
force-pushed
the
MekhanikEvgenii/fix-1346-1
branch
5 times, most recently
from
7b5e367 to
ac06de7
Compare
EvgeniiMekhanik
force-pushed
the
MekhanikEvgenii/fix-1346-1
branch
from
4ea17b3 to
dedff25
Compare
client_mem <soft_limit> <hard_limit> - controls haw many memory is used to store unanswered client requests and requests with linked responses which can not be forwarded to a client.
To track socket memory we should pass TfwHttpMsg * not TfwMsgIter * to most of http_nsg_* functions, because TfwHttpMsg has a pointer to connection and socket.
In task #498 we decide to use `client_mem` option to limit count of memory used by client. This commit is a part of this task - now Tempesta FW uses `sk->sk_rmem_alloc` to adjust memory used by Tempesta FW for this client connection.
In task #498 we decide to use `client_mem` option to limit count of memory used by client. This commit is a part of this task and the next step of implementaion. Previosly Tempesta FW uses `sk->sk_rmem_alloc` to adjust memory used by Tempesta FW for this client connection, now we adjust memory for the whole TfwClient, because the can be a lot of connection for one client and for all other cases we use limitation for TfwClient and block it if necessary.
If administrator specify `client_mem` and the memory used by all connection of current client exceeded this value Tempesta FW drops connection and block client by ip if `ip_block on;` is specified.
Previosuly we get connection when we adjust memory for skb, but it leads to several problems: - we can't adjust memory for skb before tls decryption, because skb from `tls->io_in.skb_list` are freed during connection released (but connection will be never released if we increment it's reference counter for these skbs). - We have the same problems for skbs, which are wait for appropriate tcp window to be pushed in socket write queue. Now we increment/decrement reference counter for TfwClient and adjust skb memory for requests before tls decryption.
Previously we adjust tcp send window only for http2 connection and only during making HEADER or DATA frames, but if we want to control client memory usage we should do it for all type of sending data. (We orphane skb and decrease memory usage when we pass skb to the socket write queue, so we we don't adjust tcp send window we push a lot of skbs in socket write queue and don't adjust it's memory).
- remove `client_get_light/client_put_light` functions, because after removing lock from `client` structure we don't need these functions at all. - Adjust memory usage of skb in `skb->cb`. Usually it is equal to `skb->truesize, but for some cases ( skb which was created by `pskb_copy_for_clone` for example it is different).
EvgeniiMekhanik
force-pushed
the
MekhanikEvgenii/fix-1346-1
branch
4 times, most recently
from
2ad5039 to
33eacfa
Compare
- use `skb_shift` instead of `skb_try_coalesce` to correctly adjust send window during entailing skb to socket write queue. - adjust FRAME_HEADER_SIZE during calculation send window during making frames. (There was a mistake with accuracy of send window calculation, we don't take into account, that each frame also contains frame header). - change BUG_ON to WARN_ON.
- rename tfw_cli_*_limit to tfw_cli_*_mem_limit
- rename `ss_skb_can_collapce` to `ss_skb_can_collapse`
- rename `tfw_h2_or_stream_wnd_is_exceeded` to
`tfw_h2_conn_or_stream_wnd_is_exceeded`.
- move braces `{` to the next line.
- rename `ss_skb_adjust_sk_mem` to `ss_skb_adjust_client_mem`.
- Do not dublicate code for http1 and http2 in
`tfw_connection_push`.
- Change BUG_ON to WARN_ON in some places.
We should not call `tcp_skb_pcount_set` during skb splitting, if skb belongs to Tempesta (not pushed to socket write queue), because we use skb->cb for our own purposes. And `ss_skb_split` never called from `tfw_tls_encrypt` (`tfw_tls_encrypt` is only one function which is called from Tempesta FW code, when skb is already pushed to socket write queue), so we can totally removed `tcp_skb_pcount_set`.
Do not use `skb->sk` and `skb->destructor` to check memory used by skb, use `skb->cb` for this purposes. - Implement our own version of `skb_orphan` with name `ss_skb_orphan` which is called when skb is freed in Tempesta FW code our just before pushing skb to socket write queue. - Implement wrappers over `__kfree_skb` and `kfree_skb` where we call `ss_skb_orphan` before free skb. - Check that skb is pushed to socket write queue, using new ipmlemented function `skb_tfw_is_in_socket_write_queue` from linux kernel, to skip adjusting memory used be skb, when it belongs to kernel (when `ss_skb_*` functions called from `tls_encrypt`).
EvgeniiMekhanik
force-pushed
the
MekhanikEvgenii/fix-1346-1
branch
from
33eacfa to
72b72c4
Compare
- Usually we use callbacks which are set in `skb->cb` for different purposes. So remove to callbacks, which was added in previous patches and use callbacks saved in `skb->cb`.
EvgeniiMekhanik
force-pushed
the
MekhanikEvgenii/fix-1346-1
branch
from
fc229fb to
443d7da
Compare
- Since we use pool for http memory allocation, change api of all `tfw_pool_*` functions to pass `TfwClient` and accounting memory in this structure. - Remove `TfwClient` refcounter (it not used, can be done in previous commits). - Fix unit tests to check memory accounting, cleanup memory after each test, to check that client memory is equal to zero after test.
EvgeniiMekhanik
force-pushed
the
MekhanikEvgenii/fix-1346-1
branch
from
3b27521 to
19362d3
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.