问题描述
在阅读TCP源代码时，我发现一个困惑的事情:
When reading TCP source code, I find a confused thing:
我知道TCP通过3种方式的握手有两个队列:
I know TCP has two queues in 3 way handshake:
- 第一个队列存储服务器已收到
SYN并发送回ACK + SYN的连接，我们将其称为 syn队列. - 第二个队列存储3WHS成功建立的连接，我们将其称为接受队列.
- The first queue stores connections which server has received the
SYNand send backACK + SYN, which we call as syn queue. - The second queue stores the connections that 3WHS are successful and connection established, which we call as accept queue.
但是在阅读代码时，我发现 listen()将调用 inet_csk_listen_start()，后者将调用 reqsk_queue_alloc()创建> icsk_accept_queue .并且该队列在 accept()中使用，当我们发现队列不为空时，我们将从中获得连接并返回.
But when reading codes, I find listen() will call inet_csk_listen_start(), which will call reqsk_queue_alloc() to create icsk_accept_queue. And that queue is used in accept(), when we find the queue is not empty, we will get a connection from it and return.
此外，在跟踪接收过程之后，调用堆栈就像
What's more, after tracing the receive process, the call stack is like
tcp_v4_rcv()->tcp_v4_do_rcv()->tcp_rcv_state_process()
在接收到第一次握手时，服务器状态为LISTEN.因此它将调用
The server status is LISTEN when receiving the first handshake. So it will call
`tcp_v4_conn_request()->tcp_conn_request()`
在 tcp_conn_request()
if (!want_cookie)
// Add the req into the queue
inet_csk_reqsk_queue_hash_add(sk, req, tcp_timeout_init((struct sock *)req));
但是这里的队列恰好是 icsk_accept_queue ，而不是syn队列.
But here the queue is exactly the icsk_accept_queue, not a syn queue.
void inet_csk_reqsk_queue_hash_add(struct sock *sk, struct request_sock *req,
unsigned long timeout)
{
reqsk_queue_hash_req(req, timeout);
inet_csk_reqsk_queue_added(sk);
}
static inline void inet_csk_reqsk_queue_added(struct sock *sk)
{
reqsk_queue_added(&inet_csk(sk)->icsk_accept_queue);
}
accept()将返回已建立的连接，这意味着 icsk_accept_queue 是第二个队列，但是第一个队列在哪里?
The accept() will return the established connection, which means icsk_accept_queue is the second queue, but where is the first queue?
连接从第一个队列到第二个队列在哪里更改?
Where does the connection changes from the first queue to the second?
为什么Linux将新的请求添加到 icsk_accept_queue ?
Why does the Linux add new req into icsk_accept_queue?
推荐答案
在下面的内容中，我们将遵循最典型的代码路径，并且将忽略由数据包丢失，重传以及使用非典型功能(例如TCP快速打开)引起的问题(代码注释中的TFO).
In what follows we will follow the most typical code path and will ignore issues arising from packet loss, retransmission, and the use of atypical features such as TCP fast open (TFO in the code comments).
接受调用由 intet_csk_accept 处理，该调用调用 reqsk_queue_remove 以使套接字脱离接受队列& icsk-> icsk_accept_queue :
The call to accept is processed by intet_csk_accept, which calls reqsk_queue_remove to get a socket out of the accept queue &icsk->icsk_accept_queue from the listening socket:
struct sock *inet_csk_accept(struct sock *sk, int flags, int *err, bool kern)
{
struct inet_connection_sock *icsk = inet_csk(sk);
struct request_sock_queue *queue = &icsk->icsk_accept_queue;
struct request_sock *req;
struct sock *newsk;
int error;
lock_sock(sk);
[...]
req = reqsk_queue_remove(queue, sk);
newsk = req->sk;
[...]
return newsk;
[...]
}
在 reqsk_queue_remove 中，它使用 rskq_accept_head 和 rskq_accept_tail 将套接字拉出队列并调用 sk_acceptq_removed :
In reqsk_queue_remove, it uses rskq_accept_head and rskq_accept_tail to pull a socket out of the queue and to call sk_acceptq_removed:
static inline struct request_sock *reqsk_queue_remove(struct request_sock_queue *queue,
struct sock *parent)
{
struct request_sock *req;
spin_lock_bh(&queue->rskq_lock);
req = queue->rskq_accept_head;
if (req) {
sk_acceptq_removed(parent);
WRITE_ONCE(queue->rskq_accept_head, req->dl_next);
if (queue->rskq_accept_head == NULL)
queue->rskq_accept_tail = NULL;
}
spin_unlock_bh(&queue->rskq_lock);
return req;
}
sk_acceptq_removed 减少了等待 sk_ack_backlog 接受的套接字队列的长度:
And sk_acceptq_removed reduces the length of the queue of sockets waiting to be accepted in sk_ack_backlog:
static inline void sk_acceptq_removed(struct sock *sk)
{
WRITE_ONCE(sk->sk_ack_backlog, sk->sk_ack_backlog - 1);
}
我认为，发问者已经完全理解了这一点.现在，让我们看一下接收到SYN以及3WH的最终ACK到达时发生的情况.
This, I think, is fully understood by the questioner. Now let's look at what happens when a SYN is recieved, and when the final ACK of the 3WH arrives.
首先收到SYN.再次，让我们假设TFO和SYN Cookie不在起作用，而是查看最常见的路径(至少在出现SYN泛滥时不会).
First the receipt of the SYN. Again, let's assume that TFO and SYN cookies are not in play and look at the most common path (at least not when there is a SYN flood).
通过调用 inet_csk_reqsk_queue_hash_add 然后调用 send_synack 来响应SYN:
The SYN is processed in tcp_conn_request where the connection request (not a full blown socket) is stored (we shall see where soon) by calling inet_csk_reqsk_queue_hash_add and then calling send_synack to respond to the SYN:
int tcp_conn_request(struct request_sock_ops *rsk_ops,
const struct tcp_request_sock_ops *af_ops,
struct sock *sk, struct sk_buff *skb)
{
[...]
if (!want_cookie)
inet_csk_reqsk_queue_hash_add(sk, req,
tcp_timeout_init((struct sock *)req));
af_ops->send_synack(sk, dst, &fl, req, &foc,
!want_cookie ? TCP_SYNACK_NORMAL :
TCP_SYNACK_COOKIE);
[...]
return 0;
[...]
}
inet_csk_reqsk_queue_hash_add 调用 reqsk_queue_hash_req 和 inet_csk_reqsk_queue_added 来存储请求.
void inet_csk_reqsk_queue_hash_add(struct sock *sk, struct request_sock *req,
unsigned long timeout)
{
reqsk_queue_hash_req(req, timeout);
inet_csk_reqsk_queue_added(sk);
}
reqsk_queue_hash_req 将请求放入ashash .
static void reqsk_queue_hash_req(struct request_sock *req,
unsigned long timeout)
{
[...]
inet_ehash_insert(req_to_sk(req), NULL);
[...]
}
然后 inet_csk_reqsk_queue_added 用 icsk_accept_queue 调用 reqsk_queue_added :
static inline void inet_csk_reqsk_queue_added(struct sock *sk)
{
reqsk_queue_added(&inet_csk(sk)->icsk_accept_queue);
}
增加 qlen (而不是 sk_ack_backlog ):
static inline void reqsk_queue_added(struct request_sock_queue *queue)
{
atomic_inc(&queue->young);
atomic_inc(&queue->qlen);
}
ehash是已存储所有ESTABLISHED和TIMEWAIT套接字的位置，而最近是SYN队列"存储的位置.已存储.
The ehash is where all the ESTABLISHED and TIMEWAIT sockets have been stored, and, more recently, where the SYN "queue" is stored.
请注意，将到达的连接请求存储在适当的队列中实际上没有任何目的.它们的顺序无关紧要(最终的ACK可以以任何顺序到达)，通过将它们移出侦听套接字，就不必对侦听套接字进行锁定来处理最终的ACK.
Note that there is actually no purpose in storing the arrived connection requests in a proper queue. Their order is irrelevant (the final ACKs can arrive in any order) and by moving them out of the listening socket, it is not necessary to take a lock on the listening socket to process the final ACK.
有关实现此更改的代码，请参见此提交.
See this commit for the code that effected this change.
最后，我们可以看到请求从 ehash 中删除，并作为一个完整的套接字添加到接受队列中.
Finally, we can watch the request get removed from ehash and added as a full socket to the accept queue.
3WH的最终ACK由 tcp_check_req 处理，它将创建一个完整的子套接字，然后调用 inet_csk_complete_hashdance :
The final ACK of the 3WH is processed by tcp_check_req which creates a full child socket and then calls inet_csk_complete_hashdance:
struct sock *tcp_check_req(struct sock *sk, struct sk_buff *skb,
struct request_sock *req,
bool fastopen, bool *req_stolen)
{
[...]
/* OK, ACK is valid, create big socket and
* feed this segment to it. It will repeat all
* the tests. THIS SEGMENT MUST MOVE SOCKET TO
* ESTABLISHED STATE. If it will be dropped after
* socket is created, wait for troubles.
*/
child = inet_csk(sk)->icsk_af_ops->syn_recv_sock(sk, skb, req, NULL,
req, &own_req);
[...]
return inet_csk_complete_hashdance(sk, child, req, own_req);
[...]
}
然后， inet_csk_complete_hashdance 在请求上调用 inet_csk_reqsk_queue_drop 和 reqsk_queue_removed ，并在子级上调用 inet_csk_reqsk_queue_add :
Then inet_csk_complete_hashdance calls inet_csk_reqsk_queue_drop and reqsk_queue_removed on the request, and inet_csk_reqsk_queue_add on the child:
struct sock *inet_csk_complete_hashdance(struct sock *sk, struct sock *child,
struct request_sock *req, bool own_req)
{
if (own_req) {
inet_csk_reqsk_queue_drop(sk, req);
reqsk_queue_removed(&inet_csk(sk)->icsk_accept_queue, req);
if (inet_csk_reqsk_queue_add(sk, req, child))
return child;
}
[...]
}
inet_csk_reqsk_queue_drop 调用 reqsk_queue_unlink (这会从ehash中删除请求)，以及 reqsk_queue_removed 会降低qlen:
inet_csk_reqsk_queue_drop calls reqsk_queue_unlink, which removes the request from the ehash, and reqsk_queue_removed which decrements the qlen:
void inet_csk_reqsk_queue_drop(struct sock *sk, struct request_sock *req)
{
if (reqsk_queue_unlink(req)) {
reqsk_queue_removed(&inet_csk(sk)->icsk_accept_queue, req);
reqsk_put(req);
}
}
最后， inet_csk_reqsk_queue_add 将完整的套接字添加到接受队列.
Finally, inet_csk_reqsk_queue_add adds the full socket to the accept queue.
struct sock *inet_csk_reqsk_queue_add(struct sock *sk,
struct request_sock *req,
struct sock *child)
{
struct request_sock_queue *queue = &inet_csk(sk)->icsk_accept_queue;
spin_lock(&queue->rskq_lock);
if (unlikely(sk->sk_state != TCP_LISTEN)) {
inet_child_forget(sk, req, child);
child = NULL;
} else {
req->sk = child;
req->dl_next = NULL;
if (queue->rskq_accept_head == NULL)
WRITE_ONCE(queue->rskq_accept_head, req);
else
queue->rskq_accept_tail->dl_next = req;
queue->rskq_accept_tail = req;
sk_acceptq_added(sk);
}
spin_unlock(&queue->rskq_lock);
return child;
}
TL; DR它在ehash中，此类SYN的数量为 qlen (而不是 sk_ack_backlog ，后者包含接受队列中的套接字数量)
TL;DR it is in the ehash, and the number of such SYNs is qlen (and not sk_ack_backlog, which holds the number of sockets in the accept queue).
这篇关于关于同步队列和接受队列的困惑的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！