2 @unnumbered Synchronization protocol
4 So-called synchronization protocol (SP) is used in current TCP daemon's
5 implementation. It is used for synchronizing @ref{Spool, spool}
6 directory contents between two nodes.
8 It is aimed to be very simple and effective. It uses reliable transport
9 like TCP connections. It must be effective both on single-duplex and
10 full-duplex links: for example satellites have very high throughput but
11 high-delay links, so acknowledging of each received packet, like
12 @url{https://en.wikipedia.org/wiki/XMODEM, XMODEM} does, causes
13 unacceptable performance degradation.
16 @url{http://noiseprotocol.org/noise.html#interactive-patterns,
17 @code{Noise_IK_25519_ChaChaPoly_BLAKE2b}} protocol. Each Noise packet
18 is sent inside an @url{https://tools.ietf.org/html/rfc4506, XDR} envelope:
26 @multitable @columnfractions 0.2 0.3 0.5
27 @headitem @tab XDR type @tab Value
28 @item Magic number @tab
29 8-byte, fixed length opaque data @tab
30 @verb{|N N C P S 0x00 0x00 0x01|}
32 variable length opaque data @tab
36 Peers static keys are specified as @ref{Configuration, @emph{noisepub}}
39 Payload inside Noise packets has maximum size of @emph{64 KiB - 256 B =
40 65280 B}. It is sent immediately in the first message by each side. The
41 very first payload (that is carried inside handshake messages) is always
42 padded to the maximum size with @emph{HALT} packets (read below), for
43 hiding actual number of @emph{INFO} packets (number of files available
46 Each SP payload is a concatenation of SP packets. Each packet has
47 XDR-encoded header and then corresponding XDR-encoded body. Header is
48 just an unsigned integer telling what body structure follows.
53 Stop file transmission, empty sending queue on the remote side.
54 Actually @emph{HALT} packet does not have any body, only the header
55 with the type. It is also used in the first payload for padding to
65 Dummy packet only used for determining workability of the connection.
74 Information about the file we have for transmission.
77 +------+--------------------+
78 | INFO | NICE | SIZE | HASH |
79 +------+--------------------+
82 @multitable @columnfractions 0.2 0.3 0.5
83 @headitem @tab XDR type @tab Value
86 1-255, file niceness level
88 unsigned hyper integer @tab
91 32-byte, fixed length opaque data @tab
92 Unique file identifier, its checksum
96 File transmission request. Ask remote side to queue the file for
100 +------+---------------+
101 | FREQ | HASH | OFFSET |
102 +------+---------------+
105 @multitable @columnfractions 0.2 0.3 0.5
106 @headitem @tab XDR type @tab Value
108 32-byte, fixed length opaque data @tab
109 Unique file identifier, its checksum
111 unsigned hyper integer @tab
112 Offset from which remote side must transmit the file
119 +------+-------------------------+
120 | FILE | HASH | OFFSET | PAYLOAD |
121 +------+-------------------------+
124 @multitable @columnfractions 0.2 0.3 0.5
125 @headitem @tab XDR type @tab Value
127 32-byte, fixed length opaque data @tab
128 Unique file identifier, its checksum
130 unsigned hyper integer @tab
131 Offset from which transmission goes
133 variable length opaque data @tab
138 Signal remote side that we have successfully downloaded the file.
146 @multitable @columnfractions 0.2 0.3 0.5
147 @headitem @tab XDR type @tab Value
149 32-byte, fixed length opaque data @tab
150 Unique file identifier, its checksum
155 Typical peer's behaviour is following:
157 @verbatiminclude sp.plantuml.txt
160 @item Perform @emph{Noise-IK} handshake:
164 Collects all @emph{tx}-related files information and prepares
165 payload filled with @emph{INFO}s for including in the @strong{first}
168 After receiving the first handshake message, it gains remote
169 identity knowledge and similarly prepares the payload for including
170 in the @strong{second} handshake message.
173 All payloads are padded to maximal message size with @emph{HALT}s.
175 @item If queued @emph{INFO}s are not sent completely in handshake
176 payloads, then send all of remaining in the transport stage.
178 @item When @emph{INFO} packet received:
181 @item Check that it has an acceptable niceness level.
182 Ignore it if it is too nice.
183 @item If already downloaded file exists, then queue @emph{DONE}
185 @item If @file{.seen} exists, then queue @emph{DONE} sending.
186 @item If @file{.part} exists, then queue @emph{FREQ} sending with
187 corresponding offset.
190 @item When @emph{FREQ} packet received, insert it to current sending
191 queue with niceness level sort: higher priority packets will be sent
192 first. Sending queue contains files with offsets that are needed to be
195 @item While sending queue is not empty, send @emph{FILE} packets.
196 @emph{FREQ} could contain offset equal to size -- anyway sent
197 @emph{FILE} packet with an empty payload. @emph{FILE} sending is
198 performed only if no other outgoing packets are queued: @emph{INFO}s
199 have higher priority.
201 @item When @emph{FILE} packet received, check if it is completely
202 downloaded (comparing to @emph{INFO}'s packet size information). If so,
203 then run background integrity checker on it. If check succeeds, then
204 delete @file{.part} suffix from file's name and send @emph{DONE} packet.
206 @item When @emph{DONE} packet received, delete corresponding file.
208 @item When @emph{HALT} packet received, empty file sending queue.
210 @item Each second, node checks: are there any new @emph{tx} packets
211 appeared and queues corresponding @emph{INFO} packets.
213 @item Each minute, if no packets were sent, node sends @emph{PING}
216 @item If no non-PING packets are sent and received during
217 @ref{CfgOnlineDeadline, onlinedeadline} duration, then close the
218 connection. There is no explicit indication that session is over.
220 @item If no packets are received during two minutes (two PING timeouts),
221 then close the connection.