2 @unnumbered Synchronization protocol
4 So-called synchronization protocol (SP) is used in current TCP daemon's
5 implementation. It is used for synchronizing @ref{Spool, spool}
6 directory contents between two nodes.
8 It is aimed to be very simple and effective. It uses reliable transport
9 like TCP connections. It must be effective both on single-duplex and
10 full-duplex links: for example satellites have very high throughput but
11 high-delay links, so acknowledging of each received packet, like
12 @url{https://en.wikipedia.org/wiki/XMODEM, XMODEM} does, causes
13 unacceptable performance degradation.
16 @url{http://noiseprotocol.org/noise.html#interactive-patterns,
17 @code{Noise_IK_25519_ChaChaPoly_BLAKE2b}} protocol. Each Noise packet
18 is sent inside an @url{https://tools.ietf.org/html/rfc4506, XDR} envelope:
26 @multitable @columnfractions 0.2 0.3 0.5
27 @headitem @tab XDR type @tab Value
28 @item Magic number @tab
29 8-byte, fixed length opaque data @tab
30 @verb{|N N C P S 0x00 0x00 0x01|}
32 variable length opaque data @tab
36 Peers static keys are specified as @ref{Configuration, @emph{noisepub}}
39 Payload inside Noise packets has maximum size of @emph{64 KiB - 256 B =
40 65280 B}. It is sent immediately in the first message by each side. The
41 very first payload (that is carried inside handshake messages) is always
42 padded to the maximum size with @emph{HALT} packets (read below), for
43 hiding actual number of @emph{INFO} packets (number of files available
46 Each SP payload is a concatenation of SP packets. Each packet has
47 XDR-encoded header and then corresponding XDR-encoded body. Header is
48 just an unsigned integer telling what body structure follows.
53 Stop file transmission, empty sending queue on the remote side.
54 Actually @emph{HALT} packet does not have any body, only the header
55 with the type. It is also used in the first payload for padding to
65 Information about the file we have for transmission.
68 +------+--------------------+
69 | INFO | NICE | SIZE | HASH |
70 +------+--------------------+
73 @multitable @columnfractions 0.2 0.3 0.5
74 @headitem @tab XDR type @tab Value
77 1-255, file niceness level
79 unsigned hyper integer @tab
82 32-byte, fixed length opaque data @tab
83 Unique file identifier, its checksum
87 File transmission request. Ask remote side to queue the file for
91 +------+---------------+
92 | FREQ | HASH | OFFSET |
93 +------+---------------+
96 @multitable @columnfractions 0.2 0.3 0.5
97 @headitem @tab XDR type @tab Value
99 32-byte, fixed length opaque data @tab
100 Unique file identifier, its checksum
102 unsigned hyper integer @tab
103 Offset from which remote side must transmit the file
110 +------+-------------------------+
111 | FILE | HASH | OFFSET | PAYLOAD |
112 +------+-------------------------+
115 @multitable @columnfractions 0.2 0.3 0.5
116 @headitem @tab XDR type @tab Value
118 32-byte, fixed length opaque data @tab
119 Unique file identifier, its checksum
121 unsigned hyper integer @tab
122 Offset from which transmission goes
124 variable length opaque data @tab
129 Signal remote side that we have successfully downloaded the file.
137 @multitable @columnfractions 0.2 0.3 0.5
138 @headitem @tab XDR type @tab Value
140 32-byte, fixed length opaque data @tab
141 Unique file identifier, its checksum
146 Typical peer's behaviour is following:
148 @verbatiminclude sp.utxt
151 @item Perform @emph{Noise-IK} handshake:
155 Collects all @emph{tx}-related files information and prepares
156 payload filled with @emph{INFO}s for including in the @strong{first}
159 After receiving the first handshake message, it gains remote
160 identity knowledge and similarly prepares the payload for including
161 in the @strong{second} handshake message.
164 All payloads are padded to maximal message size with @emph{HALT}s.
166 @item If queued @emph{INFO}s are not sent completely in handshake
167 payloads, then send all of remaining in the transport stage.
169 @item When @emph{INFO} packet received:
172 @item Check that it has an acceptable niceness level.
173 Ignore it if it is too nice.
174 @item If already downloaded file exists, then queue @emph{DONE}
176 @item If @file{.seen} exists, then queue @emph{DONE} sending.
177 @item If @file{.part} exists, then queue @emph{FREQ} sending with
178 corresponding offset.
181 @item When @emph{FREQ} packet received, insert it to current sending
182 queue with niceness level sort: higher priority packets will be sent
183 first. Sending queue contains files with offsets that are needed to be
186 @item While sending queue is not empty, send @emph{FILE} packets.
187 @emph{FREQ} could contain offset equal to size -- anyway sent
188 @emph{FILE} packet with an empty payload. @emph{FILE} sending is
189 performed only if no other outgoing packets are queued: @emph{INFO}s
190 have higher priority.
192 @item When @emph{FILE} packet received, check if it is completely
193 downloaded (comparing to @emph{INFO}'s packet size information). If so,
194 then run background integrity checker on it. If check succeeds, then
195 delete @file{.part} suffix from file's name and send @emph{DONE} packet.
197 @item When @emph{DONE} packet received, delete corresponding file.
198 @item When @emph{HALT} packet received, empty file sending queue.
200 @item Each second, node checks: are there any new @emph{tx} packets
201 appeared and queues corresponding @emph{INFO} packets.
203 @item If no packets are sent and received during @ref{CfgOnlineDeadline,
204 onlinedeadline} duration, then close the connection. There is no
205 explicit indication that session is over.