2 @unnumbered Synchronization protocol
4 So-called synchronization protocol (SP) is used in current TCP daemon's
5 implementation. It is used for synchronizing @ref{Spool, spool}
6 directory contents between two nodes.
8 It is aimed to be very simple and effective. It uses reliable transport
9 like TCP connections. It must be effective both on single-duplex and
10 full-duplex links: for example satellites have very high throughput but
11 high-delay links, so acknowledging of each received packet, like
12 @url{https://en.wikipedia.org/wiki/XMODEM, XMODEM} does, causes
13 unacceptable performance degradation.
16 @url{http://noiseprotocol.org/noise.html#interactive-patterns,
17 @code{Noise_IK_25519_ChaChaPoly_BLAKE2b}} protocol. Each Noise packet
18 is sent inside an @url{https://tools.ietf.org/html/rfc4506, XDR} envelope:
26 @multitable @columnfractions 0.2 0.3 0.5
27 @headitem @tab XDR type @tab Value
28 @item Magic number @tab
29 8-byte, fixed length opaque data @tab
30 @verb{|N N C P S 0x00 0x00 0x01|}
32 variable length opaque data @tab
36 Peers static keys are specified as @ref{Configuration, @emph{noisepub}}
39 Payload inside Noise packets has maximum size of @emph{64 KiB - 256 B =
40 65280 B}. It is sent immediately in the first message by each side. The
41 very first payload (that is carried inside handshake messages) is always
42 padded to the maximum size with @emph{HALT} packets (read below), for
43 hiding actual number of @emph{INFO} packets (number of files available
46 Each SP payload is a concatenation of SP packets. Each packet has
47 XDR-encoded header and then corresponding XDR-encoded body. Header is
48 just an unsigned integer telling what body structure follows.
53 Stop file transmission, empty sending queue on the remote side.
54 Actually @emph{HALT} packet does not have any body, only the header
55 with the type. It is also used in the first payload for padding to
64 Information about the file we have for transmission.
66 +------+--------------------+
67 | INFO | NICE | SIZE | HASH |
68 +------+--------------------+
70 @multitable @columnfractions 0.2 0.3 0.5
71 @headitem @tab XDR type @tab Value
74 1-255, file niceness level
76 unsigned hyper integer @tab
79 32-byte, fixed length opaque data @tab
80 Unique file identifier, its checksum
84 File transmission request. Ask remote side to queue the file for
87 +------+---------------+
88 | FREQ | HASH | OFFSET |
89 +------+---------------+
91 @multitable @columnfractions 0.2 0.3 0.5
92 @headitem @tab XDR type @tab Value
94 32-byte, fixed length opaque data @tab
95 Unique file identifier, its checksum
97 unsigned hyper integer @tab
98 Offset from which remote side must transmit the file
104 +------+-------------------------+
105 | FILE | HASH | OFFSET | PAYLOAD |
106 +------+-------------------------+
108 @multitable @columnfractions 0.2 0.3 0.5
109 @headitem @tab XDR type @tab Value
111 32-byte, fixed length opaque data @tab
112 Unique file identifier, its checksum
114 unsigned hyper integer @tab
115 Offset from which transmission goes
117 variable length opaque data @tab
122 Signal remote side that we have successfully downloaded the file.
128 @multitable @columnfractions 0.2 0.3 0.5
129 @headitem @tab XDR type @tab Value
131 32-byte, fixed length opaque data @tab
132 Unique file identifier, its checksum
137 Typical peer's behaviour is following:
139 @verbatiminclude sp.utxt
142 @item Perform @emph{Noise-IK} handshake:
146 Collects all @emph{tx}-related files information and prepares
147 payload filled with @emph{INFO}s for including in the @strong{first}
150 After receiving the first handshake message, it gains remote
151 identity knowledge and similarly prepares the payload for including
152 in the @strong{second} handshake message.
155 All payloads are padded to maximal message size with @emph{HALT}s.
157 @item If queued @emph{INFO}s are not sent completely in handshake
158 payloads, then send all of remaining in the transport stage.
160 @item When @emph{INFO} packet received:
163 @item Check that it has an acceptable niceness level.
164 Ignore it if it is too nice.
165 @item If already downloaded file exists, then queue @emph{DONE}
167 @item If @file{.seen} exists, then queue @emph{DONE} sending.
168 @item If @file{.part} exists, then queue @emph{FREQ} sending with
169 corresponding offset.
172 @item When @emph{FREQ} packet received, insert it to current sending
173 queue with niceness level sort: higher priority packets will be sent
174 first. Sending queue contains files with offsets that are needed to be
177 @item While sending queue is not empty, send @emph{FILE} packets.
178 @emph{FREQ} could contain offset equal to size -- anyway sent
179 @emph{FILE} packet with an empty payload. @emph{FILE} sending is
180 performed only if no other outgoing packets are queued: @emph{INFO}s
181 have higher priority.
183 @item When @emph{FILE} packet received, check if it is completely
184 downloaded (comparing to @emph{INFO}'s packet size information). If so,
185 then run background integrity checker on it. If check succeeds, then
186 delete @file{.part} suffix from file's name and send @emph{DONE} packet.
188 @item When @emph{DONE} packet received, delete corresponding file.
189 @item When @emph{HALT} packet received, empty file sending queue.
191 @item Each second, node checks: are there any new @emph{tx} packets
192 appeared and queues corresponding @emph{INFO} packets.
194 @item If no packets are sent and received during @ref{CfgOnlineDeadline,
195 onlinedeadline} duration, then close the connection. There is no
196 explicit indication that session is over.