3 @cindex online protocol
4 @cindex synchronization
5 @unnumbered Synchronization protocol
7 So-called synchronization protocol (SP) is used in current TCP daemon's
8 implementation. It is used for synchronizing @ref{Spool, spool}
9 directory contents between two nodes.
12 It is aimed to be very simple and effective. It uses reliable transport
13 like TCP connections. It must be effective both on single-duplex and
14 full-duplex links: for example satellites have very high throughput but
15 high-delay links, so acknowledging of each received packet, like
16 @url{https://en.wikipedia.org/wiki/XMODEM, XMODEM} does, causes
17 unacceptable performance degradation.
20 Internally it uses various timeouts and deadlines. One of them used
21 extensively is 10 seconds default deadline timeout. You can override it
22 with @env{$NNCPDEADLINE} environment variable, that could be useful with
23 very high delay links.
27 @url{http://noiseprotocol.org/noise.html#interactive-patterns,
28 @code{Noise_IK_25519_ChaChaPoly_BLAKE2b}} protocol. Each Noise packet
29 is sent inside an @url{https://tools.ietf.org/html/rfc4506, XDR} envelope:
37 @multitable @columnfractions 0.2 0.3 0.5
38 @headitem @tab XDR type @tab Value
39 @item Magic number @tab
40 8-byte, fixed length opaque data @tab
41 @verb{|N N C P S 0x00 0x00 0x01|}
43 variable length opaque data @tab
47 Peers static keys are specified as @ref{Configuration, @emph{noisepub}}
50 Payload inside Noise packets has maximum size of @emph{64 KiB - 256 B =
51 65280 B}. It is sent immediately in the first message by each side. The
52 very first payload (that is carried inside handshake messages) is always
53 padded to the maximum size with @emph{HALT} packets (read below), for
54 hiding actual number of @emph{INFO} packets (number of files available
57 Each SP payload is a concatenation of SP packets. Each packet has
58 XDR-encoded header and then corresponding XDR-encoded body. Header is
59 just an unsigned integer telling what body structure follows.
65 Stop file transmission, empty sending queue on the remote side.
66 Actually @emph{HALT} packet does not have any body, only the header
67 with the type. It is also used in the first payload for padding to
78 Dummy packet only used for determining workability of the connection.
88 Information about the file we have for transmission.
91 +------+--------------------+
92 | INFO | NICE | SIZE | HASH |
93 +------+--------------------+
96 @multitable @columnfractions 0.2 0.3 0.5
97 @headitem @tab XDR type @tab Value
100 1-255, file niceness level
102 unsigned hyper integer @tab
105 32-byte, fixed length opaque data @tab
106 Unique file identifier, its checksum
111 File transmission request. Ask remote side to queue the file for
115 +------+---------------+
116 | FREQ | HASH | OFFSET |
117 +------+---------------+
120 @multitable @columnfractions 0.2 0.3 0.5
121 @headitem @tab XDR type @tab Value
123 32-byte, fixed length opaque data @tab
124 Unique file identifier, its checksum
126 unsigned hyper integer @tab
127 Offset from which remote side must transmit the file
135 +------+-------------------------+
136 | FILE | HASH | OFFSET | PAYLOAD |
137 +------+-------------------------+
140 @multitable @columnfractions 0.2 0.3 0.5
141 @headitem @tab XDR type @tab Value
143 32-byte, fixed length opaque data @tab
144 Unique file identifier, its checksum
146 unsigned hyper integer @tab
147 Offset from which transmission goes
149 variable length opaque data @tab
155 Signal remote side that we have successfully downloaded the file.
163 @multitable @columnfractions 0.2 0.3 0.5
164 @headitem @tab XDR type @tab Value
166 32-byte, fixed length opaque data @tab
167 Unique file identifier, its checksum
172 Typical peer's behaviour is following:
174 @verbatiminclude sp.plantuml.txt
177 @item Perform @emph{Noise-IK} handshake:
181 Collects all @emph{tx}-related files information and prepares
182 payload filled with @emph{INFO}s for including in the @strong{first}
185 After receiving the first handshake message, it gains remote
186 identity knowledge and similarly prepares the payload for including
187 in the @strong{second} handshake message.
190 All payloads are padded to maximal message size with @emph{HALT}s.
192 @item If queued @emph{INFO}s are not sent completely in handshake
193 payloads, then send all of remaining in the transport stage.
195 @item When @emph{INFO} packet received:
198 @item Check that it has an acceptable niceness level.
199 Ignore it if it is too nice.
200 @item If already downloaded file exists, then queue @emph{DONE}
202 @item If @file{seen/XXX} exists, then queue @emph{DONE} sending.
203 @item If @file{.part} exists, then queue @emph{FREQ} sending with
204 corresponding offset.
207 @item When @emph{FREQ} packet received, insert it to current sending
208 queue with niceness level sort: higher priority packets will be sent
209 first. Sending queue contains files with offsets that are needed to be
212 @item While sending queue is not empty, send @emph{FILE} packets.
213 @emph{FREQ} could contain offset equal to size -- anyway sent
214 @emph{FILE} packet with an empty payload. @emph{FILE} sending is
215 performed only if no other outgoing packets are queued: @emph{INFO}s
216 have higher priority.
218 @item When @emph{FILE} packet received, check if it is completely
219 downloaded (comparing to @emph{INFO}'s packet size information). If so,
220 then run background integrity checker on it. If check succeeds, then
221 delete @file{.part} suffix from file's name and send @emph{DONE} packet.
223 @item When @emph{DONE} packet received, delete corresponding file.
225 @item When @emph{HALT} packet received, empty file sending queue.
227 @item Each second, node checks: are there any new @emph{tx} packets
228 appeared and queues corresponding @emph{INFO} packets.
230 @item Each minute, if no packets were sent, node sends @emph{PING}
233 @item If no non-PING packets are sent and received during
234 @ref{CfgOnlineDeadline, onlinedeadline} duration, then close the
235 connection. There is no explicit indication that session is over.
237 @item If no packets are received during two minutes (two PING timeouts),
238 then close the connection.