+nncp-ack
nncp-bundle
nncp-call
nncp-caller
+@cindex about
@strong{NNCP} (Node to Node copy) is a collection of utilities
simplifying secure store-and-forward files, mail and command exchanging.
See also this page @ref{Об утилитах, on russian}.
+@cindex F2F
+@cindex friend-to-friend
+@cindex E2E
+@cindex end-to-end
+@cindex darknet
+@cindex DTN
+@cindex delay tolerant
+@cindex dead drop
+@cindex onion encryption
This utilities are intended to help build up small size (dozens of
nodes) ad-hoc @url{https://en.wikipedia.org/wiki/Friend-to-friend,
friend-to-friend} (F2F) statically routed
Look for possible @ref{Use cases, use cases}!
+@cindex GPL
+@cindex free software
+@cindex licence
NNCP is @url{https://www.gnu.org/philosophy/pragmatic.html, copylefted}
@url{https://www.gnu.org/philosophy/free-sw.html, free software}
licenced under @url{https://www.gnu.org/licenses/gpl-3.0.html, GNU GPLv3}.
@node Administration
+@cindex administration
@unnumbered Administration
NNCP uses following files/directories you should be aware of:
It can contain @file{.nock} files: fully downloaded, but still not
checksummed. Can be checksummed (with @file{.nock} extension removing)
- with @command{nncp-check -nock}.
+ with @command{@ref{nncp-check} -nock}.
Also it can contain @file{seen/} and @file{hdr/} subdirectories,
that should be cleaned too from time to time.
- All of that cleaning tasks can be done with @ref{nncp-rm} utility.
+ All of that cleaning tasks can be done with @command{@ref{nncp-rm}} utility.
+ @cindex shared spool
+ @cindex setgid
+ @pindex umask
@anchor{Shared spool}
If you want to share single spool directory with multiple grouped
Unix users, then you can @command{setgid} it and assure that umask
@ref{Log} file, for example @file{/var/spool/nncp/log}. It should be
rotated. Choose you own preferable way to do it.
+ @pindex newsyslog
Example @url{https://www.newsyslog.org/manual.html, newsyslog}'s entry:
@example
/var/spool/nncp/log 644 7 100 * BCYN
@item
Optional @ref{CfgIncoming, incoming} directories where uploaded
- files are stored. Probably you want to run @ref{nncp-reass} from
- time to time to reassemble all chunked uploads. Example crontab
+ files are stored. Probably you want to run @command{@ref{nncp-reass}}
+ from time to time to reassemble all chunked uploads. Example crontab
entry:
@example
*/1 * * * * nncp-reass -all -noprogress
@end example
+@pindex daemontools
+@pindex supervise
+@pindex multilog
@item
- Possibly long running @ref{nncp-daemon}, @ref{nncp-caller},
- @ref{nncp-toss}, @ref{nncp-check} daemons. As all software, they can
+ Possibly long running @command{@ref{nncp-daemon}},
+ @command{@ref{nncp-caller}}, @command{@ref{nncp-toss}},
+ @command{@ref{nncp-check}} daemons. As all software, they can
fail and you should place them under some supervisor control.
For example you can use @url{http://cr.yp.to/daemontools.html,
# mv /var/service/.nncp-toss /var/service/nncp-toss
@end example
+@pindex inetd
@item
- @ref{nncp-daemon} can also be run as
+ @command{@ref{nncp-daemon}} can also be run as
@url{https://en.wikipedia.org/wiki/Inetd, inetd} service on UUCP's port:
@example
uucp stream tcp6 nowait nncpuser /usr/local/bin/nncp-daemon nncp-daemon -quiet -ucspi
@end example
+@cindex UCSPI
+@pindex tcpserver
@item
Or it can be also run as a @command{daemontools} daemon under
@url{http://cr.yp.to/ucspi-tcp.html, UCSPI-TCP}. In the example
@node Build-instructions
+@cindex building
@section Build instructions
Make sure that Go is installed. For example to install it from packages:
$ redo all
@end example
+@pindex info
After that you should get various @command{bin/nncp-*} binaries and
@command{bin/hjson-cli} command (only for your convenience, not
necessary installation). For example, documentation for
@command{nncp-bundle} command can be get with
@command{info doc/nncp.info -n nncp-bundle}.
+@pindex redo
+@pindex apenwarr/redo
+@pindex apenwarr/do
+@pindex redo-c
+@pindex baredo
+@pindex goredo
It uses @url{http://cr.yp.to/redo.html, redo} build system for that
examples. You can use one of its various implementations, or at least
minimalistic POSIX shell @command{contrib/do} (just replace
from that project), @url{https://github.com/leahneukirchen/redo-c, redo-c},
@url{https://github.com/gotroyb127/baredo, baredo}.
-There is @command{install} target respecting @env{$DESTDIR}. It will
-install binaries and info-documentation:
+@vindex PREFIX
+@vindex DESTDIR
+@vindex GO
+@vindex MAKEINFO
+@vindex PLANTUML
+@vindex PREFIX
+@vindex SENDMAIL
+@vindex CFGPATH
+@vindex SPOOLPATH
+@vindex LOGPATH
+@vindex BINDIR
+@vindex INFODIR
+@vindex DOCDIR
+@file{config} file contains some environment variables that are
+respected during installation:
+@env{$PREFIX},
+@env{$DESTDIR},
+@env{$GO},
+@env{$MAKEINFO},
+@env{$PLANTUML},
+@env{$PREFIX},
+@env{$SENDMAIL},
+@env{$CFGPATH},
+@env{$SPOOLPATH},
+@env{$LOGPATH},
+@env{$BINDIR},
+@env{$INFODIR},
+@env{$DOCDIR}.
+
+There is @command{install} target for binaries and info-documentation
+installation:
@example
# PREFIX=/usr/local redo install
@end example
+@vindex nofsnotify
+@cindex kqueue
+@cindex epoll
+@vindex GO_CFLAGS
NNCP depends on @code{github.com/fsnotify/fsnotify} library, that is
solely relies on OS-specific mechanisms. There is possibility that you
have either broken or unsupported ones. You can still build NNCP with
usage at all:
@example
-$ GO_CFLAGS="-tags nofsnotify" redo ...
+$ GO_CFLAGS="-tags nofsnotify" redo @dots{}
@end example
+@vindex noyggdrasil
You can also disable Yggdrasil support with @code{-tags noyggdrasil}.
@node Bundles
+@cindex bundles
+@cindex tapes
+@cindex streaming media
@unnumbered Bundles
-Usual @ref{nncp-xfer} command requires filesystem it can operate on.
-That presumes random access media storage usage, like hard drives, USB
-flash drives and similar. But media like CD-ROM and especially tape
-drives are sequential by nature. You can prepare intermediate directory
-for recording to CD-ROM disc/tape, but that requires additional storage
-and is inconvenient.
+Usual @command{@ref{nncp-xfer}} command requires filesystem it can
+operate on. That presumes random access media storage usage, like hard
+drives, USB flash drives and similar. But media like CD-ROM and
+especially tape drives are sequential by nature. You can prepare
+intermediate directory for recording to CD-ROM disc/tape, but that
+requires additional storage and is inconvenient.
-Bundles, created with @ref{nncp-bundle} command are convenient
-alternative to ordinary @command{nncp-xfer}. Bundle is just a collection
-of @ref{Encrypted, encrypted packets}, stream of packets. It could be
-sequentially streamed for recording and digested back.
+Bundles, created with @command{@ref{nncp-bundle}} command are convenient
+alternative to ordinary @command{@ref{nncp-xfer}}. Bundle is just a
+collection of @ref{Encrypted, encrypted packets}, stream of packets. It
+could be sequentially streamed for recording and digested back.
@itemize
@end itemize
+@pindex pax
+@pindex tar
Technically bundle is valid POSIX.1-2001
@url{https://pubs.opengroup.org/onlinepubs/9699919799/utilities/pax.html#tag_20_92_13_01, pax archive}
with directory/files hierarchy identical to that is used in
-@ref{nncp-xfer}: @file{NNCP/RECIPIENT/SENDER/PACKET}. So bundle can also
-be created by manual tar-ing of @command{nncp-xfer} resulting directory.
+@command{@ref{nncp-xfer}}: @file{NNCP/RECIPIENT/SENDER/PACKET}.
+So bundle can also be created by manual tar-ing of
+@command{@ref{nncp-xfer}} resulting directory.
@node Call
+@cindex call
+@vindex calls
@unnumbered Call configuration
-Call is a rule when and how node can be called by @ref{nncp-caller}.
+Call is a rule when and how node can be called by @command{@ref{nncp-caller}}.
Example list of call structures:
@table @emph
+@vindex cron
@item cron
@include cronexpr.texi
+@vindex nice
@item nice
Optional. Use that @ref{Niceness, niceness} during the call (255 is used
otherwise).
+@vindex xx
+@vindex rx
+@vindex tx
@item xx
Optional. Either @verb{|rx|} or @verb{|tx|}. Tells only to either to
receive or to transmit data during that call.
+@vindex addr
@item addr
Optional. Call only that address, instead of trying all from
@ref{CfgAddrs, @emph{addrs}} configuration option. It can be either key
Optional. Override @ref{CfgMaxOnlineTime, @emph{maxonlinetime}}
configuration option when calling.
+@vindex autotoss
@item autotoss, -doseen, -nofile, -nofreq, -noexec, -notrns
Optionally enable auto tossing: run tosser on node's spool every second
during the call. You can control either are @file{seen/} files must be
created, or skip any kind of packet processing.
+@vindex when-tx-exists
@item when-tx-exists
Call only if packets for sending exists. The check of outbound packets
existence is performed @strong{every} time we are going to make a call,
@emph{cron} configuration decides that it is time to make a call, with
@emph{when-tx-exists} option it checks packets existence first.
+@vindex nock
@anchor{CfgNoCK}
@item nock
NoCK (no-checksumming) tells not to do checksumming of received files,
-assuming that it will be done for example with @ref{nncp-check} command
-later. That can help minimizing time spent online, because HDD won't do
-simultaneous reading of the data for checksumming and writing of the
-received one, but just sequential writing of the file. Pay attention
-that you have to make a call to remote node after checksumming is done,
-to send notification about successful packet reception.
-
+assuming that it will be done for example with
+@command{@ref{nncp-check}} command later. That can help minimizing time
+spent online, because HDD won't do simultaneous reading of the data for
+checksumming and writing of the received one, but just sequential
+writing of the file. Pay attention that you have to make a call to
+remote node after checksumming is done, to send notification about
+successful packet reception.
+
+@vindex mcd-ignore
@anchor{CfgMCDIgnore}
@item mcd-ignore
Ignore @ref{MCD} announcements: do not add MCD addresses for possible
@node CfgAreas
+@vindex areas
@section Configuration areas options
@ref{Multicast} areas configuration only used with multicast packets.
The only required field is the @code{id}. You can not process multicast
packets that has unknown area identification.
+@vindex subs
@code{subs} contains a list of recipients you must relay incoming
multicast packet on.
@node Configuration directory
+@cindex configuration directory
@section Configuration directory
Optionally you can convert configuration file to the directory layout
-with @ref{nncp-cfgdir} command. And vice versa too, of course loosing
-the comment lines. Directory layout can looks like that:
+with @command{@ref{nncp-cfgdir}} command. And vice versa too, of course
+loosing the comment lines. Directory layout can looks like that:
@example
nncp-cfg-dir
Your @option{-cfg} and @env{$NNCPCFG} could point to that directory,
instead of @file{.hjson} file. It will be transparently converted to
internal JSON representation. However it can not be encrypted with the
-@ref{nncp-cfgenc}.
+@command{@ref{nncp-cfgenc}}.
+@cindex private keys
That layout should be much more machine friendly and scriptable. Each
string parameters is stored as a single line plain text file. String
arrays are newline-separated plain text files. Dictionaries are
@node CfgGeneral
+@cindex general configuration options
@section Configuration general options
Those options are in the root of configuration dictionary.
@end verbatim
@table @code
+
+@vindex spool
@item spool
Absolute path to the @ref{Spool, spool} directory.
+
+@vindex log
+@vindex FD log file descriptor
@item log
Either:
@itemize
@item @code{FD:XXX}, where @code{XXX} is a decimal file descriptor
to write records too
@end itemize
+
+@vindex umask
@item umask
Will force all invoked commands to override their umask to specified
octal mask. Useful for using with @ref{Shared spool, shared spool directories}.
+
+@vindex noprogress
@item noprogress
When enabled, disables progress showing for many commands by default.
You can always force its showing with @option{-progress} command line
option anyway.
+
+@vindex nohdr
@anchor{CfgNoHdr}
@item nohdr
@strong{nohdr} option disables @ref{HdrFile, @file{hdr/}} files usage.
+
@end table
And optional @ref{MCD, MultiCast Discovery} options:
@table @code
+
+@vindex mcd-listen
@anchor{CfgMCDListen}
@item mcd-listen
Specifies list of network interfaces regular expression
-@ref{nncp-caller} will listen for incoming @ref{MCD} announcements.
+@command{@ref{nncp-caller}} will listen for incoming @ref{MCD} announcements.
+
+@vindex mcd-send
@anchor{CfgMCDSend}
@item mcd-send
Specifies list of network interfaces regular expressions, and intervals
-in seconds, where @ref{nncp-daemon} will send @ref{MCD} announcements.
+in seconds, where @command{@ref{nncp-daemon}} will send @ref{MCD} announcements.
+
@end table
+@cindex yggdrasil aliases
@anchor{CfgYggdrasilAliases}
Optional @ref{Yggdrasil}-related aliases are used for convenience and
keeping private keys away being used directly in command line. Each
@code{PUB}, @code{PRV}, @code{PEER}, @code{BIND} value in
-@ref{nncp-daemon}'s @option{-yggdrasil} and in @code{yggdrasil:}
+@command{@ref{nncp-daemon}}'s @option{-yggdrasil} and in @code{yggdrasil:}
addresses is replaced with alias value. Moreover each entry in list of
@code{PUB}s, @code{PEER}s and @code{BIND} can be an alias too. Pay
attention, that all aliases ending with @code{prv} will be saved with
@node Configuration
+@cindex configuration file
@unnumbered Configuration file
+@cindex Hjson
NNCP uses single file configuration file in @url{https://hjson.org/,
Hjson} format (see also section about @ref{Configuration directory,
-directory layout}) . Initially it is created with @ref{nncp-cfgnew}
+directory layout}) . Initially it is created with @command{@ref{nncp-cfgnew}}
command and at minimum it can look like this:
@verbatim
}
@end verbatim
+@cindex JSON
+@pindex hjson-cli
+@pindex gojq
+@pindex gjo
Do not forget that Hjson can be safely converted to JSON and vice versa
(loosing formatting and comments of course). By default
@command{hjson-cli} utility from @code{github.com/hjson/hjson-go} is
@node CfgNeigh
+@cindex neighbour configuration options
@section Configuration neighbour options
+@vindex neigh
@strong{neigh} section contains all known neighbours information. It
always has @strong{self} neighbour that is copy of our node's public
data (public keys). It is useful for copy-paste sharing with your
If present, then node can be online called using @ref{Sync,
synchronization protocol}. Contains authentication public key.
+@vindex exec
+@pindex sendmail
@anchor{CfgExec}
- @item exec
+@item exec
Dictionary consisting of handles and corresponding command line
arguments. In example above there are @command{sendmail} handles,
@command{warcer}, @command{wgeter} and @command{flag} one. Remote
feeding @verb{|hello world\n|} to that started @command{sendmail}
process.
+@vindex incoming
@anchor{CfgIncoming}
@item incoming
Full path to directory where all file uploads will be saved. May be
omitted to forbid file uploading on that node.
+@vindex freq
@anchor{CfgFreq}
@item freq
@table @code
transmission.
@end table
+@vindex via
@anchor{CfgVia}
@item via
An array of node identifiers that will be used as a relay to that
nodes. May be omitted if direct connection exists and no relaying is
required.
+@vindex addrs
@anchor{CfgAddrs}
@item addrs
Dictionary containing known network addresses of the node. Each key
is human-readable name of the address. For direct TCP connections
- use @verb{|host:port|} format, pointing to @ref{nncp-daemon}'s
+ use @verb{|host:port|} format, pointing to @command{@ref{nncp-daemon}}'s
listening instance.
Also you can pipe connection through the external command using
start and its @code{stdin}/@code{stdout} used as a connection.
To use @ref{Yggdrasil} network for connectivity, use
- @verb{|yggdrasil:PUB;PRV;PEER[,...]|} format, read about
+ @code{yggdrasil:PUB;PRV;PEER[,@dots{}]} format, read about
@ref{CfgYggdrasilAliases, possible aliases} usage.
May be omitted if either no direct connection exists, or
- @ref{nncp-call} is used with forced address specifying.
+ @command{@ref{nncp-call}} is used with forced address specifying.
+@vindex rxrate
+@vindex txrate
@anchor{CfgXxRate}
@item rxrate/txrate
If greater than zero, then at most *rate packets per second will be
bandwidth traffic shaper: each packet has at most 64 KiB payload
size. If omitted -- no rate limits.
+@vindex onlinedeadline
@anchor{CfgOnlineDeadline}
@item onlinedeadline
Online connection deadline of nodes inactivity in seconds. It is the
delays), wait for appearing packets ready to send and notifying
remote side about their appearance.
+@vindex maxonlinetime
@anchor{CfgMaxOnlineTime}
@item maxonlinetime
If greater than zero, then it is maximal time of single connection.
@anchor{CfgCalls}
@item calls
List of @ref{Call, call configuration}s.
- Can be omitted if @ref{nncp-caller} won't be used to call that node.
+ Can be omitted if @command{@ref{nncp-caller}} won't be used to call that node.
@end table
@node CfgNotify
+@cindex email notification
+@cindex notification configuration options
+@cindex logging handles
@section Configuration notification options
That section controls what notifications are enabled and how must be
@node CfgSelf
+@cindex self-node configuration keypairs
@section Configuration self-node keypairs
@strong{self} section contains our node's private keypairs.
+@vindex ExchPrv
+@vindex ExchPub
+@vindex SignPrv
+@vindex SignPub
@strong{exch*} and @strong{sign*} are used during @ref{Encrypted,
encrypted} packet creation.
+@vindex NoisePrv
+@vindex NoisePub
@strong{noise*} are used during @ref{Sync, synchronization protocol}
-working in @ref{nncp-call}, @ref{nncp-caller}, @ref{nncp-daemon}.
+working in @command{@ref{nncp-call}}, @command{@ref{nncp-caller}},
+@command{@ref{nncp-daemon}}.
@node Chunked
+@cindex chunked
@unnumbered Chunked files
There is ability to transfer huge files with dividing them into smaller
storage devices, and/or at different time, reassembling the whole packet
on the destination node.
-Splitting is done with @ref{nncp-file, nncp-file -chunked} command and
-reassembling with @ref{nncp-reass} command.
+Splitting is done with @command{@ref{nncp-file} -chunked} command and
+reassembling with @command{@ref{nncp-reass}} command.
+@vindex .nncp.meta
+@vindex .nncp.chunk
Chunked @file{FILE} produces @file{FILE.nncp.meta},
@file{FILE.nncp.chunk0}, @file{FILE.nncp.chunk1}, @dots{} files. All
@file{.nncp.chunkXXX} can be concatenated together to produce original
@ref{MTH} checksum of each chunk
@end multitable
+@cindex ZFS recordsize
@anchor{ChunkedZFS}
It is strongly advisable to reassemble incoming chunked files on
@url{https://en.wikipedia.org/wiki/ZFS, ZFS} dataset with deduplication
@node Commands
+@cindex commands
@unnumbered Commands
Nearly all commands have the following common options:
@table @option
+@vindex NNCPCFG
@item -cfg
Path to configuration file. May be overridden by @env{$NNCPCFG}
environment variable. If file file is an encrypted @ref{EBlob,
Override @ref{CfgVia, via} configuration option for destination node.
Specified nodes must be separated with comma: @verb{|NODE1,NODE2|}.
With @verb{|-via -|} you can disable relaying at all.
+@vindex NNCPSPOOL
@item -spool
Override path to spool directory. May be specified by
@env{$NNCPSPOOL} environment variable.
+@vindex NNCPLOG
@item -log
Override path to logfile. May be specified by @env{$NNCPLOG}
environment variable.
@item -quiet
Print only errors, omit simple informational messages. In any case
those messages are logged, so you can reread them using
- @ref{nncp-log} command.
+ @command{@ref{nncp-log}} command.
@item -progress, -noprogress
Either force progress showing, or disable it.
@item -version
Print warranty information (no warranty).
@end table
+@vindex TMPDIR
+All commands respect @env{$TMPDIR} environment variable.
+
+@vindex NNCPNOSYNC
+If you set @env{$NNCPNOSYNC=1}, then all @code{fsync} operations will be
+skipped. That is dangerous option, but if there there are abilities to
+rollback possibly corrupted state to some stable snapshot, then disabled
+@code{fsync} can give considerable increase in performance.
+
@menu
Configuration file commands
* nncp-exec::
* nncp-freq::
* nncp-trns::
+* nncp-ack::
Packets sharing commands
@include cmd/nncp-exec.texi
@include cmd/nncp-freq.texi
@include cmd/nncp-trns.texi
+@include cmd/nncp-ack.texi
@include cmd/nncp-xfer.texi
@include cmd/nncp-bundle.texi
@include cmd/nncp-toss.texi
--- /dev/null
+@node nncp-ack
+@cindex packet acknowledgement
+@pindex nncp-ack
+@section nncp-ack
+
+@example
+$ nncp-ack [options] -all
+$ nncp-ack [options] -node NODE[,@dots{}]
+$ nncp-ack [options] -node NODE -pkt PKT
+@end example
+
+Send acknowledgement of successful @option{PKT} (Base32-encoded hash)
+packet receipt from @option{NODE} node. If no @option{-pkt} is
+specified, then acknowledge all packet in node's @code{rx} outbound
+spool. If @option{-all} is specified, then do that for all nodes.
+
+General workflow with acknowledgement is following, assuming that
+Alice has some outbound packets for Bob:
+
+@itemize
+
+@item Transfer encrypted packets, without deleting them locally:
+
+@example
+alice$ nncp-xfer -keep -tx -node bob /mnt/shared
+@end example
+
+@item On Bob's side retrieve those packets:
+
+@example
+bob$ nncp-xfer -rx /mnt/shared
+@end example
+
+That will also check if copied packets checksum is not mismatched.
+
+@item Create ACK packets of received ones:
+
+@example
+bob$ nncp-ack -node alice
+@end example
+
+@item Send those newly created packets back to Alice:
+
+@example
+bob$ nncp-xfer -tx /mnt/shared
+@end example
+
+@item Get those acknowledgement packets and @ref{nncp-toss, toss} them:
+
+@example
+alice$ nncp-xfer -rx /mnt/shared
+alice$ nncp-toss
+@end example
+
+Each ACK packet will remove kept corresponding outbound packets, because
+Bob explicitly confirmed their receipt.
+
+@end itemize
+
+Similarly you can use it with @command{@ref{nncp-bundle}}, but do not
+forget that by default it does not do checksumming of the packets, so
+you should either use its @option{-check} option, or run
+@command{@ref{nncp-check}} after.
@node nncp-bundle
+@pindex nncp-bundle
@section nncp-bundle
@example
-$ nncp-bundle [options] -tx [-delete] NODE [NODE @dots{}] > ...
-$ nncp-bundle [options] -rx -delete [-dryrun] [NODE @dots{}] < ...
-$ nncp-bundle [options] -rx [-check] [-dryrun] [NODE @dots{}] < ...
+$ nncp-bundle [options] -tx [-delete] NODE [NODE @dots{}] > @dots{}
+$ nncp-bundle [options] -rx -delete [-dryrun] [NODE @dots{}] < @dots{}
+$ nncp-bundle [options] -rx [-check] [-dryrun] [NODE @dots{}] < @dots{}
@end example
With @option{-tx} option, this command creates @ref{Bundles, bundle} of
tape drives could easily provide too much throughput your CPU won't be
able to verify on the fly. So if you won't @ref{nncp-toss, toss}
received packets at the place, it is advisable either to run
-@ref{nncp-check} utility for packets integrity verification, or to use
-@option{-check} option to enable on the fly integrity check.
+@command{@ref{nncp-check}} utility for packets integrity verification,
+or to use @option{-check} option to enable on the fly integrity check.
You can specify multiple @option{NODE} arguments, telling for what nodes
you want to create the stream, or take it from. If no nodes are
@node nncp-call
+@pindex nncp-call
@section nncp-call
@example
success. Optionally you can force @option{FORCEADDR} address usage,
instead of addresses taken from configuration file. You can specify
@verb{|host:port|}, @verb{#|some command#} and
-@verb{|yggdrasil:PUB;PRV;PEER[,...]|} formats.
+@code{yggdrasil:PUB;PRV;PEER[,@dots{}]} formats.
If you specify @option{-ucspi} option, then it is assumed that you run
-@command{nncp-call} command under some UCSPI-TCP compatible utility,
+@command{@ref{nncp-call}} command under some UCSPI-TCP compatible utility,
that provides read/write channels through 6/7 file descriptors.
@option{-mcd-wait} options tells to wait up to specified number of
@option{-autotoss} option runs tosser on node's spool every second
during the call. All @option{-autotoss-*} options is the same as in
-@ref{nncp-toss} command.
+@command{@ref{nncp-toss}} command.
Partly downloaded packets are stored in @file{.part} files. By default
all downloaded files are sequentially checksummed in the background,
files to @file{.nock} extension. Pay attention that checksumming can be
time consuming and connection could be lost during that check, so remote
node won't be notified that the file is finished. If you run
-@ref{nncp-check, @command{nncp-check -nock}}, that will checksum files
-and strip the @file{.nock} extension, then repeated call to remote node
-will notify about packet's completion. Also it will be notified if
-@ref{nncp-toss, tossing} created @file{seen/} file.
-Read @ref{CfgNoCK, more} about @option{-nock} option.
+@command{@ref{nncp-check} -nock}, that will checksum files and strip the
+@file{.nock} extension, then repeated call to remote node will notify about
+packet's completion. Also it will be notified if @ref{nncp-toss, tossing}
+created @file{seen/} file. Read @ref{CfgNoCK, more} about @option{-nock}
+option.
@node nncp-caller
+@pindex nncp-caller
@section nncp-caller
@example
Otherwise all nodes with specified @emph{calls} configuration
field will be called.
-Look at @ref{nncp-call} for more information.
+Look at @command{@ref{nncp-call}} for more information.
@node nncp-cfgdir
+@pindex nncp-cfgdir
@section nncp-cfgdir
@example
@node nncp-cfgenc
+@pindex nncp-cfgenc
@section nncp-cfgenc
@example
@node nncp-cfgmin
+@cindex stripped configuration
+@cindex minimized configuration
+@pindex nncp-cfgmin
@section nncp-cfgmin
@example
Print out stripped configuration version: only path to @ref{Spool,
spool}, path to log file, neighbours public keys are stayed. This is
-useful mainly for usage with @ref{nncp-xfer} that has to know only
-neighbours, without private keys involving.
+useful mainly for usage with @command{@ref{nncp-xfer}} that has to know
+only neighbours, without private keys involving.
@node nncp-cfgnew
+@pindex nncp-cfgnew
@section nncp-cfgnew
@example
@node nncp-check
+@pindex nncp-check
@section nncp-check
@example
@node nncp-cronexpr
+@pindex nncp-cronexpr
@section nncp-cronexpr
@example
@node nncp-daemon
+@pindex nncp-daemon
@section nncp-daemon
@example
Start listening TCP daemon, wait for incoming connections and run
@ref{Sync, synchronization protocol} with each of them. You can run
-@ref{nncp-toss} utility in background to process inbound packets from
-time to time.
+@command{@ref{nncp-toss}} utility in background to process inbound
+packets from time to time.
@option{-maxconn} option specifies how many simultaneous clients daemon
can handle. @option{-bind} option specifies @option{addr:port} it must
@option{-autotoss} option runs tosser on node's spool every second
during the call. All @option{-autotoss-*} options is the same as in
-@ref{nncp-toss} command.
+@command{@ref{nncp-toss}} command.
Read @ref{CfgNoCK, more} about @option{-nock} option.
@node nncp-exec
+@pindex nncp-exec
@section nncp-exec
@example
}
@end verbatim
+@vindex NNCP_SELF
+@vindex NNCP_SENDER
+@vindex NNCP_NICE
then executing @verb{|echo My message | nncp-exec -replynice 123 REMOTE
sendmail root@localhost|} will lead to execution of:
@node nncp-file
+@pindex nncp-file
@section nncp-file
@example
@node nncp-freq
+@pindex nncp-freq
@section nncp-freq
@example
@node nncp-hash
+@pindex nncp-hash
@section nncp-hash
@example
@node nncp-log
+@pindex nncp-log
@section nncp-log
@example
@node nncp-pkt
+@pindex nncp-pkt
@section nncp-pkt
@example
@option{-dump} option outputs plain packet's payload (if it is file
transmission, then it will be the file itself as an example). If it is
an encrypted packet, then it will be decrypted first, outputing the
-included plain packet, that can be fed to @command{nncp-pkt} again:
+included plain packet, that can be fed to @command{@ref{nncp-pkt}} again:
@example
Packet type: plain
@node nncp-reass
+@pindex nncp-reass
@section nncp-reass
@example
@node nncp-rm
+@pindex nncp-rm
@section nncp-rm
@example
@item If @option{-tmp} option is specified, then it will delete all
temporary files in @file{spool/tmp} directory. Files may stay in it when
-commands like @ref{nncp-file} fail for some reason.
+commands like @command{@ref{nncp-file}} fail for some reason.
@item If @option{-lock} option is specified, then all @file{.lock} files
will be deleted in your spool directory.
@node nncp-stat
+@pindex nncp-stat
@section nncp-stat
@example
@node nncp-toss
+@pindex nncp-toss
@section nncp-toss
@example
running this command as a daemon.
@option{-seen} option creates empty @file{seen/XXX} file after
-successful tossing of @file{XXX} packet. @ref{nncp-xfer},
-@ref{nncp-bundle}, @ref{nncp-daemon} and @ref{nncp-call} commands skip
-inbound packets that has been already seen, processed and tossed. This
-is helpful to prevent duplicates.
+successful tossing of @file{XXX} packet. @command{@ref{nncp-xfer}},
+@command{@ref{nncp-bundle}}, @command{@ref{nncp-daemon}} and
+@command{@ref{nncp-call}} commands skip inbound packets that has been
+already seen, processed and tossed. This is helpful to prevent
+duplicates.
@option{-nofile}, @option{-nofreq}, @option{-noexec}, @option{-notrns},
@option{-noarea} options allow disabling any kind of packet types processing.
@node nncp-trns
+@pindex nncp-trns
@section nncp-trns
@example
@node nncp-xfer
+@pindex nncp-xfer
@section nncp-xfer
@example
@option{-rx} option tells only to move inbound packets addressed to us.
@option{-tx} option tells exactly the opposite: move only outbound packets.
-@ref{nncp-cfgmin} could be useful for creating stripped minimalistic
-configuration file version without any private keys.
+@command{@ref{nncp-cfgmin}} could be useful for creating stripped
+minimalistic configuration file version without any private keys.
@file{DIR} directory has the following structure:
@file{RECIPIENT/SENDER/PACKET}, where @file{RECIPIENT} is Base32 encoded
destination node, @file{SENDER} is Base32 encoded sender node.
-Also look for @ref{nncp-bundle}, especially if you deal with CD-ROM and
-tape drives.
+Also look for @command{@ref{nncp-bundle}}, especially if you deal with
+CD-ROM and tape drives.
@multitable @columnfractions 0.40 0.15 0.15 0.15 0.15
@headitem @tab UUCP @tab FTN @tab NNCP @tab SMTP
-@item Простота настройки @tab Средне @tab Сложно @tab Легко @tab Сложно
-@item Передача почты @tab @strong{Да} @tab @strong{Да} @tab @strong{Да} @tab @strong{Да}
-@item Передача новостей @tab @strong{Да} @tab @strong{Да} @tab @strong{Да} @tab Нет
-@item Передача файлов @tab @strong{Да} @tab @strong{Да} @tab @strong{Да} @tab Нет
-@item Мультвещательная передача @tab Нет @tab @strong{Да} @tab @strong{Да} @tab Нет
-@item Разбиение файлов на части @tab Нет @tab @strong{Да} @tab @strong{Да} @tab Нет
-@item Удалённое исполнение команд @tab @strong{Да} @tab Нет @tab @strong{Да} @tab Нет
-@item Возобновляемое скачивание @tab @strong{Да} @tab @strong{Да} @tab @strong{Да} @tab Нет
-@item Приоритезация пакетов @tab @strong{Да} @tab Нет @tab @strong{Да} @tab Нет
-@item Сжатие почты @tab Нет @tab @strong{Да} @tab @strong{Да} @tab Нет
-@item Интеграция с SMTP @tab @strong{Да} @tab Нет @tab @strong{Да} @tab N/A
-@item Push/poll модель @tab @strong{Обе} @tab @strong{Обе} @tab @strong{Обе} @tab Push
-@item DTN @tab @strong{Да} @tab @strong{Да} @tab @strong{Да} @tab Нет
-@item Предполагаемый размер сети @tab Дюжины @tab Глобально @tab Дюжины @tab Глобально
-@item Маршрутизация @tab Ручное/статичное @tab Федеративное @tab Ручное/статичное @tab Федеративное
-@item Поддержка телефонной сети @tab @strong{Да} @tab @strong{Да} @tab Возможно @tab Нет
-@item Анонимные участники @tab @strong{Да} @tab Нет @tab Нет @tab @strong{Да}
-@item Аутентификация участников @tab PAP @tab PAP/CHAP @tab публичный ключ @tab Нет
-@item Шифрование пакетов @tab Нет @tab Нет @tab @strong{Да} @tab Нет
-@item Приватность метаданных @tab Нет @tab Нет @tab @strong{Да} @tab Нет
-@item Проверка целостности пакетов @tab Нет @tab Нет @tab @strong{Да} @tab Нет
-@item Дружелюбность к флоппинету @tab Нет @tab Частично @tab @strong{Да} @tab Нет
+@item Простота настройки @tab Средне @tab Сложно @tab Легко @tab Сложно
+@item Передача почты @tab @strong{Да} @tab @strong{Да} @tab @strong{Да} @tab @strong{Да}
+@item Передача новостей @tab @strong{Да} @tab @strong{Да} @tab @strong{Да} @tab Нет
+@item Передача файлов @tab @strong{Да} @tab @strong{Да} @tab @strong{Да} @tab Нет
+@item Мультвещательная передача @tab Нет @tab @strong{Да} @tab @strong{Да} @tab Нет
+@item Разбиение файлов на части @tab Нет @tab @strong{Да} @tab @strong{Да} @tab Нет
+@item Удалённое исполнение команд @tab @strong{Да} @tab Нет @tab @strong{Да} @tab Нет
+@item Возобновляемое скачивание @tab @strong{Да} @tab @strong{Да} @tab @strong{Да} @tab Нет
+@item Приоритезация пакетов @tab @strong{Да} @tab Нет @tab @strong{Да} @tab Нет
+@item Сжатие почты @tab Нет @tab @strong{Да} @tab @strong{Да} @tab Нет
+@item Интеграция с SMTP @tab @strong{Да} @tab Нет @tab @strong{Да} @tab N/A
+@item Push/poll модель @tab @strong{Обе} @tab @strong{Обе} @tab @strong{Обе} @tab Push
+@item DTN @tab @strong{Да} @tab @strong{Да} @tab @strong{Да} @tab Нет
+@item Предполагаемый размер сети @tab Дюжины @tab Глобально @tab Дюжины @tab Глобально
+@item Маршрутизация @tab Ручное/статичное @tab Федеративное @tab Ручное/статичное @tab Федеративное
+@item Поддержка телефонной сети @tab @strong{Да} @tab @strong{Да} @tab Возможно @tab Нет
+@item Анонимные участники @tab @strong{Да} @tab Нет @tab Нет @tab @strong{Да}
+@item Аутентификация участников @tab PAP @tab PAP/CHAP @tab публичный ключ @tab Нет
+@item Шифрование пакетов @tab Нет @tab Нет @tab @strong{Да} @tab Нет
+@item Приватность метаданных @tab Нет @tab Нет @tab @strong{Да} @tab Нет
+@item Проверка целостности пакетов @tab Нет @tab Нет @tab @strong{Да} @tab Нет
+@item Дружелюбность к флоппинету @tab Нет @tab Частично @tab @strong{Да} @tab Нет
@end multitable
@node Comparison
+@cindex comparison
+@cindex SMTP
+@cindex FTN
+@cindex FidoNet
+@cindex UUCP
@unnumbered Comparison with existing solutions
Here is comparison with @url{https://en.wikipedia.org/wiki/UUCP, UUCP}
-(Unix to Unix copy), FTN (@url{https://en.wikipedia.org/wiki/FidoNet,
-FidoNet} Technology Networks) and @url{https://en.wikipedia.org/wiki/SMTP, SMTP}
+(Unix to Unix copy), FTN (@url{https://en.wikipedia.org/wiki/FidoNet, FidoNet}
+Technology Networks) and @url{https://en.wikipedia.org/wiki/SMTP, SMTP}
(because it is also store-and-forward solution).
@multitable @columnfractions 0.40 0.15 0.15 0.15 0.15
@headitem @tab UUCP @tab FTN @tab NNCP @tab SMTP
-@item Ease of setup @tab Medium @tab Hard @tab Easy @tab Hard
-@item Mail transmission @tab @strong{Yes} @tab @strong{Yes} @tab @strong{Yes} @tab @strong{Yes}
-@item News transmission @tab @strong{Yes} @tab @strong{Yes} @tab @strong{Yes} @tab No
-@item File transmission @tab @strong{Yes} @tab @strong{Yes} @tab @strong{Yes} @tab No
-@item Multicast transmission @tab No @tab @strong{Yes} @tab @strong{Yes} @tab No
-@item Chunked files @tab No @tab @strong{Yes} @tab @strong{Yes} @tab No
-@item Remote command execution @tab @strong{Yes} @tab No @tab @strong{Yes} @tab No
-@item Resumable downloads @tab @strong{Yes} @tab @strong{Yes} @tab @strong{Yes} @tab No
-@item Packets prioritizing @tab @strong{Yes} @tab No @tab @strong{Yes} @tab No
-@item Mail compression @tab No @tab @strong{Yes} @tab @strong{Yes} @tab No
-@item SMTP integration @tab @strong{Yes} @tab No @tab @strong{Yes} @tab N/A
-@item Push/poll @tab @strong{Both} @tab @strong{Both} @tab @strong{Both} @tab Push
-@item DTN @tab @strong{Yes} @tab @strong{Yes} @tab @strong{Yes} @tab No
-@item Intended network size @tab Dozens @tab Global @tab Dozens @tab Global
-@item Routing @tab Manual/static @tab Federated @tab Manual/static @tab Federated
-@item PSTN support @tab @strong{Yes} @tab @strong{Yes} @tab Possible @tab No
-@item Anonymous peers @tab @strong{Yes} @tab No @tab No @tab @strong{Yes}
-@item Peers authentication @tab PAP @tab PAP/CHAP @tab public-key @tab No
-@item Packets encryption @tab No @tab No @tab @strong{Yes} @tab No
-@item Metadata privacy @tab No @tab No @tab @strong{Yes} @tab No
-@item Packets integrity check @tab No @tab No @tab @strong{Yes} @tab No
-@item Sneakernet friendliness @tab No @tab Partially @tab @strong{Yes} @tab No
+@item Ease of setup @tab Medium @tab Hard @tab Easy @tab Hard
+@item Mail transmission @tab @strong{Yes} @tab @strong{Yes} @tab @strong{Yes} @tab @strong{Yes}
+@item News transmission @tab @strong{Yes} @tab @strong{Yes} @tab @strong{Yes} @tab No
+@item File transmission @tab @strong{Yes} @tab @strong{Yes} @tab @strong{Yes} @tab No
+@item Multicast transmission @tab No @tab @strong{Yes} @tab @strong{Yes} @tab No
+@item Chunked files @tab No @tab @strong{Yes} @tab @strong{Yes} @tab No
+@item Remote command execution @tab @strong{Yes} @tab No @tab @strong{Yes} @tab No
+@item Resumable downloads @tab @strong{Yes} @tab @strong{Yes} @tab @strong{Yes} @tab No
+@item Packets prioritizing @tab @strong{Yes} @tab No @tab @strong{Yes} @tab No
+@item Mail compression @tab No @tab @strong{Yes} @tab @strong{Yes} @tab No
+@item SMTP integration @tab @strong{Yes} @tab No @tab @strong{Yes} @tab N/A
+@item Push/poll @tab @strong{Both} @tab @strong{Both} @tab @strong{Both} @tab Push
+@item DTN @tab @strong{Yes} @tab @strong{Yes} @tab @strong{Yes} @tab No
+@item Intended network size @tab Dozens @tab Global @tab Dozens @tab Global
+@item Routing @tab Manual/static @tab Federated @tab Manual/static @tab Federated
+@item PSTN support @tab @strong{Yes} @tab @strong{Yes} @tab Possible @tab No
+@item Anonymous peers @tab @strong{Yes} @tab No @tab No @tab @strong{Yes}
+@item Peers authentication @tab PAP @tab PAP/CHAP @tab public-key @tab No
+@item Packets encryption @tab No @tab No @tab @strong{Yes} @tab No
+@item Metadata privacy @tab No @tab No @tab @strong{Yes} @tab No
+@item Packets integrity check @tab No @tab No @tab @strong{Yes} @tab No
+@item Sneakernet friendliness @tab No @tab Partially @tab @strong{Yes} @tab No
@end multitable
UUCP and NNCP does not known nothing about routing. You have to
explicitly tell how to send (what hops to use) packets to each node.
+@cindex PSTN
@item PSTN support
UUCP and FidoNet always have been working with modems out-of-box.
Only many years later they gained support for working over TCP/IP
TCP daemon, but nothing prohibits using of another 8-bit aware
online transport.
+@cindex anonymity
+@cindex Sybil attack
@item Anonymous peers
NNCP and FTN are friend-to-friend networks exclusively. This is very
secure and mitigates many possible man-in-the-middle (MitM) and
@url{https://en.wikipedia.org/wiki/Sybil_attack, Sybil} attacks.
+@cindex sneakernet
+@cindex floppynet
@item Sneakernet friendliness
No one, except NNCP, supports data exchanging via removable storages
likes flash drives, CD-ROMs, tapes and hard drives out-of-box. It
@end table
+@cindex UUCP commands
Also there is
@url{https://changelog.complete.org/archives/10165-asynchronous-email-exim-over-nncp-or-uucp, copy of}
comparable commands of UUCP and NNCP, just for the interest:
@multitable @columnfractions 0.5 0.25 0.25
@headitem Purpose @tab UUCP @tab NNCP
+@pindex uucico
+@pindex uupoll
+@pindex uux
+@pindex uucp
+@pindex uuxqt
@item Connect to remote system
@tab @command{uucico -s}, @command{uupoll}
- @tab @command{nncp-call}, @command{nncp-caller}
+ @tab @command{@ref{nncp-call}}, @command{nncp-caller}
@item Receive connection (pipe, daemon, etc)
@tab @command{uucico} (@option{-l} or similar)
- @tab @command{nncp-daemon}
+ @tab @command{@ref{nncp-daemon}}
@item Request remote execution, @code{stdin} piped in
@tab @command{uux}
- @tab @command{nncp-exec}
+ @tab @command{@ref{nncp-exec}}
@item Copy file to remote machine
@tab @command{uucp}
- @tab @command{nncp-file}
+ @tab @command{@ref{nncp-file}}
@item Copy file from remote machine
@tab @command{uucp}
- @tab @command{nncp-freq}
+ @tab @command{@ref{nncp-freq}}
@item Process received requests
@tab @command{uuxqt}
- @tab @command{nncp-toss}
+ @tab @command{@ref{nncp-toss}}
@item Move outbound requests to dir (for USB stick, airgap, etc)
@tab N/A
- @tab @command{nncp-xfer}
+ @tab @command{@ref{nncp-xfer}}
@item Create streaming package of outbound requests
@tab N/A
- @tab @command{nncp-bundle}
+ @tab @command{@ref{nncp-bundle}}
@end multitable
@node Contacts
+@cindex contacts
+@cindex maillist
+@cindex IRC
+@cindex Matrix
@unnumbered Contacts
Please send questions regarding the use of NNCP, bug reports and patches to
+@cindex cron
@anchor{CronExpr}
This is copy-pasted documentation from
@code{github.com/gorhill/cronexpr} library used there.
@multitable @columnfractions .2 .1 .2 .5
@headitem Field name @tab Mandatory? @tab Allowed values @tab Allowed special characters
-@item Seconds @tab No @tab 0-59 @tab @verb{|* / , -|}
-@item Minutes @tab Yes @tab 0-59 @tab @verb{|* / , -|}
-@item Hours @tab Yes @tab 0-23 @tab @verb{|* / , -|}
-@item Day of month @tab Yes @tab 1-31 @tab @verb{|* / , - L W|}
-@item Month @tab Yes @tab 1-12 or JAN-DEC @tab @verb{|* / , -|}
-@item Day of week @tab Yes @tab 0-6 or SUN-SAT @tab @verb{|* / , - L #|}
-@item Year @tab No @tab 1970–2099 @tab @verb{|* / , -|}
+@item Seconds @tab No @tab 0-59 @tab @verb{|* / , -|}
+@item Minutes @tab Yes @tab 0-59 @tab @verb{|* / , -|}
+@item Hours @tab Yes @tab 0-23 @tab @verb{|* / , -|}
+@item Day of month @tab Yes @tab 1-31 @tab @verb{|* / , - L W|}
+@item Month @tab Yes @tab 1-12 or JAN-DEC @tab @verb{|* / , -|}
+@item Day of week @tab Yes @tab 0-6 or SUN-SAT @tab @verb{|* / , - L #|}
+@item Year @tab No @tab 1970-2099 @tab @verb{|* / , -|}
@end multitable
@node Tarballs
+@cindex tarball
+@cindex dependency
+@cindex licencing
+@cindex download
@section Prepared tarballs
You can obtain releases source code prepared tarballs from the links below.
@multitable {XXXXX} {XXXX-XX-XX} {XXXX KiB} {meta4 link sig} {xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx}
@headitem Version @tab Date @tab Size @tab Tarball @tab SHA256 checksum
+@item @ref{Release 8_5_0, 8.5.0} @tab 2022-01-26 @tab 1685 KiB
+@tab
+ @url{download/nncp-8.5.0.tar.xz.meta4, meta4}
+ @url{download/nncp-8.5.0.tar.xz, link}
+ @url{download/nncp-8.5.0.tar.xz.sig, sig}
+@tab @code{E8850D27 70C4FA74 5CF9B9A9 0CEEE5D6 05E50FA5 031A0813 49844C15 3E7BBFF4}
+
@item @ref{Release 8_4_0, 8.4.0} @tab 2022-01-25 @tab 1683 KiB
@tab
@url{download/nncp-8.4.0.tar.xz.meta4, meta4}
@node EBlob
+@cindex eblob
+@cindex encrypted configuration
@unnumbered EBlob format
EBlob is an encrypted blob (binary large object, in the terms of
low-entropy characters. Low-entropy text is much more easier to
remember, and its length provides pretty enough entropy as a result.
+@cindex password
+@cindex balloon
+@cindex Argon2
Password strengthening function is applied to that passphrase to
mitigate brute-force and dictionary attacks on it. Here,
@url{https://crypto.stanford.edu/balloon/, Balloon} memory-hard password
@end copying
@node Top
+@c dummy cindex, to pass through the current info's bug, which can skip
+@c the first index entry during searching
+@cindex 0
@top NNCP
+@cindex Pedro
@verbatiminclude pedro.txt
@include about.texi
+@cindex articles
There are also articles about its usage outside this website:
@itemize
+@item @url{https://www.complete.org/nncp/, Various related articles} by John Goerzen
@item @url{https://habr.com/en/post/330712/, NNCP: лечение online- и цензуро- зависимости store-and-forward методом} (on russian)
@item @url{https://changelog.complete.org/archives/tag/nncp, Series of descriptive articles} by John Goerzen
@item @url{https://www.youtube.com/watch?v=0i3r63pnyHM, Presentation of NNCP on Internet Freedom Conference 2019} (on russian)
* EBlob format: EBlob
* Mirrors::
* Thanks::
+* Indices::
* Contacts and feedback: Contacts
* Copying conditions: Copying
@end menu
@include eblob.texi
@include mirrors.texi
@include thanks.texi
+@include indices.texi
@include contacts.texi
@node Copying
--- /dev/null
+@node Indices
+@unnumbered Indices
+
+@node Concepts Index
+@section Concepts Index
+@printindex cp
+
+@node Programs Index
+@section Programs Index
+@printindex pg
+
+@node Variables Index
+@section Variables Index
+@printindex vr
@node Installation
+@cindex installation
+@cindex packages
+@cindex distributions
@unnumbered Installation
Possibly NNCP package already exists for your distribution:
@itemize
-@item @url{https://www.freshports.org/net/nncp/, FreeBSD ports}
+
+@cindex Arch Linux
+@cindex AUR
+@item Arch Linux @url{https://aur.archlinux.org/packages/nncp, AUR}
+
+@cindex Debian
+@item @url{https://tracker.debian.org/pkg/nncp, Debian packages}
+
+@cindex DragonFly
@item @url{https://github.com/DragonFlyBSD/DPorts/tree/master/net/nncp, DragonFly BSD ports}
+
+@cindex FreeBSD
+@item @url{https://www.freshports.org/net/nncp/, FreeBSD ports}
+
+@cindex Guix
+@item GNU @url{https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/uucp.scm, Guix}
+
+@cindex NetBSD
+@item @url{https://pkgsrc.se/wip/nncp, NetBSD package}
+
+@cindex NixOS
@item @url{https://github.com/NixOS/nixpkgs/tree/master/pkgs/tools/misc/nncp, NixOS packages}
+
+@cindex Void Linux
@item @url{https://github.com/void-linux/void-packages/blob/master/srcpkgs/nncp/template, Void Linux}
-@item @url{https://tracker.debian.org/pkg/nncp, Debian packages}
@end itemize
+@cindex POSIX
NNCP should run on any POSIX-compatible operating system.
-NNCP is written on @url{https://golang.org/, Go} programming language
+@pindex go
+@pindex texinfo
+NNCP is written on @url{https://go.dev/, Go} programming language
and you have to install Go compiler 1.13+ version.
@url{http://cr.yp.to/redo.html, redo} build system is recommended for
convenience. @url{https://www.gnu.org/software/texinfo/, Texinfo} is
@node BitTorrent
+@cindex BitTorrent integration
@section BitTorrent and huge files
If dealing with @ref{Git}, @ref{Feeds, web feeds} and @ref{Multimedia,
consumes much time. You can not wait for downloads finish, but want to
queue them after.
+@pindex aria2
@url{http://aria2.github.io/, aria2} multi-protocol download utility
could be used for solving that issue conveniently. It supports HTTP,
HTTPS, FTP, SFTP and BitTorrent protocols, together with
accelerate HTTP*/*FTP downloads by segmented multiple parallel
connections.
+@pindex aria2-downloaded.sh
You can queue you files after they are completely downloaded.
@file{aria2-downloaded.sh} contents:
@node DownloadService
+@cindex download service
+@pindex warcer.sh
+@pindex wgeter.sh
@section Downloading service
Previous sections tell about manual downloading and sending results to
@node Exim
+@cindex Exim integration
@section Integration with Exim
This section is unaltered copy-paste of
This is fairly simple in Exim.
+@pindex bsmtp
+@pindex rmail
We actually have two choices for how to do this: @command{bsmtp} or
@command{rmail} mode. bsmtp (batch SMTP) is the more modern way, and is
essentially a derivative of SMTP that explicitly can be queued
.endif
@end example
-This is pretty straightforward. We pipe to @command{nncp-exec}, run it
-as the nncp user. @command{nncp-exec} sends it to a target node and runs
+This is pretty straightforward. We pipe to @command{@ref{nncp-exec}}, run it
+as the nncp user. @command{@ref{nncp-exec}} sends it to a target node and runs
whatever that node has called @command{rsmtp} (the command to receive
bsmtp data). When the target node processes the request, it will run the
configured command and pipe the data in to it.
The @option{-bS} option is what tells Exim to receive BSMTP on @code{stdin}.
+@vindex MAIN_TRUSTED_USERS
Now, you need to tell Exim that nncp is a trusted user (able to set From
headers arbitrarily). Assuming you are running NNCP as the @code{nncp} user,
then add @code{MAIN_TRUSTED_USERS = nncp} to a file such as
@node Feeds
+@cindex Web feeds integration
+@cindex RSS feeds integration
+@cindex Atom feeds integration
@section Integration with Web feeds
+@pindex r2e
RSS and Atom feeds could be collected using
@url{https://github.com/wking/rss2email, rss2email} program. It converts
all incoming feed entries to email messages. Read about how to integrate
and add feeds you want to retrieve:
@example
-$ r2e add http://www.git.cypherpunks.ru/?p=nncp.git;a=atom
+$ r2e add http://www.nncpgo.org/download/releases.atom
@end example
and run the process:
@example
$ r2e run
@end example
+
+@pindex feeder
+But it is more flexible and advisable to use use
+@url{https://www.feeder.stargrave.org/, feeder} (NNCP's author creation)
+that also transforms feed entries to Maildir email messages. Incoming
+entries are placed in @file{new/} Maildir directories, so you can easily
+send all of them:
+
+@example
+% tar cf - **/new/*(.) | zstdmt | nncp-file - remote.node:feeds-`date -Iseconds`.tar.zst
+% feeder/dir/contrib/feeds-oldenize.zsh
+@end example
+
+Unlike @command{r2e}, @command{feeder} has also
+@url{http://www.feeder.stargrave.org/Enclosures.html, enclosures} and
+@url{http://www.feeder.stargrave.org/WARCs.html, WARCs} downloading
+capabilities, that are also stored as a files on the disk.
@node FreqIndex
+@cindex index files
@section Index files for freqing
In many cases you do not know exact files list on remote machine you
@node Git
+@cindex git integration
+@pindex git-bundle
@section Integration with Git
@url{https://git-scm.com/, Git} version control system already has all
@node Integration
+@cindex integration with existing software
@unnumbered Integration with existing software
Here is some examples of how you can solve popular tasks with NNCP,
@node Multimedia
+@cindex multimedia integration
+@cindex pindex youtube-dl
+@cindex pindex yt-dlp
+@cindex cindex YouTube
@section Integration with multimedia streaming
Many video and audio streams could be downloaded using
-@url{http://yt-dl.org/, youtube-dl} program.
-@url{https://rg3.github.io/youtube-dl/supportedsites.html, Look} how
+@url{https://github.com/yt-dlp/yt-dlp, yt-dlp} program.
+@url{https://github.com/yt-dlp/yt-dlp/blob/master/supportedsites.md, Look} how
many of them are supported, including @emph{Dailymotion}, @emph{Vimeo}
and @emph{YouTube}.
-When you multimedia becomes an ordinary file, you can transfer it easily.
+When your multimedia becomes an ordinary file, you can transfer it easily.
@example
-$ youtube-dl \
- --exec 'nncp-file @{@} remote.node:' \
+$ yt-dlp --exec 'nncp-file @{@} remote.node:' \
'https://www.youtube.com/watch?list=PLd2Cw8x5CytxPAEBwzilrhQUHt_UN10FJ'
@end example
@node Postfix
+@cindex Postfix integration
+@pindex postfix
@section Integration with Postfix
This section is taken from @url{http://www.postfix.org/UUCP_README.html,
@itemize
-@item You need an @ref{nncp-exec} program that extracts the sender
-address from mail that arrives via NNCP, and that feeds the mail into
-the Postfix @command{sendmail} command.
+@item You need an @command{@ref{nncp-exec}} program that extracts the
+sender address from mail that arrives via NNCP, and that feeds the mail
+into the Postfix @command{sendmail} command.
@item Define a @command{pipe(8)} based mail delivery transport for
delivery via NNCP:
flags=Rqhu user=nncp argv=nncp-exec -quiet $nexthop sendmail $recipient
@end example
-This runs the @command{nncp-exec} command to place outgoing mail into
+This runs the @command{@ref{nncp-exec}} command to place outgoing mail into
the NNCP queue after replacing @var{$nexthop} by the receiving NNCP
node and after replacing @var{$recipient} by the recipients. The
-@command{pipe(8)} delivery agent executes the @command{nncp-exec}
+@command{pipe(8)} delivery agent executes the @command{@ref{nncp-exec}}
command without assistance from the shell, so there are no problems with
shell meta characters in command-line parameters.
+@pindex sendmail.sh
+@vindex Return-Path
+@pindex reformail
Pay attention to @code{flags}, containing @code{R}, telling Postfix to
include @code{Return-Path:} header. Otherwise that envelope sender
information may be lost. Possibly you will also need somehow to
@verbatiminclude sendmail.sh
+@vindex From_
+@cindex mbox
+@pindex maildrop
Also pay attention that @command{maildrop} does not like @code{From_}
mbox-style header, so you possibly want:
@itemize
-@item You need an @ref{nncp-exec} program that extracts the sender
-address from mail that arrives via NNCP, and that feeds the mail into
-the Postfix @command{sendmail} command.
+@item You need an @command{@ref{nncp-exec}} program that extracts the
+sender address from mail that arrives via NNCP, and that feeds the mail
+into the Postfix @command{sendmail} command.
@item Specify that all remote mail must be sent via the @command{nncp}
mail transport to your NNCP gateway host, say, @emph{nncp-gateway}:
flags=Fqhu user=nncp argv=nncp-exec -quiet $nexthop sendmail $recipient
@end example
-This runs the @command{nncp-exec} command to place outgoing mail into
+This runs the @command{@ref{nncp-exec}} command to place outgoing mail into
the NNCP queue. It substitutes the hostname (@emph{nncp-gateway}, or
whatever you specified) and the recipients before execution of the
-command. The @command{nncp-exec} command is executed without assistance
+command. The @command{@ref{nncp-exec}} command is executed without assistance
from the shell, so there are no problems with shell meta characters.
@item Execute the command @command{postfix reload} to make the changes
@node PPP
+@cindex PPP
+@cindex serial link
+@cindex serial connection
@section Serial connection
-It is not trivial to run online @command{nncp-daemon},
-@command{nncp-call} and @command{nncp-caller} commands over the serial
-link, because it is link without built-in error detection. For efficient
-usage you have to use some kind of window-sliding error correction
-protocol, like Kermit, ZMODEM, UUCP's g-protocol and similar well known
-ones.
+It is not trivial to run online @command{@ref{nncp-daemon}},
+@command{@ref{nncp-call}} and @command{@ref{nncp-caller}} commands over
+the serial link, because it is link without built-in error detection.
+For efficient usage you have to use some kind of window-sliding error
+correction protocol, like Kermit, ZMODEM, UUCP's g-protocol and similar
+well known ones.
However TCP is already satisfying and existing protocol for the same
purposes. So it would be more easier to bring up the IP interconnection
@node WARCs
+@cindex WARC
+@pindex wget
@section Integration with Web pages
Simple HTML web page can be downloaded very easily for sending and
[@dots{}] http://www.example.com/
@end example
+@pindex tofuproxy
That command will create @file{www.example.com-XXX.warc} web archive.
It could produce specialized segmented
@url{https://en.wikipedia.org/wiki/Gzip, gzip} and
@node Integrity
+@cindex integrity check
+@cindex authenticity check
+@cindex OpenPGP
+@cindex gpg
+@cindex GnuPG
+@cindex WKD
@section Tarballs integrity check
You @strong{have to} check downloaded archives integrity and verify
@node Log
+@cindex log format
+@cindex logging
+@cindex recfile
+@pindex recutils
@unnumbered Log format
Log is a plaintext file consisting of
@url{https://www.gnu.org/software/recutils/, recfile} records. It can be
-read by human, but it is better to use either @ref{nncp-log}, or
-@command{recutils} utilities for selecting and formatting the required
+read by human, but it is better to use either @command{@ref{nncp-log}},
+or @command{recutils} utilities for selecting and formatting the required
fields.
Two example records from it:
@node MCD
+@cindex MCD
+@cindex multicast discovery
@unnumbered MultiCast Discovery
MCD is an addition to online @ref{Sync, synchronization protocol}, that
@itemize
@item
- @ref{nncp-daemon} sends multicast messages about its presence from
- time to time. See @ref{CfgMCDSend, mcd-send} configuration option.
+ @command{@ref{nncp-daemon}} sends multicast messages about its
+ presence from time to time. See @ref{CfgMCDSend, mcd-send}
+ configuration option.
@item
- When @ref{nncp-caller} sees them, it adds them as the most
+ When @command{@ref{nncp-caller}} sees them, it adds them as the most
preferred addresses to already known ones. If MCD address
announcement was not refreshed after two minutes -- it is removed.
See @ref{CfgMCDListen, mcd-listen} and
@strong{@verb{|ff02::4e4e:4350|}} (hexadecimal ASCII @verb{|NNCP|})
multicast address and @strong{5400} port. Operating system will use IPv6
link-local address as a source one, with the port taken from
-@command{nncp-daemon}'s @option{-bind} option. That way, IP packet
+@command{@ref{nncp-daemon}}'s @option{-bind} option. That way, IP packet
itself will carry the link-scope reachable address of the daemon.
@node Mirrors
+@cindex mirror
@unnumbered Mirrors
Main NNCP website is hosted on two geographically distant servers
of those servers supports TLS and another just proxies the traffic to
it. So TLS-capable version has less availability.
+@cindex DANE
+@cindex DNSCurve
+@cindex ca.cypherpunks.ru
It can be authenticated with
-@url{http://ca.cypherpunks.ru/, ca.cypherpunks.ru} certificate, through the
+@url{http://www.ca.cypherpunks.ru/, ca.cypherpunks.ru} certificate, through the
@url{https://datatracker.ietf.org/doc/html/rfc6698, DANE} record, that
in turn can be authenticated with
@url{https://dnscurve.org/, DNSCurve}-secured
@url{https://en.wikipedia.org/wiki/OpenPGP, OpenPGP}
@url{http://www.stargrave.org/Contacts.html, public key}.
+Also there is @url{https://yggdrasil-network.github.io/, Yggdrasil}
+accessible address: @url{http://y.www.nncpgo.org/}.
+
@table @asis
+@cindex quux.org
@item @url{https://nncp.mirrors.quux.org/}
Its creation @url{http://lists.cypherpunks.ru/archive/nncp-devel/2108/0310.html, announcement}.
@node MTH
+@cindex MTH
+@cindex hashing
+@cindex merkle tree
+@cindex BLAKE3
@unnumbered Merkle Tree Hashing
NNCP uses @url{https://github.com/BLAKE3-team/BLAKE3, BLAKE3} hash
@node Multicast
+@cindex multicast area
@unnumbered Multicast areas
NNCP has ability to multicast packets: send single packet to multiple
Each multicast group is identified by so-called @strong{area}. Area
consists of private/public Curve25519 keypairs for @ref{Encrypted area,
packets encryption}, identity (BLAKE2b-256 hash of the public key) and
-possible subscribers. Areas are created with @ref{nncp-cfgnew} command.
+possible subscribers. Areas are created with @command{@ref{nncp-cfgnew}}
+command.
You can make either file or exec transmissions to the areas. Those
ordinary file/exec packets are double wrapped in:
can should it to the subscribers further
@end itemize
+@vindex MsgHash
Area's message identity (@code{MsgHash}) is the hash of the encrypted
packet header. Because the area packet, containing the encrypted packet,
is relayed as-is without any modifications, that area message's hash
@enumerate
@item
-@command{nncp-file} creates an encrypted packet with area packet and
+@command{@ref{nncp-file}} creates an encrypted packet with area packet and
encrypted packet inside it, with our own @code{self} node as a recipient
(in the @file{SPOOL/SELF/tx} directory). It also creates the
@file{SPOOL/SELF/area/AREA/MsgHash} file.
@item
-@command{nncp-toss} sees @file{tx/} file and "opens" it, applying the
+@command{@ref{nncp-toss}} sees @file{tx/} file and "opens" it, applying the
area message tossing procedure as described above. That will create
outgoing packets in @file{SPOOL/nodeB/tx} and @file{SPOOL/nodeD/tx}
directories with @file{SPOOL/nodeB/area/AREA/MsgHash}
@node Новости
@section Новости
+@node Релиз 8.6.0
+@subsection Релиз 8.6.0
+@itemize
+
+@item
+@command{nncp-xfer} проверяет сходится ли контрольная сумма
+скопированного локально пакета и исходного.
+
+@item
+@command{nncp-call} имеет плохой код возврата если в очередях на приём и
+отправку остаются незаконченные задания.
+
+@item
+Появилась @command{nncp-ack} команда, которая отправляет явное
+подтверждение доставки пакета (ACK пакет). Это подтверждение удаляет
+упомянутый пакет из исходящего spool-а.
+
+@item
+Появилась возможность отключения @code{fsync} операции
+@env{$NNCPNOSYNC=1} переменной окружения.
+
+@item
+Добавлены разнообразные индексы в документации, что должно помочь при
+поиске в ней.
+
+@end itemize
+
@node Релиз 8.5.0
@subsection Релиз 8.5.0
@itemize
качестве обычного TCP сервера внутри overlay сети.
@item
-Конфигурационные строки Yggdrasil вида @code{prv;bind1,bind2;pub...}
+Конфигурационные строки Yggdrasil вида @code{prv;bind1,bind2;pub@dots{}}
заменены на URL-подобные (@code{yggdrasils://PRV?bind=BIND1&bind=BIND2&pub=PUB}).
@item
@itemize
@item
-Исправлено падение при fsync директорий после создания @file{.seen} файлов.
+Исправлено падение при @code{fsync} директорий после создания @file{.seen} файлов.
@end itemize
невозможности online аутентификации нод.
@item
-Явная синхронизация (fsync) директорий для гарантированного
+Явная синхронизация (@code{fsync}) директорий для гарантированного
переименования файлов.
@end itemize
нужно использовать @verb{|nncp-exec NODE sendmail RECIPIENT|}.
@item
@option{sendmail} опция конфигурационного файла заменена на более гибкую
- @option{exec}. @verb{|sendmail: [...]|} нужно заменить на @verb{|exec:
- sendmail: [...]|}.
+ @option{exec}. @code{sendmail: [@dots{}]} нужно заменить на
+ @code{exec: sendmail: [@dots{}]}.
@end itemize
@item
@node News
+@cindex news
@unnumbered News
See also this page @ref{Новости, on russian}.
+@node Release 8_6_0
+@section Release 8.6.0
+@itemize
+
+@item
+@command{nncp-xfer} checks if locally copied packet's checksum differs
+from the source's one.
+
+@item
+@command{nncp-call} has bad return code if its queues contain unfinished jobs.
+
+@item
+@command{nncp-ack} command appeared, that sends explicit packet receipt
+acknowledgement (ACK packet). That acknowledgement deletes referenced
+packet from the outbound spool.
+
+@item
+Ability to turn @code{fsync} operation off using @env{$NNCPNOSYNC=1}
+environment variable.
+
+@item
+Added various indices in documentation, that should help searching in it.
+
+@end itemize
+
@node Release 8_5_0
@section Release 8.5.0
@itemize
inside overlay network.
@item
-Yggdrasil's @code{prv;bind1,bind2;pub...}-like configuration strings are
+Yggdrasil's @code{prv;bind1,bind2;pub@dots{}}-like configuration strings are
replaced with URL-like ones (@code{yggdrasils://PRV?bind=BIND1&bind=BIND2&pub=PUB}).
@item
@itemize
@item
-Fixed failing directories fsync after @file{.seen} file creation.
+Fixed failing directories @code{fsync} after @file{.seen} file creation.
@end itemize
configuration files, that lead to inability to authenticate online peers.
@item
-Explicit directories fsync-ing for guaranteed files renaming.
+Explicit directories @code{fsync}-ing for guaranteed files renaming.
@end itemize
you must use @verb{|nncp-exec NODE sendmail RECIPIENT|}.
@item
@option{sendmail} configuration file option is replaced with
- @option{exec}. @verb{|sendmail: [...]|} must be replaced with
- @verb{|exec: sendmail: [...]|}.
+ @option{exec}. @code{sendmail: [@dots{}]} must be replaced with
+ @code{exec: sendmail: [@dots{}]}.
@end itemize
@item
@node Niceness
+@cindex niceness
+@cindex priority
@unnumbered Niceness
Each transmitted packet has niceness level, as Unix has @command{nice}
higher priority packets, like mail messages, will pass first, even when
lower priority packet was already been partially downloaded.
-There are default niceness levels built-in for @ref{nncp-exec},
-@ref{nncp-file} and @ref{nncp-freq} commands. But pay attention that it
-can give information about underlying payload to the adversary!
+There are default niceness levels built-in for
+@command{@ref{nncp-exec}}, @command{@ref{nncp-file}} and
+@command{@ref{nncp-freq}} commands. But pay attention that it can give
+information about underlying payload to the adversary!
There are 1-255 niceness levels. They could be specified either as
integer, or using aliases with delta modifiers:
@table @emph
+@vindex FLASH
@item FLASH (F)
Urgent priority.
+@vindex PRIORITY
@item PRIORITY (P)
High priority. Command execution/mail use that priority by default.
+@vindex NORMAL
@item NORMAL (N)
Normal priority. File requests use that priority by default.
+@vindex BULK
@item BULK (B)
Bundles shipped on a "least effort" basis. File transmission use that
priority by default.
@node Encrypted area
+@cindex encrypted area packet
@section Encrypted area packet
@ref{Multicast} area messages contains the encrypted packet, that is
@node Encrypted
+@cindex encrypted packet
+@cindex AEAD
+@cindex ChaCha20-Poly1305
@section Encrypted packet
Encrypted packets are the only files found in spools, in exchangeable
@node Packet
+@cindex packet format
+@cindex XDR
@unnumbered Packet format
All packets are
@node Plain
+@cindex plain packet
@section Plain packet
Plain packet contains either the whole file, or file request (freq), or
@item trns (transition)
@item exec-fat (uncompressed exec)
@item area (@ref{Multicast, multicast} area message)
+ @item ack (receipt acknowledgement)
@end enumerate
@item Niceness @tab
unsigned integer @tab
@item UTF-8 encoded, zero byte separated, exec's arguments
@item Node's id the transition packet must be relayed on
@item Multicast area's id
+ @item Packet's id (its @ref{MTH} hash)
@end itemize
@end multitable
compressed exec body
@item Whole encrypted packet we need to relay on
@item Multicast area message wrap with another encrypted packet inside
+@item Nothing, if it is acknowledgement packet
@end itemize
Also depending on packet's type, niceness level means:
@end example
See also @ref{Encrypted area, encrypted area packet}.
+@item ack
+@example
+ +------- PATH --------+
+ / \
++-------------------------+
+| PKT ID | 0x00 ... 0x00 |
++-------------------------+
+ \ /
+ PATHLEN
+@end example
+
@end table
@node Sources
+@cindex sources
+@cindex source code
+@cindex git
@section Development source code
Development source code contains the latest version of the code. It may
$ git clone --branch develop git://git.cypherpunks.ru/nncp.git nncp
@end example
-You can also use @url{https://git.cypherpunks.ru/nncp.git}.
+You can also use
+@url{https://git.cypherpunks.ru/nncp.git},
+@url{http://git.cypherpunks.ru/nncp.git},
+@url{http://y.git.cypherpunks.ru/nncp.git} URLs instead.
@node Sync
+@cindex sync protocol
+@cindex online protocol
+@cindex synchronization
@unnumbered Synchronization protocol
So-called synchronization protocol (SP) is used in current TCP daemon's
implementation. It is used for synchronizing @ref{Spool, spool}
directory contents between two nodes.
+@cindex XMODEM
It is aimed to be very simple and effective. It uses reliable transport
like TCP connections. It must be effective both on single-duplex and
full-duplex links: for example satellites have very high throughput but
@url{https://en.wikipedia.org/wiki/XMODEM, XMODEM} does, causes
unacceptable performance degradation.
+@vindex NNCPDEADLINE
Internally it uses various timeouts and deadlines. One of them used
extensively is 10 seconds default deadline timeout. You can override it
with @env{$NNCPDEADLINE} environment variable, that could be useful with
very high delay links.
+@cindex Noise-IK
SP works on top of
@url{http://noiseprotocol.org/noise.html#interactive-patterns,
@code{Noise_IK_25519_ChaChaPoly_BLAKE2b}} protocol. Each Noise packet
@table @emph
+@cindex HALT payload
@item HALT
Stop file transmission, empty sending queue on the remote side.
Actually @emph{HALT} packet does not have any body, only the header
+------+
@end verbatim
+@cindex PING payload
@item PING
Dummy packet only used for determining workability of the connection.
+------+
@end verbatim
+@cindex INFO payload
@item INFO
Information about the file we have for transmission.
Unique file identifier, its checksum
@end multitable
+@cindex FREQ payload
@item FREQ
File transmission request. Ask remote side to queue the file for
transmission.
Offset from which remote side must transmit the file
@end multitable
+@cindex FILE payload
@item FILE
Chunk of file.
Chunk of file itself
@end multitable
+@cindex DONE payload
@item DONE
Signal remote side that we have successfully downloaded the file.
@node Spool
+@cindex spool directory
@unnumbered Spool directory
Spool directory holds @ref{Encrypted, encrypted packets} received from
@table @file
+@cindex tmp directory
@item tmp
directory contains various temporary files that under normal
circumstances are renamed to necessary files inside other directories.
@item 2WHBV3TPZHDOZGUJEH563ZEK7M33J4UESRFO4PDKWD5KZNPROABQ
is an example Base32-encoded neighbour identifier.
+@cindex rx directory
+@cindex tx directory
@item rx, tx
directories are for incoming and outgoing encrypted packets. @file{rx}
contains currently unfinished, non-checked, unprocessed, etc packets.
+@cindex lock files
@item toss.lock, rx.lock, tx.lock
Lock files. Only single process can work with @file{rx}/@file{tx}
directories at once.
encoded @ref{MTH} hash of the whole contents. It can be integrity checked
anytime.
+@cindex part files
@item LYT64MWSNDK34CVYOO7TA6ZCJ3NWI2OUDBBMX2A4QWF34FIRY4DQ.part
is an example @strong{partly} received file. It can appear only when
online transfer is used. Its filename is sent by remote side and until
file is fully downloaded -- it plays no role.
+@cindex nock files
@item LYT64MWSNDK34CVYOO7TA6ZCJ3NWI2OUDBBMX2A4QWF34FIRY4DQ.nock
non-checksummed (NoCK) @strong{fully} received file. Its checksum is
-verified against its filename either by @ref{nncp-check}, or by working
-online daemons. If it is correct, then its extension is trimmed.
+verified against its filename either by @command{@ref{nncp-check}}, or
+by working online daemons. If it is correct, then its extension is trimmed.
+@cindex seen files
@item seen/LYT64MWSNDK34CVYOO7TA6ZCJ3NWI2OUDBBMX2A4QWF34FIRY4DQ
-@ref{nncp-toss} utility can be invoked with @option{-seen} option,
-leading to creation of @file{seen/} files, telling that the file with
-specified hash has already been processed before. It could be useful
-when there are use-cases where multiple ways of packets transfer
+@command{@ref{nncp-toss}} utility can be invoked with @option{-seen}
+option, leading to creation of @file{seen/} files, telling that the file
+with specified hash has already been processed before. It could be
+useful when there are use-cases where multiple ways of packets transfer
available and there is possibility of duplicates reception. You have to
manually remove them, when you do not need them (probably because they
are expired).
+@cindex hdr files
@anchor{HdrFile}
@item hdr/LYT64MWSNDK34CVYOO7TA6ZCJ3NWI2OUDBBMX2A4QWF34FIRY4DQ
If no @ref{CfgNoHdr, nohdr} option is enabled in configuration file,
then @file{hdr/} files are automatically created for every ordinary
(fully received and checksummed) packet. It literally contains just the
header of the corresponding packet. It will be automatically created
-even during simple @ref{nncp-stat} call. On filesystems with big
-blocksize (ZFS for example) it can greatly help listing the packets in
-directories, because it prevents unnecessary read-amplification. On
+even during simple @command{@ref{nncp-stat}} call. On filesystems with
+big blocksize (ZFS for example) it can greatly help listing the packets
+in directories, because it prevents unnecessary read-amplification. On
other filesystems probably it won't help at all, or even harm
performance.
There is a hack: you can create more dense @file{hdr/} allocation by
-removing all @file{hdr/} files and then running @command{nncp-stat},
+removing all @file{hdr/} files and then running @command{@ref{nncp-stat}},
that will recreate them. In many cases many @file{hdr/} files will be
allocated more or less linearly on the disk, decreasing listing time
even more.
@node Thanks
+@cindex thanks
@unnumbered Thanks
There are people deserving to be thanked for helping this project:
@emph{bob-airgap} доступен путём посылки промежуточного ретранслируемого
пакета через узел @emph{bob}.
-Любая команда типа @command{nncp-file myfile bob-airgap:} автоматически
+Любая команда типа @command{@ref{nncp-file} myfile bob-airgap:} автоматически
создаст инкапсулированный пакет: один непосредственно для целевой точки,
а другой несущий его для промежуточного узла.
пакете, кроме его полного размера и приоритета. Все промежуточные пакеты
тоже зашифрованы: используя хорошо известную технологию
@url{https://ru.wikipedia.org/wiki/%D0%9B%D1%83%D0%BA%D0%BE%D0%B2%D0%B0%D1%8F_%D0%BC%D0%B0%D1%80%D1%88%D1%80%D1%83%D1%82%D0%B8%D0%B7%D0%B0%D1%86%D0%B8%D1%8F,
-луковой маршрутизации}. @emph{bob} не может прочитать пакеты
-@emph{bob-airgap}.
+луковичного шифрования}. @emph{bob} не может прочитать пакеты @emph{bob-airgap}.
хотите проходить любому трафику когда узел доступен через ЛВС (LAN).
Вы легко можете настроить ваши предпочтения в @ref{Call, настройках
-звонков} для @ref{nncp-caller} команды, используемой при online связи.
+звонков} для @command{@ref{nncp-caller}} команды, используемой при
+online связи.
@verbatim
neigh: {
KISS}!
Просто скажите вашим обоим Postfix/Exim-ам (на сервере и ноутбуке)
-отправлять сообщения через NNCP (@ref{nncp-exec}) на заданный узел.
+отправлять сообщения через NNCP (@command{@ref{nncp-exec}}) на заданный узел.
Более подробно читайте для Postfix @ref{Postfix, здесь}, а для Exim
@ref{Exim, здесь}. Вся почта будет сохранятся в NNCP @ref{Spool, спуле},
который после обмена данных и распаковки вызовет локальный
Представьте, что вы послали два файла узлу @emph{bob}. Вставьте USB
устройство (SD гораздо предпочтительнее!) хранения, подмонтируйте и
-запустите @ref{nncp-xfer}:
+запустите @command{@ref{nncp-xfer}}:
@example
$ nncp-xfer -node bob /media/usbstick
@end example
чтобы найти все пакеты относящиеся к их узлу и локально скопируют для
-дальнейшей обработки. @command{nncp-xfer} это единственная команда
+дальнейшей обработки. @command{@ref{nncp-xfer}} это единственная команда
используемая с переносными устройствами хранения.
+
+Вы также можете опционально использовать явное подтверждение приёма
+пакетов, как описано в @command{@ref{nncp-ack}}.
@node UsecasePOPRU
@subsection Легковесная и быстрая замена POP3/IMAP4
-@ref{nncp-daemon} может быть соединён с @ref{nncp-caller} длительное
-время -- он создаёт TCP соединение на многие часы. Когда SMTP сервер
-получает письмо, то вызывает @ref{nncp-exec} для создания исходящего
+@command{@ref{nncp-daemon}} может быть соединён с
+@command{@ref{nncp-caller}} длительное время -- он создаёт TCP
+соединение на многие часы. Когда SMTP сервер получает письмо, то
+вызывает @command{@ref{nncp-exec}} для создания исходящего
зашифрованного пакета. Демон ежесекундно проверяет исходящую директорию
и сразу же посылает оповещение о недоставленных пакетах противоположной
стороне, которая сразу же их может скачать.
вы не "распаковываете" эти пакеты сразу же на том же самом устройстве.
Распаковка (чтение этих зашифрованных пакетов с извлечением переданных
файлов и почтовых сообщений) может и должна бы быть произведена на
-отдельном компьютере (@ref{nncp-cfgmin} команда может помочь с созданием
-конфигурационного файла без приватных ключей для этой цели).
+отдельном компьютере (@command{@ref{nncp-cfgmin}} команда может помочь с
+созданием конфигурационного файла без приватных ключей для этой цели).
Если вы действительно хотите взять с собой приватные ключи, то
-@ref{nncp-cfgenc} команда способна зашифровать ваш конфигурационный
-файл. Парольная фраза вами введённая усиливается функцией нагружающей и
-центральный процессор и память.
+@command{@ref{nncp-cfgenc}} команда способна зашифровать ваш
+конфигурационный файл. Парольная фраза вами введённая усиливается
+Ñ\84Ñ\83нкÑ\86ией нагÑ\80Ñ\83жаÑ\8eÑ\89ей и Ñ\86енÑ\82Ñ\80алÑ\8cнÑ\8bй пÑ\80оÑ\86еÑ\81Ñ\81оÑ\80 и памÑ\8fÑ\82Ñ\8c.
@node UsecaseAirgap
+@cindex air-gap
@section Highly secure isolated air-gap computers
If you worry much about security, then air-gapped computer could be the
@emph{bob-airgap} can be reached by sending intermediate relay packet
through the @emph{bob}.
-Any command like @command{nncp-file myfile bob-airgap:} will
+Any command like @command{@ref{nncp-file} myfile bob-airgap:} will
automatically create an encapsulated packet: one for the destination
endpoint, and other carrying it for intermediate relaying node.
Pay attention that relaying node knows nothing about the packet inside,
but just its size and priority. Transition packets are encrypted too:
using well-known @url{https://en.wikipedia.org/wiki/Onion_routing, onion
-routing} technology. @emph{bob} can not read @emph{bob-airgap}'s packets.
+encryption} technology. @emph{bob} can not read @emph{bob-airgap}'s packets.
through in anytime. Also you wish to pass any kind of traffic when the
node is available through the LAN.
-You can easily set your preferences in @ref{Call, call
-configurations} for @ref{nncp-caller} command used in online
-communications.
+You can easily set your preferences in @ref{Call, call configurations}
+for @command{@ref{nncp-caller}} command used in online communications.
@verbatim
neigh: {
@node Use cases
+@cindex use cases
@unnumbered Use cases
See also this page @ref{Сценарии, on russian}.
@url{https://en.wikipedia.org/wiki/KISS_principle, KISS}!
Just tell both of your Postfix/Exim (on the server and notebook) to drop
-email as a mail via NNCP (@ref{nncp-exec}) to specified node.
+email as a mail via NNCP (@command{@ref{nncp-exec}}) to specified node.
More information for Postfix is @ref{Postfix, here} and for Exim is
@ref{Exim, here}. All mail will be stored in NNCP @ref{Spool, spool},
@node UsecaseNoLink
+@cindex extreme environments
+@cindex lack of link
@section Extreme terrestrial environments, no link
This is some kind of too slow link. Offline delivery methods is the only
media for transferring packets to other nodes.
Assume that you send two files to @emph{bob} node. Insert USB storage
-device (SD is preferable!), mount it and run @ref{nncp-xfer}:
+device (SD is preferable!), mount it and run @command{@ref{nncp-xfer}}:
@example
$ nncp-xfer -node bob /media/usbstick
@end example
to find all packets related to their node and copy them locally for
-further processing. @command{nncp-xfer} is the only command used with
+further processing. @command{@ref{nncp-xfer}} is the only command used with
removable devices.
+
+You can also optionally wait for explicit packets receipt
+acknowledgement as described in @command{@ref{nncp-ack}}.
@node UsecasePOP
+@cindex POP3 replacement
+@cindex IMAP4 replacement
@section Lightweight fast POP3/IMAP4 replacement
-@ref{nncp-daemon} can be connected with @ref{nncp-caller} for a long
-time -- it can create TCP connection that lasts for many hours. When
-SMTP server receives mail, it will call @ref{nncp-exec} creating an
-outbound encrypted packet. Daemon checks outbound directory each second
-and immediately sends notification about undelivered packets to remote
-side, that also downloads it at once.
+@command{@ref{nncp-daemon}} can be connected with
+@command{@ref{nncp-caller}} for a long time -- it can create TCP
+connection that lasts for many hours. When SMTP server receives mail, it
+will call @command{@ref{nncp-exec}} creating an outbound encrypted
+packet. Daemon checks outbound directory each second and immediately
+sends notification about undelivered packets to remote side, that also
+downloads it at once.
There are only dozens of bytes notifying about incoming packets, dozens
of bytes telling to download those packets. Mail packets are compressed
@node UsecaseQoS
+@cindex expensive link
+@cindex slow link
+@cindex bad QoS
@section Slow/expensive link for high-volume data, bad QoS
Assume that you can give your relatively cheap 2 TiB removable hard
@node UsecaseSatelliteLinks
+@cindex satellite link
@section Satellite links
Satellite links have @strong{very} high delays together with high
@node UsecaseSpy
+@cindex reconnaissance, spying, intelligence, covert operations
@section Reconnaissance, spying, intelligence, covert agents
Those guys know how Internet is a dangerous place incompatible with
with it (don't you?), you do not "toss" those packets immediately on the
same device. Tossing (reading those encrypted packets and extracting
transferred files and mail messages) could and should be done on a
-separate computer (@ref{nncp-cfgmin} command could help creating
+separate computer (@command{@ref{nncp-cfgmin}} command could help creating
configuration file without private keys for that purpose).
-If you really want to carry your private keys, then @ref{nncp-cfgenc}
+If you really want to carry your private keys, then @command{@ref{nncp-cfgenc}}
command will be able to encrypt your configuration file. Passphrase you
enter is strengthened with both CPU and memory hard function.
@node UsecaseUnreliable
+@cindex unreliable link
@section Unreliable/expensive communication link
Assume that you have got slow modem/radio/cellular link that frequently
@node Workflow
+@cindex workflow
@unnumbered Workflow
NNCP consists of several utilities. As a rule you will have the
following workflow:
@enumerate
-@item Run @ref{nncp-cfgnew} on each node to create an initial
+
+@item Run @command{@ref{nncp-cfgnew}} on each node to create an initial
@ref{Configuration, configuration} file.
+
@item Tune it up and set at least @ref{Spool, spool} and log paths.
+
@item Share your public keys and reachability addressees with your
neighbours. Add their keys to your configuration file and do any other
required configuration about their reachability, permissions of file or
freq transmission.
-@item Use @ref{nncp-file}, @ref{nncp-freq}, @ref{nncp-exec}
-(look @ref{Postfix, how} Postfix and @ref{Exim, how} Exim SMTP servers
-could be configured) commands to queue file, freq and exec
-transmissions. Repeat as many times any time as you wish.
+
+@item Use @command{@ref{nncp-file}}, @command{@ref{nncp-freq}},
+@command{@ref{nncp-exec}} (look @ref{Postfix, how} Postfix and
+@ref{Exim, how} Exim SMTP servers could be configured) commands to queue
+file, freq and exec transmissions. Repeat as many times any time as you
+wish.
+
@item Depending on connection methods, either:
@itemize
- @item run @ref{nncp-daemon} to accept remotely initiated connections
- to your node
- @item run either @ref{nncp-call} or @ref{nncp-caller} to initiate
- connection to required nodes from time to time
- @item use @ref{nncp-xfer} with removable storage devices for copying
- packets to/from other nodes
- @item use @ref{nncp-bundle} with either sequential storage devices
- or broadcasting transmissions for copying packets
+ @item run @command{@ref{nncp-daemon}} to accept remotely initiated
+ connections to your node
+ @item run either @command{@ref{nncp-call}} or
+ @command{@ref{nncp-caller}} to initiate connection to required nodes
+ from time to time
+ @item use @command{@ref{nncp-xfer}} with removable storage devices
+ for copying packets to/from other nodes
+ @item use @command{@ref{nncp-bundle}} with either sequential storage
+ devices or broadcasting transmissions for copying packets
@end itemize
+
@item After successful packet exchanging (or just simply from time to
-time), run @ref{nncp-toss} for tossing (decrypting and processing) all
-inbound queues to receive exec messages, files, file requests and relay
-transition packets to other nodes.
+time), run @command{@ref{nncp-toss}} for tossing (decrypting and
+processing) all inbound queues to receive exec messages, files, file
+requests and relay transition packets to other nodes.
+
+@item Optionally do not forget about explicit receipt acknowledgement
+ability with @command{@ref{nncp-ack}}.
+
@end enumerate
@itemize
@item If you wish to encrypt your configuration file containing your
-private keys, then use @ref{nncp-cfgenc} utility. You can always use an
-encrypted config without decrypting it in temporary memory file.
+private keys, then use @command{@ref{nncp-cfgenc}} utility. You can
+always use an encrypted config without decrypting it in temporary memory file.
@item If you wish to strip off any private keys from your config, then
-use @ref{nncp-cfgmin} utility. It will be useful for transferring
+use @command{@ref{nncp-cfgmin}} utility. It will be useful for transferring
messages with offline methods, but tossing them later on the machine
with private keys.
@end itemize
@node Yggdrasil
+@cindex yggdrasil
@unnumbered Yggdrasil support
NNCP is able to act as a node of
full-featured TUN network interface, there is pure Go built-in stack,
responsible for IPv6 and TCP protocols support. You do not need to think
about network interfaces, addressing and firewall setup at all:
-@ref{nncp-daemon} acts as Yggdrasil IPv6 reachable host, listening on
-single TCP port. You can reach it using ordinary non-Yggdrasil capable
-version of @ref{nncp-call}, calling corresponding 200::/7 IPv6 address
-through native Yggdrasil daemon created TUN interface.
-@ref{nncp-daemon}, @ref{nncp-call}* can freely peer with Yggdrasil
-nodes, reusing existing infrastructure.
+@command{@ref{nncp-daemon}} acts as Yggdrasil IPv6 reachable host,
+listening on single TCP port. You can reach it using ordinary
+non-Yggdrasil capable version of @command{@ref{nncp-call}}, calling
+corresponding 200::/7 IPv6 address through native Yggdrasil daemon
+created TUN interface. @command{@ref{nncp-daemon}},
+@command{@ref{nncp-call}}* can freely peer with Yggdrasil nodes, reusing
+existing infrastructure.
Only minor modifications were done to current NNCP's tools:
@itemize
-@item @ref{nncp-daemon} has @option{-yggdrasil yggdrasils://} option,
-making it also as a Yggdrasil listener network node. It can
+@cindex yggdrasils schema
+@item @command{@ref{nncp-daemon}} has @option{-yggdrasil yggdrasils://}
+option, making it also as a Yggdrasil listener network node. It can
automatically connect to other peers and participate in routing. It does
not have to answer NNCP's online protocol requests at all and just can
be some intermediate routing point in the whole mesh network.
-@item @ref{nncp-call}/@ref{nncp-caller} commands understand
+@cindex yggdrasilc schema
+@item @command{@ref{nncp-call}}/@command{@ref{nncp-caller}} commands understand
@code{yggdrasilc://} addresses, pointing to the desired Yggdrasil's
public key (that also acts as the destination host's address). Yggdrasil
background goroutine is automatically started, connecting to the
specified Yggdrasil entrypoints, calling remote NNCP node and initiating
NNCP's native @ref{Sync, online protocol} handshake on top of that.
-@item @ref{nncp-cfgnew} is able to generate ed25519 keypair.
+@item @command{@ref{nncp-cfgnew}} is able to generate ed25519 keypair.
@item @ref{CfgYggdrasilAliases, Configuration file} optionally contains
@code{yggdrasil-aliases} map.
You should share that public key with other NNCP peers.
@item
-Start @ref{nncp-daemon} listening on Yggdrasil's incoming connections.
+Start @command{@ref{nncp-daemon}} listening on Yggdrasil's incoming connections.
You have to specify:
@itemize
+@vindex PrivateKey
@item
Your private key (generated above). Yggdrasil's @code{PrivateKey} analogue.
@item
Optional non-default port you will listen on Yggdrasil's IPv6 address.
+@vindex Listen
@item
Optional list of bind addresses, used for peering between the nodes.
Yggdrasil's @code{Listen} analogue.
+@vindex Peers
@item
Optional list of peer addresses you should connect to.
Yggdrasil's @code{Peers} analogue.
+@vindex AllowedPublicKeys
@item
Optional list of allowed peer public keys, allowed for incoming peering
connections from. Yggdrasil's @code{AllowedPublicKeys} analogue.
@settitle NEWS
@node News
@unnumbered News
-`sed -n '5,$p' < doc/news.texi`
+`sed -n '6,$p' < doc/news.texi`
@bye
EOF
mkinfo --output NEWS $texi
sed "/^Comment:/d ; s/ARMORED FILE/SIGNATURE/" > "$tarball".asc
meta4-create -file "$tarball" -mtime "$tarball" -sig "$tarball".asc \
http://www.nncpgo.org/download/"$tarball" \
+ http://y.www.nncpgo.org/download/"$tarball" \
https://nncp.mirrors.quux.org/download/"$tarball" > "$tarball".meta4
size=$(( $(stat -f %z $tarball) / 1024 ))
PORTNAME= nncp
-DISTVERSION= 7.6.0
+DISTVERSION= 8.6.0
CATEGORIES= net
MASTER_SITES= http://www.nncpgo.org/download/
+bin/nncp-ack
bin/nncp-bundle
bin/nncp-call
bin/nncp-caller
AutoTossNoExec bool
AutoTossNoTrns bool
AutoTossNoArea bool
+ AutoTossNoACK bool
}
func (ctx *Ctx) CallNode(
ctx.LogI("call-started", les, func(les LEs) string {
return fmt.Sprintf("Connection to %s (%s)", node.Name, addr)
})
- state.Wait()
+ isGood = state.Wait()
ctx.LogI("call-finished", append(
les,
LE{"Duration", int64(state.Duration.Seconds())},
humanize.IBytes(uint64(state.TxSpeed)),
)
})
- isGood = true
conn.Close()
break
} else {
CfgPathEnv = "NNCPCFG"
CfgSpoolEnv = "NNCPSPOOL"
CfgLogEnv = "NNCPLOG"
+ CfgNoSync = "NNCPNOSYNC"
)
var (
--- /dev/null
+/*
+NNCP -- Node to Node copy, utilities for store-and-forward data exchange
+Copyright (C) 2016-2022 Sergey Matveev <stargrave@stargrave.org>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, version 3 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+// Send packet receipt acknowledgement via NNCP.
+package main
+
+import (
+ "flag"
+ "fmt"
+ "log"
+ "os"
+ "path/filepath"
+ "strings"
+
+ "go.cypherpunks.ru/nncp/v8"
+)
+
+func usage() {
+ fmt.Fprintf(os.Stderr, nncp.UsageHeader())
+ fmt.Fprintf(os.Stderr, "nncp-ack -- send packet receipt acknowledgement\n\n")
+ fmt.Fprintf(os.Stderr, "Usage: %s [options] -all\n", os.Args[0])
+ fmt.Fprintf(os.Stderr, "Usage: %s -node NODE[,...]\n", os.Args[0])
+ fmt.Fprintf(os.Stderr, "Usage: %s -node NODE -pkt PKT\n", os.Args[0])
+ fmt.Fprintln(os.Stderr, "Options:")
+ flag.PrintDefaults()
+}
+
+func main() {
+ var (
+ cfgPath = flag.String("cfg", nncp.DefaultCfgPath, "Path to configuration file")
+ niceRaw = flag.String("nice", nncp.NicenessFmt(nncp.DefaultNiceFreq), "Outbound packet niceness")
+ minSizeRaw = flag.Uint64("minsize", 0, "Minimal required resulting packet size, in KiB")
+ viaOverride = flag.String("via", "", "Override Via path to destination node (ignored with -all)")
+ spoolPath = flag.String("spool", "", "Override path to spool")
+ logPath = flag.String("log", "", "Override path to logfile")
+ doAll = flag.Bool("all", false, "ACK all rx packet for all nodes")
+ nodesRaw = flag.String("node", "", "ACK rx packets for that node")
+ pktRaw = flag.String("pkt", "", "ACK only that packet")
+ quiet = flag.Bool("quiet", false, "Print only errors")
+ showPrgrs = flag.Bool("progress", false, "Force progress showing")
+ omitPrgrs = flag.Bool("noprogress", false, "Omit progress showing")
+ debug = flag.Bool("debug", false, "Print debug messages")
+ version = flag.Bool("version", false, "Print version information")
+ warranty = flag.Bool("warranty", false, "Print warranty information")
+ )
+ log.SetFlags(log.Lshortfile)
+ flag.Usage = usage
+ flag.Parse()
+ if *warranty {
+ fmt.Println(nncp.Warranty)
+ return
+ }
+ if *version {
+ fmt.Println(nncp.VersionGet())
+ return
+ }
+ nice, err := nncp.NicenessParse(*niceRaw)
+ if err != nil {
+ log.Fatalln(err)
+ }
+
+ ctx, err := nncp.CtxFromCmdline(
+ *cfgPath,
+ *spoolPath,
+ *logPath,
+ *quiet,
+ *showPrgrs,
+ *omitPrgrs,
+ *debug,
+ )
+ if err != nil {
+ log.Fatalln("Error during initialization:", err)
+ }
+ if ctx.Self == nil {
+ log.Fatalln("Config lacks private keys")
+ }
+
+ ctx.Umask()
+ minSize := int64(*minSizeRaw) * 1024
+
+ var nodes []*nncp.Node
+ if *nodesRaw != "" {
+ for _, nodeRaw := range strings.Split(*nodesRaw, ",") {
+ node, err := ctx.FindNode(nodeRaw)
+ if err != nil {
+ log.Fatalln("Invalid -node specified:", err)
+ }
+ nodes = append(nodes, node)
+ }
+ }
+ if *doAll {
+ if len(nodes) != 0 {
+ usage()
+ os.Exit(1)
+ }
+ for _, node := range ctx.Neigh {
+ nodes = append(nodes, node)
+ }
+ } else if len(nodes) == 0 {
+ usage()
+ os.Exit(1)
+ }
+
+ if *pktRaw != "" {
+ if len(nodes) != 1 {
+ usage()
+ os.Exit(1)
+ }
+ nncp.ViaOverride(*viaOverride, ctx, nodes[0])
+ if err = ctx.TxACK(nodes[0], nice, *pktRaw, minSize); err != nil {
+ log.Fatalln(err)
+ }
+ return
+ }
+
+ for _, node := range nodes {
+ for job := range ctx.Jobs(node.Id, nncp.TRx) {
+ pktName := filepath.Base(job.Path)
+ if err = ctx.TxACK(node, nice, pktName, minSize); err != nil {
+ log.Fatalln(err)
+ }
+ }
+ }
+}
if err = bufTmp.Flush(); err != nil {
log.Fatalln("Error during flushing:", err)
}
- if err = tmp.Sync(); err != nil {
- log.Fatalln("Error during syncing:", err)
+ if !nncp.NoSync {
+ if err = tmp.Sync(); err != nil {
+ log.Fatalln("Error during syncing:", err)
+ }
}
if err = tmp.Close(); err != nil {
log.Fatalln("Error during closing:", err)
autoTossNoExec = flag.Bool("autotoss-noexec", false, "Do not process \"exec\" packets during tossing")
autoTossNoTrns = flag.Bool("autotoss-notrns", false, "Do not process \"trns\" packets during tossing")
autoTossNoArea = flag.Bool("autotoss-noarea", false, "Do not process \"area\" packets during tossing")
+ autoTossNoACK = flag.Bool("autotoss-noack", false, "Do not process \"ack\" packets during tossing")
)
log.SetFlags(log.Lshortfile)
flag.Usage = usage
*autoTossNoExec,
*autoTossNoTrns,
*autoTossNoArea,
+ *autoTossNoACK,
)
}
autoTossNoExec = flag.Bool("autotoss-noexec", false, "Do not process \"exec\" packets during tossing")
autoTossNoTrns = flag.Bool("autotoss-notrns", false, "Do not process \"trns\" packets during tossing")
autoTossNoArea = flag.Bool("autotoss-noarea", false, "Do not process \"area\" packets during tossing")
+ autoTossNoACK = flag.Bool("autotoss-noack", false, "Do not process \"ack\" packets during tossing")
)
log.SetFlags(log.Lshortfile)
flag.Usage = usage
call.AutoTossNoExec || *autoTossNoExec,
call.AutoTossNoTrns || *autoTossNoTrns,
call.AutoTossNoArea || *autoTossNoArea,
+ call.AutoTossNoACK || *autoTossNoACK,
)
}
autoTossNoExec = flag.Bool("autotoss-noexec", false, "Do not process \"exec\" packets during tossing")
autoTossNoTrns = flag.Bool("autotoss-notrns", false, "Do not process \"trns\" packets during tossing")
autoTossNoArea = flag.Bool("autotoss-noarea", false, "Do not process \"area\" packets during tossing")
+ autoTossNoACK = flag.Bool("autotoss-noack", false, "Do not process \"ack\" packets during tossing")
)
log.SetFlags(log.Lshortfile)
flag.Usage = usage
*autoTossNoExec,
*autoTossNoTrns,
*autoTossNoArea,
+ *autoTossNoACK,
)
}
<-nodeIdC // call completion
*autoTossNoExec,
*autoTossNoTrns,
*autoTossNoArea,
+ *autoTossNoACK,
)
}
<-nodeIdC // call completion
payloadType = "exec uncompressed"
case nncp.PktTypeArea:
payloadType = "area"
+ case nncp.PktTypeACK:
+ payloadType = "acknowledgement"
}
var path string
switch pkt.Type {
if areaId, err := nncp.AreaIdFromString(path); err == nil {
path = fmt.Sprintf("%s (%s)", path, ctx.AreaName(areaId))
}
+ case nncp.PktTypeACK:
+ path = nncp.Base32Codec.EncodeToString(pkt.Path[:pkt.PathLen])
default:
path = string(pkt.Path[:pkt.PathLen])
}
log.Fatalln("Can not flush:", err)
}
if tmp != nil {
- if err = tmp.Sync(); err != nil {
- log.Fatalln("Can not sync:", err)
+ if !nncp.NoSync {
+ if err = tmp.Sync(); err != nil {
+ log.Fatalln("Can not sync:", err)
+ }
}
if err = tmp.Close(); err != nil {
log.Fatalln("Can not close:", err)
noExec = flag.Bool("noexec", false, "Do not process \"exec\" packets")
noTrns = flag.Bool("notrns", false, "Do not process \"transitional\" packets")
noArea = flag.Bool("noarea", false, "Do not process \"area\" packets")
+ noACK = flag.Bool("noack", false, "Do not process \"ack\" packets")
spoolPath = flag.String("spool", "", "Override path to spool")
logPath = flag.String("log", "", "Override path to logfile")
quiet = flag.Bool("quiet", false, "Print only errors")
node.Id,
nncp.TRx,
nice,
- *dryRun, *doSeen, *noFile, *noFreq, *noExec, *noTrns, *noArea,
+ *dryRun, *doSeen, *noFile, *noFreq, *noExec, *noTrns, *noArea, *noACK,
) || isBad
if nodeId == *ctx.SelfId {
isBad = ctx.Toss(
node.Id,
nncp.TTx,
nice,
- *dryRun, false, true, true, true, true, *noArea,
+ *dryRun, false, true, true, true, true, *noArea, *noACK,
) || isBad
}
}
nodeId,
nncp.TRx,
nice,
- *dryRun, *doSeen, *noFile, *noFreq, *noExec, *noTrns, *noArea,
+ *dryRun, *doSeen, *noFile, *noFreq, *noExec, *noTrns, *noArea, *noACK,
)
if *nodeId == *ctx.SelfId {
ctx.Toss(
nodeId,
nncp.TTx,
nice,
- *dryRun, false, true, true, true, true, *noArea,
+ *dryRun, false, true, true, true, true, *noArea, *noACK,
)
}
}
fd.Close()
continue
}
- if _, err = fd.Seek(0, 0); err != nil {
+ if _, err = fd.Seek(0, io.SeekStart); err != nil {
log.Fatalln(err)
}
tmp, err := ctx.NewTmpFileWHash()
w.CloseWithError(err)
}
}()
- if _, err = nncp.CopyProgressed(
+ _, err = nncp.CopyProgressed(
tmp.W, r, "Rx",
append(
les,
nncp.LE{K: "FullSize", V: fiInt.Size()},
),
ctx.ShowPrgrs,
- ); err != nil {
+ )
+ fd.Close()
+ if err != nil {
ctx.LogE("xfer-rx", les, err, logMsg)
+ tmp.Cancel()
isBad = true
+ continue
}
- fd.Close()
- if isBad {
+ if err = tmp.W.Flush(); err != nil {
+ ctx.LogE("xfer-rx", les, err, logMsg)
+ tmp.Cancel()
+ isBad = true
+ continue
+ }
+ if tmp.Checksum() != fiInt.Name() {
+ ctx.LogE("xfer-rx", les, errors.New("checksum mismatch"), logMsg)
tmp.Cancel()
+ isBad = true
continue
}
if err = tmp.Commit(filepath.Join(
isBad = true
continue
}
- if err = tmp.Sync(); err != nil {
- tmp.Close()
- ctx.LogE("xfer-tx-sync", les, err, func(les nncp.LEs) string {
- return logMsg(les) + ": syncing"
- })
- isBad = true
- continue
+ if !nncp.NoSync {
+ if err = tmp.Sync(); err != nil {
+ tmp.Close()
+ ctx.LogE("xfer-tx-sync", les, err, func(les nncp.LEs) string {
+ return logMsg(les) + ": syncing"
+ })
+ isBad = true
+ continue
+ }
}
if err = tmp.Close(); err != nil {
ctx.LogE("xfer-tx-close", les, err, func(les nncp.LEs) string {
github.com/google/btree v1.0.1 // indirect
github.com/gorhill/cronexpr v0.0.0-20180427100037-88b0669f7d75
github.com/hjson/hjson-go v3.1.0+incompatible
- github.com/klauspost/compress v1.13.6
+ github.com/klauspost/compress v1.14.4
+ github.com/klauspost/cpuid/v2 v2.0.11 // indirect
github.com/yggdrasil-network/yggdrasil-go v0.4.2
go.cypherpunks.ru/balloon v1.1.1
go.cypherpunks.ru/recfile v0.4.3
- golang.org/x/crypto v0.0.0-20210921155107-089bfa567519
- golang.org/x/net v0.0.0-20210614182718-04defd469f4e
- golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e
+ golang.org/x/crypto v0.0.0-20220214200702-86341886e292
+ golang.org/x/net v0.0.0-20220225172249-27dd8689420f
+ golang.org/x/sys v0.0.0-20220227234510-4e6760a101f9
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211
golang.org/x/time v0.0.0-20191024005414-555d28b269f0 // indirect
inet.af/netstack v0.0.0-20211120045802-8aa80cf23d3c
- lukechampine.com/blake3 v1.1.6
+ lukechampine.com/blake3 v1.1.7
)
go 1.13
github.com/hjson/hjson-go v3.1.0+incompatible h1:DY/9yE8ey8Zv22bY+mHV1uk2yRy0h8tKhZ77hEdi0Aw=
github.com/hjson/hjson-go v3.1.0+incompatible/go.mod h1:qsetwF8NlsTsOTwZTApNlTCerV+b2GjYRRcIk4JMFio=
github.com/kardianos/minwinsvc v1.0.0/go.mod h1:Bgd0oc+D0Qo3bBytmNtyRKVlp85dAloLKhfxanPFFRc=
-github.com/klauspost/compress v1.13.6 h1:P76CopJELS0TiO2mebmnzgWaajssP/EszplttgQxcgc=
-github.com/klauspost/compress v1.13.6/go.mod h1:/3/Vjq9QcHkK5uEr5lBEmyoZ1iFhe47etQ6QUkpK6sk=
-github.com/klauspost/cpuid/v2 v2.0.9 h1:lgaqFMSdTdQYdZ04uHyN2d/eKdOMyi2YLSvlQIBFYa4=
+github.com/klauspost/compress v1.14.4 h1:eijASRJcobkVtSt81Olfh7JX43osYLwy5krOJo6YEu4=
+github.com/klauspost/compress v1.14.4/go.mod h1:/3/Vjq9QcHkK5uEr5lBEmyoZ1iFhe47etQ6QUkpK6sk=
github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
+github.com/klauspost/cpuid/v2 v2.0.11 h1:i2lw1Pm7Yi/4O6XCSyJWqEHI2MDw2FzUK6o/D21xn2A=
+github.com/klauspost/cpuid/v2 v2.0.11/go.mod h1:g2LTdtYhdyuGPqyWyv7qRAmj1WBqxuObKfj5c0PQa7c=
github.com/kr/pretty v0.2.1 h1:Fmg33tUaq4/8ym9TJN1x7sLJnHVwhP33CNkpYV/7rwI=
github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
golang.org/x/crypto v0.0.0-20210421170649-83a5a9bb288b/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4=
golang.org/x/crypto v0.0.0-20210506145944-38f3c27a63bf/go.mod h1:P+XmwS30IXTQdn5tA2iutPOUgjI07+tq3H3K9MVA1s8=
golang.org/x/crypto v0.0.0-20210513164829-c07d793c2f9a/go.mod h1:P+XmwS30IXTQdn5tA2iutPOUgjI07+tq3H3K9MVA1s8=
-golang.org/x/crypto v0.0.0-20210921155107-089bfa567519 h1:7I4JAnoQBe7ZtJcBaYHi5UtiO8tQHbUSXxL+pnGRANg=
-golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
+golang.org/x/crypto v0.0.0-20220214200702-86341886e292 h1:f+lwQ+GtmgoY+A2YaQxlSOnDjXcQ7ZRLWOHbC6HtRqE=
+golang.org/x/crypto v0.0.0-20220214200702-86341886e292/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20210510120150-4163338589ed/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20210610132358-84b48f89b13b/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
-golang.org/x/net v0.0.0-20210614182718-04defd469f4e h1:XpT3nA5TvE525Ne3hInMh6+GETgn27Zfm9dxsThnX2Q=
-golang.org/x/net v0.0.0-20210614182718-04defd469f4e/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
+golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
+golang.org/x/net v0.0.0-20220225172249-27dd8689420f h1:oA4XRj0qtSt8Yo1Zms0CUlsT3KG69V2UGQWPBxujDmc=
+golang.org/x/net v0.0.0-20220225172249-27dd8689420f/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190606203320-7fc4e5ec1444/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210611083646-a4fc73990273/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e h1:fLOSk5Q00efkSvAm+4xcoXD+RRmLmmulPn5I3Y9F2EM=
golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20220227234510-4e6760a101f9 h1:nhht2DYV/Sn3qOayu8lM+cU1ii9sTLUeBQwQQfUHtrs=
+golang.org/x/sys v0.0.0-20220227234510-4e6760a101f9/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 h1:JGgROgKl9N8DuW20oFS5gxc+lE67/N3FcwmBPMe7ArY=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7-0.20210503195748-5c7c50ebbd4f/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
+golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0 h1:/5xXl8Y5W96D+TtHSlonuFqGHIWVuyCkGJLwGh9JJFs=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
inet.af/netstack v0.0.0-20211120045802-8aa80cf23d3c h1:nr31qYr+91rWD8klUkPx3eGTZzumCC414UJG1QRKZTc=
inet.af/netstack v0.0.0-20211120045802-8aa80cf23d3c/go.mod h1:KOJdAzQzMLKzwFEdOOnrnSrLIhaFVB+NQoME/e5wllA=
-lukechampine.com/blake3 v1.1.6 h1:H3cROdztr7RCfoaTpGZFQsrqvweFLrqS73j7L7cmR5c=
-lukechampine.com/blake3 v1.1.6/go.mod h1:tkKEOtDkNtklkXtLNEOGNq5tcV90tJiA1vAA12R78LA=
+lukechampine.com/blake3 v1.1.7 h1:GgRMhmdsuK8+ii6UZFDL8Nb+VyMwadAgcJyfYHxG6n0=
+lukechampine.com/blake3 v1.1.7/go.mod h1:tkKEOtDkNtklkXtLNEOGNq5tcV90tJiA1vAA12R78LA=
const Base32Encoded32Len = 52
var (
- Version string = "8.5.0"
+ Version string = "8.6.0"
Base32Codec *base32.Encoding = base32.StdEncoding.WithPadding(base32.NoPadding)
)
PktTypeTrns PktType = iota
PktTypeExecFat PktType = iota
PktTypeArea PktType = iota
+ PktTypeACK PktType = iota
MaxPathSize = 1<<8 - 1
return nil
}
-func (state *SPState) Wait() {
+func (state *SPState) Wait() bool {
state.wg.Wait()
close(state.payloads)
close(state.pings)
if txDuration > 0 {
state.TxSpeed = state.TxBytes / txDuration
}
+ nothingLeft := len(state.queueTheir) == 0
for _, s := range state.fds {
+ nothingLeft = false
s.fd.Close()
}
for pktName := range state.progressBars {
ProgressKill(pktName)
}
+ return nothingLeft
}
func (state *SPState) ProcessSP(payload []byte) ([][]byte, error) {
humanize.IBytes(uint64(fullsize)),
)
}
- err = fd.Sync()
- if err != nil {
- state.Ctx.LogE("sp-file-sync", lesp, err, func(les LEs) string {
- return logMsg(les) + ": syncing"
- })
- state.closeFd(filePathPart)
- continue
+ if !NoSync {
+ err = fd.Sync()
+ if err != nil {
+ state.Ctx.LogE("sp-file-sync", lesp, err, func(les LEs) string {
+ return logMsg(les) + ": syncing"
+ })
+ state.closeFd(filePathPart)
+ continue
+ }
}
if hasherAndOffset != nil {
delete(state.fileHashers, filePath)
"time"
)
+var NoSync bool
+
+func init() {
+ NoSync = os.Getenv(CfgNoSync) != ""
+}
+
func TempFile(dir, prefix string) (*os.File, error) {
// Assume that probability of suffix collision is negligible
suffix := strconv.FormatInt(time.Now().UnixNano()+int64(os.Getpid()), 16)
}
func DirSync(dirPath string) error {
+ if NoSync {
+ return nil
+ }
fd, err := os.Open(dirPath)
if err != nil {
return err
tmp.Fd.Close()
return err
}
- if err = tmp.Fd.Sync(); err != nil {
- tmp.Fd.Close()
- return err
+ if !NoSync {
+ if err = tmp.Fd.Sync(); err != nil {
+ tmp.Fd.Close()
+ return err
+ }
}
if err = tmp.Fd.Close(); err != nil {
return err
pktSize uint64,
jobPath string,
decompressor *zstd.Decoder,
- dryRun, doSeen, noFile, noFreq, noExec, noTrns, noArea bool,
+ dryRun, doSeen, noFile, noFreq, noExec, noTrns, noArea, noACK bool,
) error {
defer pipeR.Close()
sendmail := ctx.Neigh[*ctx.SelfId].Exec["sendmail"]
})
return err
}
- if err = tmp.Sync(); err != nil {
- tmp.Close()
- ctx.LogE("rx-sync", les, err, func(les LEs) string {
- return fmt.Sprintf(
- "Tossing file %s/%s (%s): %s: syncing",
- sender.Name, pktName,
- humanize.IBytes(pktSize), dst,
- )
- })
- return err
+ if !NoSync {
+ if err = tmp.Sync(); err != nil {
+ tmp.Close()
+ ctx.LogE("rx-sync", les, err, func(les LEs) string {
+ return fmt.Sprintf(
+ "Tossing file %s/%s (%s): %s: syncing",
+ sender.Name, pktName,
+ humanize.IBytes(pktSize), dst,
+ )
+ })
+ return err
+ }
}
if err = tmp.Close(); err != nil {
ctx.LogE("rx-close", les, err, func(les LEs) string {
uint64(pktSizeWithoutEnc(int64(pktSize))),
"",
decompressor,
- dryRun, doSeen, noFile, noFreq, noExec, noTrns, noArea,
+ dryRun, doSeen, noFile, noFreq, noExec, noTrns, noArea, noACK,
)
}()
_, _, _, err = PktEncRead(
}
}
+ case PktTypeACK:
+ if noACK {
+ return nil
+ }
+ hsh := Base32Codec.EncodeToString(pkt.Path[:MTHSize])
+ les := append(les, LE{"Type", "ack"}, LE{"Pkt", hsh})
+ logMsg := func(les LEs) string {
+ return fmt.Sprintf("Tossing ack %s/%s: %s", sender.Name, pktName, hsh)
+ }
+ ctx.LogD("rx-ack", les, logMsg)
+ pktPath := filepath.Join(ctx.Spool, sender.Id.String(), string(TTx), hsh)
+ if _, err := os.Stat(pktPath); err == nil {
+ if !dryRun {
+ if err = os.Remove(pktPath); err != nil {
+ ctx.LogE("rx-ack", les, err, func(les LEs) string {
+ return logMsg(les) + ": removing packet"
+ })
+ return err
+ }
+ }
+ } else {
+ ctx.LogD("rx-ack", les, func(les LEs) string {
+ return logMsg(les) + ": already disappeared"
+ })
+ }
+ if !dryRun && doSeen {
+ if err := ensureDir(filepath.Dir(jobPath), SeenDir); err != nil {
+ return err
+ }
+ if fd, err := os.Create(jobPath2Seen(jobPath)); err == nil {
+ fd.Close()
+ if err = DirSync(filepath.Dir(jobPath)); err != nil {
+ ctx.LogE("rx-dirsync", les, err, func(les LEs) string {
+ return fmt.Sprintf(
+ "Tossing file %s/%s (%s): %s: dirsyncing",
+ sender.Name, pktName,
+ humanize.IBytes(pktSize),
+ filepath.Base(jobPath),
+ )
+ })
+ return err
+ }
+ }
+ }
+ if !dryRun {
+ if err = os.Remove(jobPath); err != nil {
+ ctx.LogE("rx", les, err, func(les LEs) string {
+ return logMsg(les) + ": removing job"
+ })
+ return err
+ } else if ctx.HdrUsage {
+ os.Remove(JobPath2Hdr(jobPath))
+ }
+ }
+ ctx.LogI("rx", les, func(les LEs) string {
+ return fmt.Sprintf("Got ACK packet from %s of %s", sender.Name, hsh)
+ })
+
default:
err = errors.New("unknown type")
ctx.LogE(
nodeId *NodeId,
xx TRxTx,
nice uint8,
- dryRun, doSeen, noFile, noFreq, noExec, noTrns, noArea bool,
+ dryRun, doSeen, noFile, noFreq, noExec, noTrns, noArea, noACK bool,
) bool {
dirLock, err := ctx.LockDir(nodeId, "toss")
if err != nil {
uint64(pktSizeWithoutEnc(job.Size)),
job.Path,
decompressor,
- dryRun, doSeen, noFile, noFreq, noExec, noTrns, noArea,
+ dryRun, doSeen, noFile, noFreq, noExec, noTrns, noArea, noACK,
)
}()
pipeWB := bufio.NewWriter(pipeW)
func (ctx *Ctx) AutoToss(
nodeId *NodeId,
nice uint8,
- doSeen, noFile, noFreq, noExec, noTrns, noArea bool,
+ doSeen, noFile, noFreq, noExec, noTrns, noArea, noACK bool,
) (chan struct{}, chan bool) {
dw, err := ctx.NewDirWatcher(
filepath.Join(ctx.Spool, nodeId.String(), string(TRx)),
case <-dw.C:
bad = !ctx.Toss(
nodeId, TRx, nice, false,
- doSeen, noFile, noFreq, noExec, noTrns, noArea) || bad
+ doSeen, noFile, noFreq, noExec, noTrns, noArea, noACK) || bad
}
}
}()
continue
}
ctx.Toss(ctx.Self.Id, TRx, DefaultNiceExec-1,
- false, false, false, false, false, false, false)
+ false, false, false, false, false, false, false, false)
if len(dirFiles(rxPath)) == 0 {
return false
}
ctx.Neigh[*nodeOur.Id].Exec = make(map[string][]string)
ctx.Neigh[*nodeOur.Id].Exec[handle] = []string{"/bin/sh", "-c", "false"}
ctx.Toss(ctx.Self.Id, TRx, DefaultNiceExec,
- false, false, false, false, false, false, false)
+ false, false, false, false, false, false, false, false)
if len(dirFiles(rxPath)) == 0 {
return false
}
),
}
ctx.Toss(ctx.Self.Id, TRx, DefaultNiceExec,
- false, false, false, false, false, false, false)
+ false, false, false, false, false, false, false, false)
if len(dirFiles(rxPath)) != 0 {
return false
}
rxPath := filepath.Join(spool, ctx.Self.Id.String(), string(TRx))
os.Rename(filepath.Join(spool, ctx.Self.Id.String(), string(TTx)), rxPath)
ctx.Toss(ctx.Self.Id, TRx, DefaultNiceFile,
- false, false, false, false, false, false, false)
+ false, false, false, false, false, false, false, false)
if len(dirFiles(rxPath)) == 0 {
return false
}
ctx.Neigh[*nodeOur.Id].Incoming = &incomingPath
if ctx.Toss(ctx.Self.Id, TRx, DefaultNiceFile,
- false, false, false, false, false, false, false) {
+ false, false, false, false, false, false, false, false) {
return false
}
if len(dirFiles(rxPath)) != 0 {
os.Rename(filepath.Join(spool, ctx.Self.Id.String(), string(TTx)), rxPath)
ctx.Neigh[*nodeOur.Id].Incoming = &incomingPath
ctx.Toss(ctx.Self.Id, TRx, DefaultNiceFile,
- false, false, false, false, false, false, false)
+ false, false, false, false, false, false, false, false)
expected := make(map[string]struct{})
expected["samefile"] = struct{}{}
for i := 0; i < files-1; i++ {
os.Rename(txPath, rxPath)
os.MkdirAll(txPath, os.FileMode(0700))
ctx.Toss(ctx.Self.Id, TRx, DefaultNiceFreq,
- false, false, false, false, false, false, false)
+ false, false, false, false, false, false, false, false)
if len(dirFiles(txPath)) != 0 || len(dirFiles(rxPath)) == 0 {
return false
}
ctx.Neigh[*nodeOur.Id].FreqPath = &spool
ctx.Toss(ctx.Self.Id, TRx, DefaultNiceFreq,
- false, false, false, false, false, false, false)
+ false, false, false, false, false, false, false, false)
if len(dirFiles(txPath)) != 0 || len(dirFiles(rxPath)) == 0 {
return false
}
}
}
ctx.Toss(ctx.Self.Id, TRx, DefaultNiceFreq,
- false, false, false, false, false, false, false)
+ false, false, false, false, false, false, false, false)
if len(dirFiles(txPath)) == 0 || len(dirFiles(rxPath)) != 0 {
return false
}
}
}
ctx.Toss(ctx.Self.Id, TRx, 123,
- false, false, false, false, false, false, false)
+ false, false, false, false, false, false, false, false)
if len(dirFiles(rxPath)) != 0 {
return false
}
}
logMsg := func(les LEs) string {
return fmt.Sprintf(
- "File %s (%s) sent to %s:%s",
+ "File %s (%s) is sent to %s:%s",
srcPath,
humanize.IBytes(uint64(finalSize)),
ctx.NodeName(node.Id),
}
logMsg := func(les LEs) string {
return fmt.Sprintf(
- "File %s (%s) sent to %s:%s",
+ "File %s (%s) is sent to %s:%s",
srcPath,
humanize.IBytes(uint64(size)),
ctx.NodeName(node.Id),
}
logMsg := func(les LEs) string {
return fmt.Sprintf(
- "File %s (%s) sent to %s:%s",
+ "File %s (%s) is sent to %s:%s",
srcPath,
humanize.IBytes(uint64(metaPktSize)),
ctx.NodeName(node.Id),
}
logMsg := func(les LEs) string {
return fmt.Sprintf(
- "File request from %s:%s to %s sent",
+ "File request from %s:%s to %s is sent",
ctx.NodeName(node.Id), srcPath,
dstPath,
)
}
logMsg := func(les LEs) string {
return fmt.Sprintf(
- "Exec sent to %s@%s (%s)",
+ "Exec is sent to %s@%s (%s)",
ctx.NodeName(node.Id), dst, humanize.IBytes(uint64(size)),
)
}
os.Symlink(nodePath, filepath.Join(ctx.Spool, node.Name))
return err
}
+
+func (ctx *Ctx) TxACK(
+ node *Node,
+ nice uint8,
+ hsh string,
+ minSize int64,
+) error {
+ hshRaw, err := Base32Codec.DecodeString(hsh)
+ if err != nil {
+ return err
+ }
+ if len(hshRaw) != MTHSize {
+ return errors.New("Invalid packet id size")
+ }
+ pkt, err := NewPkt(PktTypeACK, nice, []byte(hshRaw))
+ if err != nil {
+ return err
+ }
+ src := bytes.NewReader([]byte{})
+ _, _, err = ctx.Tx(node, pkt, nice, 0, minSize, MaxFileSize, src, hsh, nil)
+ les := LEs{
+ {"Type", "ack"},
+ {"Node", node.Id},
+ {"Nice", int(nice)},
+ {"Pkt", hsh},
+ }
+ logMsg := func(les LEs) string {
+ return fmt.Sprintf("ACK to %s of %s is sent", ctx.NodeName(node.Id), hsh)
+ }
+ if err == nil {
+ ctx.LogI("tx", les, logMsg)
+ } else {
+ ctx.LogE("tx", les, err, logMsg)
+ }
+ return err
+}