Bundles feature

[nncp.git] / doc / cmds.texi

@node Commands
@unnumbered Commands

Nearly all commands have the following common options:

@table @option
@item -cfg
    Path to configuration file. May be overridden by @env{NNCPCFG}
    environment variable. If file file is an encrypted @ref{EBlob,
    eblob}, then ask for passphrase to decrypt it first.
@item -debug
    Print debug messages. Normally this option should not be used.
@item -minsize
    @anchor{OptMinSize}
    Minimal required resulting packet size, in KiBs. For example if you
    send 2 KiB file and set @option{-minsize 4}, then resulting packet
    will be 4 KiB (containing file itself and some junk).
@item -nice
    Set desired outgoing packet @ref{Niceness, niceness level}.
    1-255 values are allowed.
@item -node
    Process only single specified node.
@item -quiet
    Print only errors, omit simple informational messages. In any case
    those messages are logged, so you can reread them using
    @ref{nncp-log} command.
@item -version
    Print version information.
@item -warranty
    Print warranty information (no warranty).
@end table

@node nncp-bundle
@section nncp-bundle

@verbatim
% nncp-bundle [options] -tx [-delete] NODE [NODE ...] > ...
% nncp-bundle [options] -rx -delete [NODE ...] < ...
% nncp-bundle [options] -rx [-check] [NODE ...] < ...
@end verbatim

With @option{-tx} option, this command creates @ref{Bundles, bundle} of
@ref{Encrypted, encrypted packets} from the spool directory and writes
it to stdout.

With @option{-rx} option, this command takes bundle from stdin and
copies all found packets for our node to the spool directory. Pay
attention that @strong{no} integrity checking is done by default. Modern
tape drives could easily provide too much throughput your CPU won't be
able to verify on the fly. So if you won't @ref{nncp-toss, toss}
received packets at the place, it is advised to run @ref{nncp-check}
utility for packets integrity verification, or use @option{-check}
option to enable on the fly integrity check.

You can specify multiple @option{NODE} arguments, telling for what nodes
you want to create the stream, or take it from. If no nodes are
specified for @option{-rx} mode, then all packets aimed at us will be
processed.

When packets are sent through the stream, they are still kept in the
spool directory, because there is no assurance that they are transferred
to the media (media (CD-ROM, tape drive, raw hard drive) can end). If
you want to forcefully delete them (after they are successfully flushed
to stdout) anyway, use @option{-delete} option.

But you can verify produced stream after, by digesting it by yourself
with @option{-rx} and @option{-delete} options -- in that mode, stream
packets integrity will be checked and they will be deleted from the
spool if everything is good. So it is advisable to recheck your streams:

@verbatim
% nncp-bundle -tx ALICE BOB WHATEVER | cdrecord -tao -
% dd if=/dev/cd0 bs=2048 | nncp-bundle -rx -delete
@end verbatim

@node nncp-call
@section nncp-call

@verbatim
% nncp-call [options] [-onlinedeadline INT] [-maxonlinetime INT] [-rx|-tx]
                      NODE[:ADDR] [FORCEADDR]
@end verbatim

Call (connect to) specified @option{NODE} and run @ref{Sync,
synchronization} protocol with the @ref{nncp-daemon, daemon} on the
remote side. Normally this command could be run any time you wish to
either check for incoming packets, or to send out queued ones.
Synchronization protocol allows resuming and bidirectional packets
transfer.

If @option{-rx} option is specified then only inbound packets
transmission is performed. If @option{-tx} option is specified, then
only outbound transmission is performed. @option{-onlinedeadline}
overrides @ref{CfgOnlineDeadline, @emph{onlinedeadline}}.
@option{-maxonlinetime} overrides @ref{CfgMaxOnlineTime,
@emph{maxonlinetime}}.

@node nncp-caller
@section nncp-caller

@verbatim
% nncp-caller [options] [NODE ...]
@end verbatim

Croned daemon that calls remote nodes from time to time, according to
their @ref{CfgCalls, @emph{calls}} configuration field.

Optional number of @option{NODE}s tells to ignore other ones.
Otherwise all nodes with specified @emph{calls} configuration
field will be called.

@option{-onlinedeadline} overrides @ref{CfgOnlineDeadline,
@emph{onlinedeadline}} configuration option.

Each @option{NODE} can contain several uniquely identified
@option{ADDR}esses in @ref{CfgAddrs, configuration} file. If you do
not specify the exact one, then all will be tried until the first
success. Optionally you can force @option{FORCEADDR} address usage,
instead of addresses taken from configuration file.

Pay attention that this command runs integrity check for each completely
received packet in the background. This can be time consuming.
Connection could be lost during that check and remote node won't be
notified that file is done. But after successful integrity check that
file is renamed from @file{.part} one and when you rerun
@command{nncp-call} again, remote node will receive completion
notification.

@node nncp-cfgenc
@section nncp-cfgenc

@verbatim
% nncp-cfgmin [options] [-s INT] [-t INT] [-p INT] cfg.yaml > cfg.yaml.eblob
% nncp-cfgmin [options] -d cfg.yaml.eblob > cfg.yaml
@end verbatim

This command allows you to encrypt provided @file{cfg.yaml} file with
the passphrase, producing @ref{EBlob, eblob}, to safely keep your
configuration file with private keys. This utility was written for users
who do not want (or can not) to use either @url{https://gnupg.org/,
GnuPG} or similar tools. That @file{eblob} file can be used directly in
@option{-cfg} option of nearly all commands.

@option{-s}, @option{-t}, @option{-p} are used to tune @file{eblob}'s
password strengthening function. Space memory cost (@option{-s}),
specified in number of BLAKE2b-256 blocks (32 bytes), tells how many
memory must be used for hashing -- bigger values are better, but slower.
Time cost (@option{-t}) tells how many rounds/iterations must be
performed -- bigger is better, but slower. Number of parallel jobs
(@option{-p}) tells how many computation processes will be run: this is
the same as running that number of independent hashers and then joining
their result together.

When invoked for encryption, passphrase is entered manually twice. When
invoked for decryption (@option{-d} option), it is asked once and exits
if passphrase can not decrypt @file{eblob}.

@option{-dump} options parses @file{eblob} and prints parameters used
during its creation. For example:
@verbatim
% nncp-cfgenc -dump /usr/local/etc/nncp.yaml.eblob
Strengthening function: Balloon with BLAKE2b-256
Memory space cost: 1048576 bytes
Number of rounds: 16
Number of parallel jobs: 2
Blob size: 2494
@end verbatim

@node nncp-cfgmin
@section nncp-cfgmin

@verbatim
% nncp-cfgmin [options] > stripped.yaml
@end verbatim

Print out stripped configuration version: only path to @ref{Spool,
spool}, path to log file, neighbours public keys are stayed. This is
useful mainly for usage with @ref{nncp-xfer} that has to know only
neighbours, without private keys involving.

@node nncp-cfgnew
@section nncp-cfgnew

@verbatim
% nncp-cfgnew [options] > new.yaml
@end verbatim

Generate new node configuration: private keys, example configuration
file and print it to stdout. You must use this command when you setup
the new node.

Pay attention that private keys generation consumes an entropy from your
operating system.

@node nncp-check
@section nncp-check

@verbatim
% nncp-check [options]
@end verbatim

Perform @ref{Spool, spool} directory integrity check. Read all files
that has Base32-encoded filenames and compare it with recalculated
BLAKE2b hash output of their contents. That supplementary command is
not used often in practice, if ever.

@node nncp-daemon
@section nncp-daemon

@verbatim
% nncp-daemon [options] [-maxconn INT] [-bind ADDR]
@end verbatim

Start listening TCP daemon, wait for incoming connections and run
@ref{Sync, synchronization protocol} with each of them. You can run
@ref{nncp-toss} utility in background to process inbound packets from
time to time.

@option{-maxconn} option specifies how many simultaneous clients daemon
can handle. @option{-bind} option specifies @option{addr:port} it must
bind to and listen.

@node nncp-file
@section nncp-file

@verbatim
% nncp-file [options] [-chunked INT] SRC NODE:[DST]
@end verbatim

Send @file{SRC} file to remote @option{NODE}. @file{DST} specifies
destination file name in remote's @ref{CfgIncoming, incoming}
directory. If this file already exists there, then counter will be
appended to it.

This command queues file in @ref{Spool, spool} directory immediately
(through the temporary file of course) -- so pay attention that sending
2 GiB file will create 2 GiB outbound encrypted packet.

If @file{SRC} equals to @file{-}, then create an encrypted temporary
file and copy everything taken from stdin to it and use for outbound
packet creation. Pay attention that if you want to send 1 GiB of data
taken from stdin, then you have to have 2 GiB of disk space for that
temporary file and resulting encrypted packet. You can control where
temporary file will be stored using @env{TMPDIR} environment variable.
Encryption is performed with
@url{https://www.schneier.com/academic/twofish/, Twofish} algorithm, 256
bit random key, zero IV, in
@url{https://en.wikipedia.org/wiki/Counter_mode#Counter_.28CTR.29, CTR}
mode.

If @option{-chunked} is specified, then source file will be split
@ref{Chunked, on chunks}. @option{INT} is the desired chunk size in
KiBs. This mode is more CPU hungry. Pay attention that chunk is saved in
spool directory immediately and it is not deleted if any error occurs.
@option{-minsize} option is applied per each chunk.

If @ref{CfgNotify, notification} is enabled on the remote side for
file transmissions, then it will sent simple letter after successful
file receiving.

@node nncp-freq
@section nncp-freq

@verbatim
% nncp-freq [options] NODE:SRC [DST]
@end verbatim

Send file request to @option{NODE}, asking it to send its @file{SRC}
file from @ref{CfgFreq, freq} directory to our node under @file{DST}
filename in our @ref{CfgIncoming, incoming} one. If @file{DST} is not
specified, then last element of @file{SRC} will be used.

If @ref{CfgNotify, notification} is enabled on the remote side for
file request, then it will sent simple letter after successful file
queuing.

@node nncp-log
@section nncp-log

@verbatim
% nncp-log [options]
@end verbatim

Parse @ref{Log, log} file and print out its records in human-readable form.

@node nncp-mail
@section nncp-mail

@verbatim
% nncp-mail [options] NODE USER ...
@end verbatim

Send mail, that is read from stdin, to @option{NODE} and specified
@option{USER}s. Mail message will be compressed. After receiving, remote
side will execute specified @ref{CfgSendmail, sendmail} command with
@option{USER}s appended as a command line argument and feed decompressed
mail body to that command's stdin.

@node nncp-pkt
@section nncp-pkt

@verbatim
% nncp-pkt [options] < pkt
% nncp-pkt [options] [-decompress] -dump < pkt > payload
@end verbatim

Low level packet parser. Normally it should not be used, but can help in
debugging.

By default it will print packet's type, for example:
@verbatim
Packet type: encrypted
Niceness: 64
Sender: 2WHBV3TPZHDOZGUJEH563ZEK7M33J4UESRFO4PDKWD5KZNPROABQ
@end verbatim

If you specify @option{-dump} option and provide an @ref{Encrypted,
encrypted} packet, then it will verify and decrypt it to stdout.
Encrypted packets contain @ref{Plain, plain} ones, that also can be fed
to @command{nncp-pkt}:

@verbatim
Packet type: plain
Payload type: transitional
Path: VHMTRWDOXPLK7BR55ICZ5N32ZJUMRKZEMFNGGCEAXV66GG43PEBQ

Packet type: plain
Payload type: mail
Path: stargrave@stargrave.org
@end verbatim

And with the @option{-dump} option it will give you the actual payload
(the whole file, mail message, and so on). @option{-decompress} option
tries to zlib-decompress the data from plain packet (useful for mail
packets).

@node nncp-reass
@section nncp-reass

@verbatim
% nncp-reass [options] [-dryrun] [-keep] [-dump] [-stdout] FILE.nncp.meta
% nncp-reass [options] [-dryrun] [-keep] {-all | -node NODE}
@end verbatim

Reassemble @ref{Chunked, chunked file} after @ref{nncp-toss, tossing}.

When called with @option{FILE} option, this command will reassemble only
it. When called with @option{-node} option, this command will try to
reassemble all @file{.nncp.meta} files found in @option{NODE}'s
@ref{CfgIncoming, incoming} directory. When called with @option{-all}
option, then cycle through all known nodes to do the same.

Reassembling process does the following:

@enumerate
@item Parses @ref{Chunked, @file{.nncp.meta}} file.
@item Checks existence and size of every @file{.nncp.chunkXXX}.
@item Verifies integrity of every chunk.
@item Concatenates all chunks, simultaneously removing them from filesystem.
@end enumerate

That process reads the whole data twice. Be sure to have free disk
space for at least one chunk. Decrypted chunk files as a rule are saved
in pseudo-random order, so removing them during reassembly process will
likely lead to filesystem fragmentation. Reassembly process on
filesystems with deduplication capability should be rather lightweight.

If @option{-dryrun} option is specified, then only existence and
integrity checking are performed.

If @option{-keep} option is specified, then no
@file{.nncp.meta}/@file{.nncp.chunkXXX} files are deleted during
reassembly process.

@option{-stdout} option outputs reassembled file to stdout, instead of
saving to temporary file with renaming after. This could be useful for
reassembling on separate filesystem to lower fragmentation effect,
and/or separate storage device for higher performance.

@option{-dump} option prints meta-file contents in human-friendly form.
It is useful mainly for debugging purposes. For example:
@verbatim
Original filename: testfile
File size: 3.8 MiB (3987795 bytes)
Chunk size: 1.0 MiB (1048576 bytes)
Number of chunks: 4
Checksums:
    0: eac60d819edf40b8ecdacd0b9a5a8c62de2d15eef3c8ca719eafa0be9b894017
    1: 013a07e659f2e353d0e4339c3375c96c7fffaa2fa00875635f440bbc4631052a
    2: f4f883975a663f2252328707a30e71b2678f933b2f3103db8475b03293e4316e
    3: 0e9e229501bf0ca42d4aa07393d19406d40b179f3922a3986ef12b41019b45a3
@end verbatim

@node nncp-rm
@section nncp-rm

@verbatim
% nncp-rm [options] NODE PKT
@end verbatim

Remove specified packet (Base32 name) in @option{NODE}'s queues. This
command is useful when you want to remove the packet that is failing to
be processed.

@node nncp-stat
@section nncp-stat

@verbatim
% nncp-stat [options]
@end verbatim

Print current @ref{Spool, spool} statistics about unsent and unprocessed
packets. For each node and each niceness level there will be printed how
many packets (with the total size) are in inbound (Rx) and outbound (Tx)
queues.

@node nncp-toss
@section nncp-toss

@verbatim
% nncp-toss [options] [-dryrun] [-cycle INT]
@end verbatim

Perform "tossing" operation on all inbound packets. This is the tool
that decrypts all packets and processes all payload packets in them:
copies files, sends mails, sends out file requests and relays transition
packets. It should be run after each online/offline exchange.

@option{-dryrun} option does not perform any writing and sending, just
tells what it will do.

@option{-cycle} option tells not to quit, but to repeat tossing every
@option{INT} seconds in an infinite loop. That can be useful when
running this command as a daemon.

@node nncp-xfer
@section nncp-xfer

@verbatim
% nncp-xfer [options] [-mkdir] [-keep] [-rx|-tx] DIR
@end verbatim

Search for directory in @file{DIR} containing inbound packets for us and
move them to local @ref{Spool, spool} directory. Also search for known
neighbours directories and move locally queued outbound packets to them.
This command is used for offline packets transmission.

If @option{-mkdir} option is specified, then outbound neighbour(s)
directories will be created. This is useful for the first time usage,
when storage device does not have any directories tree.

If @option{-keep} option is specified, then keep copied files, do not
remove them.

@option{-rx} option tells only to move inbound packets addressed to us.
@option{-tx} option tells exactly the opposite: move only outbound packets.

@ref{nncp-cfgmin} could be useful for creating stripped minimalistic
configuration file version without any private keys.

@file{DIR} directory has the following structure:
@file{RECIPIENT/SENDER/PACKET}, where @file{RECIPIENT} is Base32 encoded
destination node, @file{SENDER} is Base32 encoded sender node.