+++ /dev/null
-i4cdqgcarfjdjnba6y4jnf498asg8c6p
checksummed. Can be checksummed (with @file{.nock} extension removing)
with @command{nncp-check -nock}.
- Also it can contain @file{.seen} files, that should be cleaned too
- from time to time.
+ Also it can contain @file{seen/} and @file{hdr/} subdirectories,
+ that should be cleaned too from time to time.
All of that cleaning tasks can be done with @ref{nncp-rm} utility.
@example
# PREFIX=/usr/local redo install
@end example
+
+NNCP depends on @code{github.com/fsnotify/fsnotify} library, that is
+solely relies on OS-specific mechanisms. There is possibility that you
+have either broken or unsupported ones. You can still build NNCP with
+@code{-tags nofsnotify} build option, to skip @code{fsnotify} library
+usage at all.
@item autotoss, -doseen, -nofile, -nofreq, -noexec, -notrns
Optionally enable auto tossing: run tosser on node's spool every second
-during the call. You can control either are @file{.seen} files must be
+during the call. You can control either are @file{seen/} files must be
created, or skip any kind of packet processing.
@item when-tx-exists
-Call only if packets for sending exists.
+Call only if packets for sending exists. The check of outbound packets
+existence is performed @strong{every} time we are going to make a call,
+but @emph{when-tx-exists} does not influence @emph{cron}. When
+@emph{cron} configuration decides that it is time to make a call, with
+@emph{when-tx-exists} option it checks packets existence first.
@anchor{CfgNoCK}
@item nock
option anyway.
@anchor{CfgNoHdr}
@item nohdr
-@strong{nohdr} option disables @ref{HdrFile, .hdr} files usage.
+@strong{nohdr} option disables @ref{HdrFile, @file{hdr/}} files usage.
@end table
And optional @ref{MCD, MultiCast Discovery} options:
@ref{nncp-check, @command{nncp-check -nock}}, that will checksum files
and strip the @file{.nock} extension, then repeated call to remote node
will notify about packet's completion. Also it will be notified if
-@ref{nncp-toss, tossing} created @file{.seen} file.
+@ref{nncp-toss, tossing} created @file{seen/} file.
Read @ref{CfgNoCK, more} about @option{-nock} option.
@item @option{-part} option deletes @file{.part}ly downloaded files.
-@item @option{-seen} option deletes @file{.seen} files. But it does not
-apply to @ref{Multicast, multicast areas} @file{.seen} ones!
+@item @option{-seen} option deletes @file{seen/} files. But it does not
+apply to @ref{Multicast, multicast areas} ones!
@item @option{-nock} option deletes non-checksummed (non-verified)
@file{.nock} files.
-@item @option{-hdr} option deletes cached @file{.hdr} files.
+@item @option{-hdr} option deletes cached @file{hdr/} files.
-@item @option{-area} option deletes @file{.seen} files in @file{area/}
-subdirectories.
+@item @option{-area} option deletes seen files in @file{area/} subdirectories.
@end itemize
@option{INT} seconds in an infinite loop. That can be useful when
running this command as a daemon.
-@option{-seen} option creates empty @file{XXX.seen} file after
+@option{-seen} option creates empty @file{seen/XXX} file after
successful tossing of @file{XXX} packet. @ref{nncp-xfer},
@ref{nncp-bundle}, @ref{nncp-daemon} and @ref{nncp-call} commands skip
inbound packets that has been already seen, processed and tossed. This
@item @code{github.com/davecgh/go-xdr} @tab ISC
@item @code{github.com/dustin/go-humanize} @tab MIT
@item @code{github.com/flynn/noise} @tab BSD 3-Clause
+@item @code{github.com/fsnotify/fsnotify} @tab BSD 3-Clause
@item @code{github.com/gorhill/cronexpr} @tab GNU GPLv3
@item @code{github.com/gosuri/uilive} @tab MIT
@item @code{github.com/hjson/hjson-go} @tab MIT
@multitable {XXXXX} {XXXX-XX-XX} {XXXX KiB} {link sign} {xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx}
@headitem Version @tab Date @tab Size @tab Tarball @tab SHA256 checksum
+@item @ref{Release 7_6_0, 7.6.0} @tab 2021-08-08 @tab 1153 KiB
+@tab @url{download/nncp-7.6.0.tar.xz, link} @url{download/nncp-7.6.0.tar.xz.sig, sign}
+@tab @code{00852E80 70415154 197A5555 DDAE636E 6E3940EC DD53D39E A69E5FF1 531BA4C6}
+
@item @ref{Release 7_5_1, 7.5.1} @tab 2021-08-05 @tab 1147 KiB
@tab @url{download/nncp-7.5.1.tar.xz, link} @url{download/nncp-7.5.1.tar.xz.sig, sign}
@tab @code{B093A745 C2EB9F5F E8341ED2 A6F1EE75 701B2646 B5701BAA F4E760D9 32CDD91A}
@end example
@item
-@verbatiminclude .well-known/openpgpkey/hu/i4cdqgcarfjdjnba6y4jnf498asg8c6p.asc
+@verbatiminclude .well-known/openpgpkey/nncpgo.org/hu/i4cdqgcarfjdjnba6y4jnf498asg8c6p.asc
@end itemize
--- /dev/null
+redo-ifchange \
+ ../config \
+ ../VERSION \
+ *.texi \
+ cfg/*.texi \
+ cmd/*.texi \
+ integration/*.texi \
+ pedro.txt \
+ pkt/*.texi \
+ sp.plantuml.txt \
+ usecases.ru/*.texi \
+ usecases/*.texi
+. ../config
+${MAKEINFO:-makeinfo} \
+ -D "VERSION `cat ../VERSION`" \
+ $MAKEINFO_OPTS \
+ --set-customization-variable SECTION_NAME_IN_TITLE=1 \
+ --set-customization-variable TREE_TRANSFORMATIONS=complete_tree_nodes_menus \
+ --set-customization-variable CLOSE_QUOTE_SYMBOL=\" \
+ --set-customization-variable OPEN_QUOTE_SYMBOL=\" \
+ --output $3 index.texi
@itemize
@item check if that message was already seen (sent or received from)
before by the destination node: check existence of
- @file{SPOOL/NODE/area/AREA/MsgHash.seen} file. Skip that node if
+ @file{SPOOL/NODE/area/AREA/MsgHash} file. Skip that node if
it exists
@item if subscriber's node is not the one we received the packet
from, then create outgoing encrypted packet to it, with that
area packet inside
- @item create corresponding @file{MsgHash.seen} file
+ @item create corresponding @file{MsgHash} file
@item "rewind" the outer encrypted file to the beginning and repeat
the whole cycle again, while all of subscribers will "seen" that
area's message.
consumption.
@end itemize
@item check if we have seen that area's message before by looking at
- @file{SPOOL/SELF/area/AREA/MsgHash.seen}. If so, remove the packet,
+ @file{SPOOL/SELF/area/AREA/MsgHash}. If so, remove the packet,
because it is just a ordinary possible duplicate, finish its processing
@item check if we have got corresponding area's private key. If no key
exists, then remove the packet, finish its processing -- we just
@command{nncp-file} creates an encrypted packet with area packet and
encrypted packet inside it, with our own @code{self} node as a recipient
(in the @file{SPOOL/SELF/tx} directory). It also creates the
-@file{SPOOL/SELF/area/AREA/MSGHASH.seen} file.
+@file{SPOOL/SELF/area/AREA/MsgHash} file.
@item
@command{nncp-toss} sees @file{tx/} file and "opens" it, applying the
area message tossing procedure as described above. That will create
outgoing packets in @file{SPOOL/nodeB/tx} and @file{SPOOL/nodeD/tx}
-directories with @file{SPOOL/nodeB/area/AREA/MSGHASH.seen}
-@file{SPOOL/nodeD/area/AREA/MSGHASH.seen} files. Because we already have
-@file{SPOOL/SELF/area/AREA/MSGHASH.seen}, that packet is removed then.
+directories with @file{SPOOL/nodeB/area/AREA/MsgHash}
+@file{SPOOL/nodeD/area/AREA/MsgHash} files. Because we already have
+@file{SPOOL/SELF/area/AREA/MsgHash}, that packet is removed then.
@item
When @code{nodeB} receives the encrypted packet, it sees the area one
@item
@code{nodeD} receives packets from both @code{nodeA} and @code{nodeB}.
Only one of them processed, and other is ignored because corresponding
-@file{MSGHASH.seen} file will exist.
+@file{MsgHash} file will exist.
If @code{nodeD} will receive packet from the @code{nodeB} first, it will
relay it to the @code{nodeA} also, that will silently remove it when
@node Новости
@section Новости
+@node Релиз 7.7.0
+@subsection Релиз 7.7.0
+@itemize
+
+@item
+Экспериментальная поддержка @code{kqueue} и @code{inotify} оповещений об
+изменениях в spool директориях, для сокращения накладных расходов на их
+частое чтение.
+
+@item
+@file{.seen} и @file{.hdr} файлы находятся в @file{seen/} и @file{hdr/}
+поддиректориях теперь, дабы ускорить сканирование spool областей.
+Необходима миграция текущих файлов:
+
+@example
+$ find $NNCPSPOOL -type f -name "*.hdr" -exec rm @{@} +
+
+$ find $NNCPSPOOL -type d -name rx | while read rx ; do
+ cd $rx
+ mkdir -p seen
+ find . -type f -name "*.seen" | while read fn ; do
+ mv $fn seen/$@{fn%.seen@}
+ done
+done
+
+$ find $NNCPSPOOL -type d -name area | while read area ; do
+ find $area -type f -name "*.seen" | while read fn ; do
+ mv $fn $@{fn%.seen@}
+ done
+done
+@end example
+
+@end itemize
+
@node Релиз 7.6.0
@subsection Релиз 7.6.0
@itemize
See also this page @ref{Новости, on russian}.
+@node Release 7_7_0
+@section Release 7.7.0
+@itemize
+
+@item
+Experimental @code{kqueue} and @code{inotify} based notifications
+support about spool directory changes, for reducing their often reading
+overhead.
+
+@item
+@file{.seen} and @file{.hdr} files moved to @file{seen/} and @file{hdr/}
+subdirectories, for faster scanning of spool directories.
+Current files migration required:
+
+@example
+$ find $NNCPSPOOL -type f -name "*.hdr" -exec rm @{@} +
+
+$ find $NNCPSPOOL -type d -name rx | while read rx ; do
+ cd $rx
+ mkdir -p seen
+ find . -type f -name "*.seen" | while read fn ; do
+ mv $fn seen/$@{fn%.seen@}
+ done
+done
+
+$ find $NNCPSPOOL -type d -name area | while read area ; do
+ find $area -type f -name "*.seen" | while read fn ; do
+ mv $fn $@{fn%.seen@}
+ done
+done
+@end example
+
+@end itemize
+
@node Release 7_6_0
@section Release 7.6.0
@itemize
+redo-ifchange makeinfo.rc
rm -fr nncp.html
MAKEINFO_OPTS="$MAKEINFO_OPTS --html --css-include style.css"
MAKEINFO_OPTS="$MAKEINFO_OPTS --set-customization-variable FORMAT_MENU=menu"
MAKEINFO_OPTS="$MAKEINFO_OPTS --set-customization-variable SHOW_TITLE=0"
MAKEINFO_OPTS="$MAKEINFO_OPTS --set-customization-variable DATE_IN_HEADER=1"
-MAKEINFO_OPTS="$MAKEINFO_OPTS" . nncp.info.do
-cp -r .well-known $3
+MAKEINFO_OPTS="$MAKEINFO_OPTS" . makeinfo.rc
-redo-ifchange \
- ../config \
- ../VERSION \
- *.texi \
- cfg/*.texi \
- cmd/*.texi \
- integration/*.texi \
- pedro.txt \
- pkt/*.texi \
- sp.plantuml.txt \
- usecases.ru/*.texi \
- usecases/*.texi
-. ../config
-${MAKEINFO:-makeinfo} \
- -D "VERSION `cat ../VERSION`" \
- $MAKEINFO_OPTS \
- --set-customization-variable SECTION_NAME_IN_TITLE=1 \
- --set-customization-variable TREE_TRANSFORMATIONS=complete_tree_nodes_menus \
- --set-customization-variable CLOSE_QUOTE_SYMBOL=\" \
- --set-customization-variable OPEN_QUOTE_SYMBOL=\" \
- --output $3 index.texi
+redo-ifchange makeinfo.rc
+MAKEINFO_OPTS="--no-split" . makeinfo.rc
Ignore it if it is too nice.
@item If already downloaded file exists, then queue @emph{DONE}
sending.
- @item If @file{.seen} exists, then queue @emph{DONE} sending.
+ @item If @file{seen/XXX} exists, then queue @emph{DONE} sending.
@item If @file{.part} exists, then queue @emph{FREQ} sending with
corresponding offset.
@end itemize
verified against its filename either by @ref{nncp-check}, or by working
online daemons. If it is correct, then its extension is trimmed.
-@item LYT64MWSNDK34CVYOO7TA6ZCJ3NWI2OUDBBMX2A4QWF34FIRY4DQ.seen
+@item seen/LYT64MWSNDK34CVYOO7TA6ZCJ3NWI2OUDBBMX2A4QWF34FIRY4DQ
@ref{nncp-toss} utility can be invoked with @option{-seen} option,
-leading to creation of @file{.seen} files, telling that the file with
+leading to creation of @file{seen/} files, telling that the file with
specified hash has already been processed before. It could be useful
when there are use-cases where multiple ways of packets transfer
available and there is possibility of duplicates reception. You have to
are expired).
@anchor{HdrFile}
-@item LYT64MWSNDK34CVYOO7TA6ZCJ3NWI2OUDBBMX2A4QWF34FIRY4DQ.hdr
+@item hdr/LYT64MWSNDK34CVYOO7TA6ZCJ3NWI2OUDBBMX2A4QWF34FIRY4DQ
If no @ref{CfgNoHdr, nohdr} option is enabled in configuration file,
-then @file{.hdr} files are automatically created for every ordinary
+then @file{hdr/} files are automatically created for every ordinary
(fully received and checksummed) packet. It literally contains just the
header of the corresponding packet. It will be automatically created
even during simple @ref{nncp-stat} call. On filesystems with big
other filesystems probably it won't help at all, or even harm
performance.
-There is a hack: you can create more dense @file{.hdr} allocation by
-removing all @file{.hdr} files and then running @command{nncp-stat},
-that will recreate them. In many cases many @file{.hdr} files will be
+There is a hack: you can create more dense @file{hdr/} allocation by
+removing all @file{hdr/} files and then running @command{nncp-stat},
+that will recreate them. In many cases many @file{hdr/} files will be
allocated more or less linearly on the disk, decreasing listing time
even more.
EOF
perl -i -ne 'print unless /include pedro/' doc/index.texi doc/about.ru.texi
perl -p -i -e 's/^(.verbatiminclude) .*$/$1 PUBKEY.asc/g' doc/integrity.texi
-mv doc/.well-known/openpgpkey/hu/i4cdqgcarfjdjnba6y4jnf498asg8c6p.asc PUBKEY.asc
+mv doc/.well-known/openpgpkey/nncpgo.org/hu/i4cdqgcarfjdjnba6y4jnf498asg8c6p.asc PUBKEY.asc
ln -s ../PUBKEY.asc doc
redo doc
mkinfo --output THANKS $texi
rm -f $texi
-rm -r doc/.well-known doc/nncp.html/.well-known
+rm -r doc/.well-known
########################################################################
if err = os.Remove(job.Path); err != nil {
log.Fatalln("Error during deletion:", err)
} else if ctx.HdrUsage {
- os.Remove(job.Path + nncp.HdrSuffix)
+ os.Remove(nncp.JobPath2Hdr(job.Path))
}
}
ctx.LogI(
if !*dryRun {
os.Remove(dstPath)
if ctx.HdrUsage {
- os.Remove(dstPath + nncp.HdrSuffix)
+ os.Remove(nncp.JobPath2Hdr(dstPath))
}
}
} else {
})
continue
}
- if _, err = os.Stat(dstPath + nncp.SeenSuffix); err == nil || !os.IsNotExist(err) {
+ if _, err = os.Stat(filepath.Join(
+ dstDirPath, nncp.SeenDir, pktName,
+ )); err == nil || !os.IsNotExist(err) {
ctx.LogD("bundle-rx-seen", les, func(les nncp.LEs) string {
return logMsg(les) + ": packet already seen"
})
maxOnlineTimeSec = flag.Uint("maxonlinetime", 0, "Override maxonlinetime option")
autoToss = flag.Bool("autotoss", false, "Toss after call is finished")
- autoTossDoSeen = flag.Bool("autotoss-seen", false, "Create .seen files during tossing")
+ autoTossDoSeen = flag.Bool("autotoss-seen", false, "Create seen/ files during tossing")
autoTossNoFile = flag.Bool("autotoss-nofile", false, "Do not process \"file\" packets during tossing")
autoTossNoFreq = flag.Bool("autotoss-nofreq", false, "Do not process \"freq\" packets during tossing")
autoTossNoExec = flag.Bool("autotoss-noexec", false, "Do not process \"exec\" packets during tossing")
warranty = flag.Bool("warranty", false, "Print warranty information")
autoToss = flag.Bool("autotoss", false, "Toss after call is finished")
- autoTossDoSeen = flag.Bool("autotoss-seen", false, "Create .seen files during tossing")
+ autoTossDoSeen = flag.Bool("autotoss-seen", false, "Create seen/ files during tossing")
autoTossNoFile = flag.Bool("autotoss-nofile", false, "Do not process \"file\" packets during tossing")
autoTossNoFreq = flag.Bool("autotoss-nofreq", false, "Do not process \"freq\" packets during tossing")
autoTossNoExec = flag.Bool("autotoss-noexec", false, "Do not process \"exec\" packets during tossing")
# umask: "022"
# Omit progress showing by default
# noprogress: true
- # Do not use .hdr files
+ # Do not use hdr/ files
# nohdr: true
# MultiCast Discovery:
warranty = flag.Bool("warranty", false, "Print warranty information")
autoToss = flag.Bool("autotoss", false, "Toss after call is finished")
- autoTossDoSeen = flag.Bool("autotoss-seen", false, "Create .seen files during tossing")
+ autoTossDoSeen = flag.Bool("autotoss-seen", false, "Create seen/ files during tossing")
autoTossNoFile = flag.Bool("autotoss-nofile", false, "Do not process \"file\" packets during tossing")
autoTossNoFreq = flag.Bool("autotoss-nofreq", false, "Do not process \"freq\" packets during tossing")
autoTossNoExec = flag.Bool("autotoss-noexec", false, "Do not process \"exec\" packets during tossing")
cfgPath = flag.String("cfg", nncp.DefaultCfgPath, "Path to configuration file")
doAll = flag.Bool("all", false, "Apply remove rules to all nodes")
doTmp = flag.Bool("tmp", false, "Remove all temporary files")
- doHdr = flag.Bool("hdr", false, "Remove all .hdr files")
+ doHdr = flag.Bool("hdr", false, "Remove all hdr/ files")
doLock = flag.Bool("lock", false, "Remove all lock files")
nodeRaw = flag.String("node", "", "Node to remove files in")
doRx = flag.Bool("rx", false, "Process received packets")
doTx = flag.Bool("tx", false, "Process transfered packets")
doPart = flag.Bool("part", false, "Remove only .part files")
- doSeen = flag.Bool("seen", false, "Remove only .seen files")
+ doSeen = flag.Bool("seen", false, "Remove only seen/ files")
doNoCK = flag.Bool("nock", false, "Remove only .nock files")
- doArea = flag.Bool("area", false, "Remove only area/*.seen files")
+ doArea = flag.Bool("area", false, "Remove only area/* seen files")
older = flag.String("older", "", "XXX{smhd}: only older than XXX number of time units")
dryRun = flag.Bool("dryrun", false, "Do not actually remove files")
pktRaw = flag.String("pkt", "", "Packet to remove")
})
return nil
}
- if (*doSeen && strings.HasSuffix(info.Name(), nncp.SeenSuffix)) ||
- (*doNoCK && strings.HasSuffix(info.Name(), nncp.NoCKSuffix)) ||
- (*doHdr && strings.HasSuffix(info.Name(), nncp.HdrSuffix)) ||
+ if (*doNoCK && strings.HasSuffix(info.Name(), nncp.NoCKSuffix)) ||
(*doPart && strings.HasSuffix(info.Name(), nncp.PartSuffix)) {
ctx.LogI("rm", nncp.LEs{{K: "File", V: path}}, logMsg)
if *dryRun {
return nil
})
}
- if *pktRaw != "" || *doRx || *doSeen || *doNoCK || *doHdr || *doPart {
+ if *pktRaw != "" || *doRx || *doNoCK || *doPart {
if err = remove(nncp.TRx); err != nil {
log.Fatalln("Can not remove:", err)
}
log.Fatalln("Can not remove:", err)
}
}
+ removeSub := func(p string, everything bool) error {
+ return filepath.Walk(
+ p, func(path string, info os.FileInfo, err error) error {
+ if err != nil {
+ if os.IsNotExist(err) {
+ return nil
+ }
+ return err
+ }
+ if info.IsDir() {
+ return nil
+ }
+ logMsg := func(les nncp.LEs) string {
+ return fmt.Sprintf("File %s: removed", path)
+ }
+ if everything {
+ ctx.LogI("rm", nncp.LEs{{K: "File", V: path}}, logMsg)
+ if *dryRun {
+ return nil
+ }
+ return os.Remove(path)
+ }
+ if now.Sub(info.ModTime()) < oldBoundary {
+ ctx.LogD(
+ "rm-skip", nncp.LEs{{K: "File", V: path}},
+ func(les nncp.LEs) string {
+ return fmt.Sprintf("File %s: too fresh, skipping", path)
+ },
+ )
+ } else if !*dryRun {
+ return os.Remove(path)
+ }
+ return nil
+ },
+ )
+ }
+ if *doRx || *doSeen {
+ if err = removeSub(filepath.Join(
+ ctx.Spool, node.Id.String(), string(nncp.TRx), nncp.SeenDir,
+ ), *doSeen); err != nil {
+ log.Fatalln("Can not remove:", err)
+ }
+ }
+ if *doRx || *doHdr {
+ if err = removeSub(filepath.Join(
+ ctx.Spool, node.Id.String(), string(nncp.TRx), nncp.HdrDir,
+ ), *doHdr); err != nil {
+ log.Fatalln("Can not remove:", err)
+ }
+ }
+ if *doTx || *doHdr {
+ if err = removeSub(filepath.Join(
+ ctx.Spool, node.Id.String(), string(nncp.TTx), nncp.HdrDir,
+ ), *doHdr); err != nil {
+ log.Fatalln("Can not remove:", err)
+ }
+ }
if *doArea {
if err = filepath.Walk(
filepath.Join(ctx.Spool, node.Id.String(), nncp.AreaDir),
})
return nil
}
- if strings.HasSuffix(info.Name(), nncp.SeenSuffix) {
- ctx.LogI(
- "rm",
- nncp.LEs{{K: "File", V: path}},
- func(les nncp.LEs) string {
- return fmt.Sprintf("File %s: removed", path)
- },
- )
- if *dryRun {
- return nil
- }
- return os.Remove(path)
+ ctx.LogI(
+ "rm",
+ nncp.LEs{{K: "File", V: path}},
+ func(les nncp.LEs) string {
+ return fmt.Sprintf("File %s: removed", path)
+ },
+ )
+ if *dryRun {
+ return nil
}
- return nil
+ return os.Remove(path)
}); err != nil {
log.Fatalln("Can not remove:", err)
}
"fmt"
"log"
"os"
+ "path/filepath"
"time"
"go.cypherpunks.ru/nncp/v7"
nodeRaw = flag.String("node", "", "Process only that node")
niceRaw = flag.String("nice", nncp.NicenessFmt(255), "Minimal required niceness")
dryRun = flag.Bool("dryrun", false, "Do not actually write any tossed data")
- doSeen = flag.Bool("seen", false, "Create .seen files")
+ doSeen = flag.Bool("seen", false, "Create seen/ files")
cycle = flag.Uint("cycle", 0, "Repeat tossing after N seconds in infinite loop")
noFile = flag.Bool("nofile", false, "Do not process \"file\" packets")
noFreq = flag.Bool("nofreq", false, "Do not process \"freq\" packets")
ctx.Umask()
-Cycle:
- isBad := false
+ if *cycle == 0 {
+ isBad := false
+ for nodeId, node := range ctx.Neigh {
+ if nodeOnly != nil && nodeId != *nodeOnly.Id {
+ continue
+ }
+ isBad = ctx.Toss(
+ node.Id,
+ nncp.TRx,
+ nice,
+ *dryRun, *doSeen, *noFile, *noFreq, *noExec, *noTrns, *noArea,
+ ) || isBad
+ if nodeId == *ctx.SelfId {
+ isBad = ctx.Toss(
+ node.Id,
+ nncp.TTx,
+ nice,
+ *dryRun, false, true, true, true, true, *noArea,
+ ) || isBad
+ }
+ }
+ if isBad {
+ os.Exit(1)
+ }
+ return
+ }
+
+ nodeIds := make(chan *nncp.NodeId)
for nodeId, node := range ctx.Neigh {
if nodeOnly != nil && nodeId != *nodeOnly.Id {
continue
}
- isBad = ctx.Toss(
- node.Id,
+ dw, err := ctx.NewDirWatcher(
+ filepath.Join(ctx.Spool, node.Id.String(), string(nncp.TRx)),
+ time.Second*time.Duration(*cycle),
+ )
+ if err != nil {
+ log.Fatalln(err)
+ }
+ go func(nodeId *nncp.NodeId) {
+ for range dw.C {
+ nodeIds <- nodeId
+ }
+ }(node.Id)
+ }
+ for nodeId := range nodeIds {
+ ctx.Toss(
+ nodeId,
nncp.TRx,
nice,
*dryRun, *doSeen, *noFile, *noFreq, *noExec, *noTrns, *noArea,
- ) || isBad
- if nodeId == *ctx.SelfId {
- isBad = ctx.Toss(
- node.Id,
+ )
+ if *nodeId == *ctx.SelfId {
+ ctx.Toss(
+ nodeId,
nncp.TTx,
nice,
*dryRun, false, true, true, true, true, *noArea,
- ) || isBad
+ )
}
}
- if *cycle > 0 {
- time.Sleep(time.Duration(*cycle) * time.Second)
- goto Cycle
- }
- if isBad {
- os.Exit(1)
- }
}
})
continue
}
- if _, err = os.Stat(filepath.Join(dstPath, pktName+nncp.SeenSuffix)); err == nil || !os.IsNotExist(err) {
+ if _, err = os.Stat(filepath.Join(
+ dstPath, nncp.SeenDir, pktName,
+ )); err == nil || !os.IsNotExist(err) {
ctx.LogD("xfer-tx-seen", les, func(les nncp.LEs) string {
return logMsg(les) + ": already seen"
})
})
isBad = true
} else if ctx.HdrUsage {
- os.Remove(job.Path + nncp.HdrSuffix)
+ os.Remove(nncp.JobPath2Hdr(job.Path))
}
}
}
return node, nil
}
-func (ctx *Ctx) ensureRxDir(nodeId *NodeId) error {
- dirPath := filepath.Join(ctx.Spool, nodeId.String(), string(TRx))
- logMsg := func(les LEs) string {
- return fmt.Sprintf("Ensuring directory %s existence", dirPath)
+func ensureDir(dirs ...string) error {
+ p := filepath.Join(dirs...)
+ fi, err := os.Stat(p)
+ if err == nil {
+ if fi.IsDir() {
+ return nil
+ }
+ return fmt.Errorf("%s: is not a directory", p)
}
- if err := os.MkdirAll(dirPath, os.FileMode(0777)); err != nil {
- ctx.LogE("dir-ensure-mkdir", LEs{{"Dir", dirPath}}, err, logMsg)
+ if !os.IsNotExist(err) {
return err
}
- fd, err := os.Open(dirPath)
+ return os.MkdirAll(p, os.FileMode(0777))
+}
+
+func (ctx *Ctx) ensureRxDir(nodeId *NodeId) error {
+ dirPath := filepath.Join(ctx.Spool, nodeId.String(), string(TRx))
+ err := ensureDir(dirPath)
if err != nil {
- ctx.LogE("dir-ensure-open", LEs{{"Dir", dirPath}}, err, logMsg)
- return err
+ ctx.LogE("dir-ensure-mkdir", LEs{{"Dir", dirPath}}, err, func(les LEs) string {
+ return fmt.Sprintf("Ensuring directory %s existence", dirPath)
+ })
}
- return fd.Close()
+ return err
}
func CtxFromCmdline(
+//go:build !netbsd
// +build !netbsd
// NNCP -- Node to Node copy, utilities for store-and-forward data exchange
+//go:build netbsd
// +build netbsd
// NNCP -- Node to Node copy, utilities for store-and-forward data exchange
--- /dev/null
+//go:build !nofsnotify
+// +build !nofsnotify
+
+/*
+NNCP -- Node to Node copy, utilities for store-and-forward data exchange
+Copyright (C) 2016-2021 Sergey Matveev <stargrave@stargrave.org>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, version 3 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+package nncp
+
+import (
+ "fmt"
+ "time"
+
+ "github.com/fsnotify/fsnotify"
+)
+
+type DirWatcher struct {
+ w *fsnotify.Watcher
+ C chan struct{}
+ isDead chan struct{}
+}
+
+func (ctx *Ctx) NewDirWatcher(dir string, d time.Duration) (*DirWatcher, error) {
+ w, err := fsnotify.NewWatcher()
+ if err != nil {
+ return nil, err
+ }
+ err = ensureDir(dir)
+ if err != nil {
+ return nil, err
+ }
+ err = w.Add(dir)
+ if err != nil {
+ w.Close()
+ return nil, err
+ }
+ dw := DirWatcher{
+ w: w,
+ C: make(chan struct{}),
+ isDead: make(chan struct{}),
+ }
+ go func() {
+ ticker := time.NewTicker(d)
+ dw.C <- struct{}{}
+ hasEvents := false
+ for {
+ select {
+ case err := <-w.Errors:
+ ctx.LogE("dir-watch", LEs{{"Dir", dir}}, err, func(les LEs) string {
+ return "fsnotify error: " + err.Error()
+ })
+ case e := <-w.Events:
+ ctx.LogD("dir-watch-event", LEs{{"Dir", dir}}, func(les LEs) string {
+ return fmt.Sprintf("fsnotify event: %v", e)
+ })
+ if e.Op&(fsnotify.Create|fsnotify.Rename) > 0 {
+ hasEvents = true
+ }
+ case <-ticker.C:
+ if hasEvents {
+ dw.C <- struct{}{}
+ hasEvents = false
+ }
+ case <-dw.isDead:
+ w.Close()
+ ticker.Stop()
+ close(dw.C)
+ return
+ }
+ }
+ }()
+ return &dw, err
+}
+
+func (dw *DirWatcher) Close() {
+ close(dw.isDead)
+ for range dw.C {
+ }
+}
--- /dev/null
+//go:build nofsnotify
+// +build nofsnotify
+
+/*
+NNCP -- Node to Node copy, utilities for store-and-forward data exchange
+Copyright (C) 2016-2021 Sergey Matveev <stargrave@stargrave.org>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, version 3 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+package nncp
+
+import (
+ "time"
+)
+
+type DirWatcher struct {
+ C chan struct{}
+ ticker *time.Ticker
+}
+
+func (ctx *Ctx) NewDirWatcher(dir string, d time.Duration) (*DirWatcher, error) {
+ dw := DirWatcher{C: make(chan struct{}), ticker: time.NewTicker(d)}
+ go func() {
+ for range dw.ticker.C {
+ dw.C <- struct{}{}
+ }
+ }()
+ return &dw, nil
+}
+
+func (dw *DirWatcher) Close() {
+ dw.ticker.Stop()
+ for range dw.C {
+ }
+}
github.com/davecgh/go-xdr v0.0.0-20161123171359-e6a2ba005892
github.com/dustin/go-humanize v1.0.0
github.com/flynn/noise v1.0.0
+ github.com/fsnotify/fsnotify v1.5.1
github.com/gorhill/cronexpr v0.0.0-20180427100037-88b0669f7d75
github.com/hjson/hjson-go v3.1.0+incompatible
github.com/klauspost/compress v1.13.1
go.cypherpunks.ru/recfile v0.4.3
golang.org/x/crypto v0.0.0-20210711020723-a769d52b0f97
golang.org/x/net v0.0.0-20210614182718-04defd469f4e
- golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c
+ golang.org/x/sys v0.0.0-20210823070655-63515b42dcdf
golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b
lukechampine.com/blake3 v1.1.5
)
github.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk=
github.com/flynn/noise v1.0.0 h1:DlTHqmzmvcEiKj+4RYo/imoswx/4r6iBlCMfVtrMXpQ=
github.com/flynn/noise v1.0.0/go.mod h1:xbMo+0i6+IGbYdJhF31t2eR1BIU0CYc12+BNAKwUTag=
+github.com/fsnotify/fsnotify v1.5.1 h1:mZcQUHVQUQWoPXXtuf9yuEXKudkV2sx1E06UadKWpgI=
+github.com/fsnotify/fsnotify v1.5.1/go.mod h1:T3375wBYaZdLLcVNkcVbzGHY7f1l/uK5T5Ai1i3InKU=
github.com/golang/snappy v0.0.3 h1:fHPg5GQYlCeLIPB9BZqMVR5nR9A+IM5zcgeTdjMYmLA=
github.com/golang/snappy v0.0.3/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/gorhill/cronexpr v0.0.0-20180427100037-88b0669f7d75 h1:f0n1xnMSmBLzVfsMMvriDyA75NB/oBgILX2GcHXIQzY=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c h1:F1jZWGFhYfh0Ci55sIpILtKKK8p3i2/krTr0H1rg74I=
golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20210823070655-63515b42dcdf h1:2ucpDCmfkl8Bd/FsLtiD653Wf96cW37s+iGx93zsu4k=
+golang.org/x/sys v0.0.0-20210823070655-63515b42dcdf/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b h1:9zKuko04nR4gjZ4+DNjHqRlAJqbJETHwiNKDqTfOjfE=
golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
TRx TRxTx = "rx"
TTx TRxTx = "tx"
- HdrSuffix = ".hdr"
+ HdrDir = "hdr"
)
type Job struct {
HshValue *[MTHSize]byte
}
+func JobPath2Hdr(jobPath string) string {
+ return filepath.Join(filepath.Dir(jobPath), HdrDir, filepath.Base(jobPath))
+}
+
func (ctx *Ctx) HdrRead(r io.Reader) (*PktEnc, []byte, error) {
var pktEnc PktEnc
_, err := xdr.Unmarshal(r, &pktEnc)
os.Remove(tmpHdr.Name())
return err
}
- if err = os.Rename(tmpHdr.Name(), tgt+HdrSuffix); err != nil {
+ if err = ensureDir(filepath.Dir(tgt), HdrDir); err != nil {
+ ctx.LogE("hdr-write-ensure-mkdir", nil, err, func(les LEs) string {
+ return "Header writing: ensuring directory"
+ })
+ return err
+ }
+ if err = os.Rename(tmpHdr.Name(), JobPath2Hdr(tgt)); err != nil {
ctx.LogE("hdr-write-rename", nil, err, func(les LEs) string {
return "Header writing: renaming"
})
if nock || part {
fd, err = os.Open(pth)
} else {
- fd, err = os.Open(pth + HdrSuffix)
+ fd, err = os.Open(JobPath2Hdr(pth))
if err != nil && os.IsNotExist(err) {
hdrExists = false
fd, err = os.Open(pth)
"bytes"
"fmt"
"os"
+ "sync"
"time"
"go.cypherpunks.ru/recfile"
const LogFdPrefix = "FD:"
-var LogFd *os.File
+var (
+ LogFd *os.File
+ LogFdLock sync.Mutex
+)
type LE struct {
K string
func (ctx *Ctx) Log(rec string) {
if LogFd != nil {
+ LogFdLock.Lock()
LogFd.WriteString(rec)
+ LogFdLock.Unlock()
return
}
fdLock, err := os.OpenFile(
const Base32Encoded32Len = 52
var (
- Version string = "7.6.0"
+ Version string = "7.7.0"
Base32Codec *base32.Encoding = base32.StdEncoding.WithPadding(base32.NoPadding)
)
"errors"
"fmt"
"io"
+ "log"
"os"
"path/filepath"
"sort"
if !state.listOnly && (state.xxOnly == "" || state.xxOnly == TTx) {
state.wg.Add(1)
go func() {
- ticker := time.NewTicker(time.Second)
+ dw, err := state.Ctx.NewDirWatcher(
+ filepath.Join(state.Ctx.Spool, state.Node.Id.String(), string(TTx)),
+ time.Second,
+ )
+ if err != nil {
+ state.Ctx.LogE("sp-queue-dir-watch", les, err, logMsg)
+ log.Fatalln(err)
+ }
for {
select {
case <-state.isDead:
+ dw.Close()
state.wg.Done()
- ticker.Stop()
return
- case <-ticker.C:
+ case <-dw.C:
for _, payload := range state.Ctx.infosOur(
state.Node.Id,
state.Nice,
}
continue
}
- if _, err = os.Stat(pktPath + SeenSuffix); err == nil {
+ if _, err = os.Stat(filepath.Join(
+ state.Ctx.Spool, state.Node.Id.String(), string(TRx),
+ SeenDir, Base32Codec.EncodeToString(info.Hash[:]),
+ )); err == nil {
state.Ctx.LogI("sp-info-seen", lesp, func(les LEs) string {
return logMsg(les) + ": already seen"
})
return fmt.Sprintf("Packet %s is sent", pktName)
})
if state.Ctx.HdrUsage {
- os.Remove(pth + HdrSuffix)
+ os.Remove(JobPath2Hdr(pth))
}
} else {
state.Ctx.LogE("sp-done", lesp, err, logMsg)
func (ctx *Ctx) NewTmpFile() (*os.File, error) {
jobsPath := filepath.Join(ctx.Spool, "tmp")
- if err := os.MkdirAll(jobsPath, os.FileMode(0777)); err != nil {
+ if err := ensureDir(jobsPath); err != nil {
return nil, err
}
fd, err := TempFile(jobsPath, "")
func (tmp *TmpFileWHash) Commit(dir string) error {
var err error
- if err = os.MkdirAll(dir, os.FileMode(0777)); err != nil {
+ if err = ensureDir(dir); err != nil {
return err
}
if err = tmp.W.Flush(); err != nil {
)
const (
- SeenSuffix = ".seen"
+ SeenDir = "seen"
)
+func jobPath2Seen(jobPath string) string {
+ return filepath.Join(filepath.Dir(jobPath), SeenDir, filepath.Base(jobPath))
+}
+
func newNotification(fromTo *FromToJSON, subject string, body []byte) io.Reader {
lines := []string{
"From: " + fromTo.From,
})
if !dryRun && jobPath != "" {
if doSeen {
- if fd, err := os.Create(jobPath + SeenSuffix); err == nil {
+ if err := ensureDir(filepath.Dir(jobPath), SeenDir); err != nil {
+ return err
+ }
+ if fd, err := os.Create(jobPath2Seen(jobPath)); err == nil {
fd.Close()
if err = DirSync(filepath.Dir(jobPath)); err != nil {
ctx.LogE("rx-dirsync", les, err, func(les LEs) string {
})
return err
} else if ctx.HdrUsage {
- os.Remove(jobPath + HdrSuffix)
+ os.Remove(JobPath2Hdr(jobPath))
}
}
if !dryRun {
if jobPath != "" {
if doSeen {
- if fd, err := os.Create(jobPath + SeenSuffix); err == nil {
+ if err := ensureDir(filepath.Dir(jobPath), SeenDir); err != nil {
+ return err
+ }
+ if fd, err := os.Create(jobPath2Seen(jobPath)); err == nil {
fd.Close()
if err = DirSync(filepath.Dir(jobPath)); err != nil {
ctx.LogE("rx-dirsync", les, err, func(les LEs) string {
})
return err
} else if ctx.HdrUsage {
- os.Remove(jobPath + HdrSuffix)
+ os.Remove(JobPath2Hdr(jobPath))
}
}
if len(sendmail) > 0 && ctx.NotifyFile != nil {
if !dryRun {
if jobPath != "" {
if doSeen {
- if fd, err := os.Create(jobPath + SeenSuffix); err == nil {
+ if err := ensureDir(filepath.Dir(jobPath), SeenDir); err != nil {
+ return err
+ }
+ if fd, err := os.Create(jobPath2Seen(jobPath)); err == nil {
fd.Close()
if err = DirSync(filepath.Dir(jobPath)); err != nil {
ctx.LogE("rx-dirsync", les, err, func(les LEs) string {
})
return err
} else if ctx.HdrUsage {
- os.Remove(jobPath + HdrSuffix)
+ os.Remove(JobPath2Hdr(jobPath))
}
}
if len(sendmail) > 0 && ctx.NotifyFreq != nil {
})
if !dryRun && jobPath != "" {
if doSeen {
- if fd, err := os.Create(jobPath + SeenSuffix); err == nil {
+ if err := ensureDir(filepath.Dir(jobPath), SeenDir); err != nil {
+ return err
+ }
+ if fd, err := os.Create(jobPath2Seen(jobPath)); err == nil {
fd.Close()
if err = DirSync(filepath.Dir(jobPath)); err != nil {
ctx.LogE("rx-dirsync", les, err, func(les LEs) string {
})
return err
} else if ctx.HdrUsage {
- os.Remove(jobPath + HdrSuffix)
+ os.Remove(JobPath2Hdr(jobPath))
}
}
seenDir := filepath.Join(
ctx.Spool, nodeId.String(), AreaDir, area.Id.String(),
)
- seenPath := filepath.Join(seenDir, msgHash+SeenSuffix)
+ seenPath := filepath.Join(seenDir, msgHash)
logMsgNode := func(les LEs) string {
return fmt.Sprintf(
"%s: echoing to: %s", logMsg(les), node.Name,
seenDir := filepath.Join(
ctx.Spool, nodeId.String(), AreaDir, area.Id.String(),
)
- seenPath := filepath.Join(seenDir, msgHash+SeenSuffix)
+ seenPath := filepath.Join(seenDir, msgHash)
logMsgNode := func(les LEs) string {
return fmt.Sprintf("%s: echo to: %s", logMsg(les), node.Name)
}
seenDir := filepath.Join(
ctx.Spool, ctx.SelfId.String(), AreaDir, area.Id.String(),
)
- seenPath := filepath.Join(seenDir, msgHash+SeenSuffix)
+ seenPath := filepath.Join(seenDir, msgHash)
if _, err := os.Stat(seenPath); err == nil {
ctx.LogD("rx-area-seen", les, func(les LEs) string {
return logMsg(les) + ": already seen"
})
return err
} else if ctx.HdrUsage {
- os.Remove(jobPath + HdrSuffix)
+ os.Remove(JobPath2Hdr(jobPath))
}
}
return nil
})
return err
} else if ctx.HdrUsage {
- os.Remove(jobPath + HdrSuffix)
+ os.Remove(JobPath2Hdr(jobPath))
}
}
nice uint8,
doSeen, noFile, noFreq, noExec, noTrns, noArea bool,
) (chan struct{}, chan bool) {
+ dw, err := ctx.NewDirWatcher(
+ filepath.Join(ctx.Spool, nodeId.String(), string(TRx)),
+ time.Second,
+ )
+ if err != nil {
+ log.Fatalln(err)
+ }
finish := make(chan struct{})
badCode := make(chan bool)
go func() {
for {
select {
case <-finish:
+ dw.Close()
badCode <- bad
- break
- default:
+ return
+ case <-dw.C:
+ bad = !ctx.Toss(
+ nodeId, TRx, nice, false,
+ doSeen, noFile, noFreq, noExec, noTrns, noArea) || bad
}
- time.Sleep(time.Second)
- bad = !ctx.Toss(
- nodeId, TRx, nice, false,
- doSeen, noFile, noFreq, noExec, noTrns, noArea) || bad
}
}()
return finish, badCode
seenDir := filepath.Join(
ctx.Spool, ctx.SelfId.String(), AreaDir, areaId.String(),
)
- seenPath := filepath.Join(seenDir, msgHash+SeenSuffix)
+ seenPath := filepath.Join(seenDir, msgHash)
les := LEs{
{"Node", node.Id},
{"Nice", int(nice)},
msgHash,
)
}
- if err = os.MkdirAll(seenDir, os.FileMode(0777)); err != nil {
+ if err = ensureDir(seenDir); err != nil {
ctx.LogE("tx-mkdir", les, err, logMsg)
return lastNode, err
}