* Web feeds: Feeds.
* Web pages: WARCs.
* BitTorrent and huge files: BitTorrent.
+* Downloading service: DownloadService.
* Git::
* Multimedia streaming: Multimedia.
@end menu
@itemize
-@item You need an @ref{nncp-mail} program that extracts the sender
+@item You need an @ref{nncp-exec} program that extracts the sender
address from mail that arrives via NNCP, and that feeds the mail into
the Postfix @command{sendmail} command.
@verbatim
/usr/local/etc/postfix/master.cf:
nncp unix - n n - - pipe
- flags=F user=nncp argv=nncp-mail -quiet $nexthop $recipient
+ flags=F user=nncp argv=nncp-exec -quiet $nexthop sendmail $recipient
@end verbatim
-This runs the @command{nncp-mail} command to place outgoing mail into
+This runs the @command{nncp-exec} command to place outgoing mail into
the NNCP queue after replacing @var{$nexthop} by the the receiving NNCP
node and after replacing @var{$recipient} by the recipients. The
-@command{pipe(8)} delivery agent executes the @command{nncp-mail}
+@command{pipe(8)} delivery agent executes the @command{nncp-exec}
command without assistance from the shell, so there are no problems with
shell meta characters in command-line parameters.
@itemize
-@item You need an @ref{nncp-mail} program that extracts the sender
+@item You need an @ref{nncp-exec} program that extracts the sender
address from mail that arrives via NNCP, and that feeds the mail into
the Postfix @command{sendmail} command.
@verbatim
/usr/local/etc/postfix/master.cf:
nncp unix - n n - - pipe
- flags=F user=nncp argv=nncp-mail -quiet $nexthop $recipient
+ flags=F user=nncp argv=nncp-exec -quiet $nexthop sendmail $recipient
@end verbatim
-This runs the @command{nncp-mail} command to place outgoing mail into
+This runs the @command{nncp-exec} command to place outgoing mail into
the NNCP queue. It substitutes the hostname (@emph{nncp-gateway}, or
whatever you specified) and the recipients before executing the command.
-The @command{nncp-mail} command is executed without assistance from the
+The @command{nncp-exec} command is executed without assistance from the
shell, so there are no problems with shell meta characters.
@item Execute the command @command{postfix reload} to make the changes
--progress
@end verbatim
-Also you can create separate NNCP node those mail receiver will be the
-script downloading website's page and send you its WARC representation
-as a file. You can configure @option{sendmail} option like this:
-
-@verbatim
-% cat /usr/local/etc/nncp.yaml
-[...]
- stargrave.org:
- [...]
- sendmail: ["/bin/sh", "/path/to/warcer.sh"]
-[...]
-@end verbatim
-
-And @file{warcer.sh} contents are:
-
-@verbatim
-#!/bin/sh -ex
-
-user_agent="Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27"
-
-name="$1"
-read cmdline
-
-tmp=$(mktemp -d)
-cd $tmp
-warc_name=$name-$(date '+%Y%M%d%H%m%S')
-wget \
- --page-requisites \
- --convert-links \
- --adjust-extension \
- --restrict-file-names=ascii \
- --span-hosts \
- --random-wait \
- --execute robots=off \
- --user-agent "$user_agent" \
- --reject '*.woff*,*.ttf,*.eot,*.js' \
- --tries 10 \
- --warc-file $warc_name \
- --no-warc-compression \
- --no-warc-keep-log \
- $cmdline || :
-xz -9 "$warc_name".warc
-nncp-file "$warc_name".warc.xz $NNCP_SENDER:
-rm -r $tmp
-@end verbatim
-
-Now you can queueu that node to send you some website's page:
-
-@verbatim
-% echo http://www.nncpgo.org/Postfix.html |
- nncp-mail remote.node nncp-postfix-page
-@end verbatim
-
@node BitTorrent
@section BitTorrent and huge files
and all that downloaded (@file{nncp.txz}, @file{nncp.txz.sig}) files
will be sent to @file{remote.node} when finished.
+@node DownloadService
+@section Downloading service
+
+Previous sections tell about manual downloading and sending results to
+remote node. But one wish to remotely initiate downloading. That can be
+easily solved with @ref{CfgExec, exec} handles.
+
+@verbatim
+exec:
+ warcer: ["/bin/sh", "/path/to/warcer.sh"]
+ wgeter: ["/bin/sh", "/path/to/wgeter.sh"]
+ aria2c: [
+ "/usr/local/bin/aria2c",
+ "--on-download-complete", "send-downloaded.sh",
+ "--on-bt-download-complete", "send-downloaded.sh"
+ ]
+@end verbatim
+
+@file{warcer.sh} contents:
+
+@verbatim
+#!/bin/sh -ex
+
+user_agent="Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27"
+
+name="$1"
+read cmdline
+
+tmp=$(mktemp -d)
+cd $tmp
+warc_name=$name-$(date '+%Y%M%d%H%m%S')
+wget \
+ --page-requisites \
+ --convert-links \
+ --adjust-extension \
+ --restrict-file-names=ascii \
+ --span-hosts \
+ --random-wait \
+ --execute robots=off \
+ --user-agent "$user_agent" \
+ --reject '*.woff*,*.ttf,*.eot,*.js' \
+ --tries 10 \
+ --warc-file $warc_name \
+ --no-warc-compression \
+ --no-warc-keep-log \
+ $cmdline || :
+xz -9 "$warc_name".warc
+nncp-file -nice $NNCP_NICE "$warc_name".warc.xz $NNCP_SENDER:
+rm -r $tmp
+@end verbatim
+
+@file{wgeter.sh} contents:
+
+@verbatim
+#!/bin/sh -ex
+
+name="$1"
+read cmdline
+tmp=$(mktemp)
+wget --output-document=$tmp $cmdline
+xz -9 $tmp
+nncp-file -nice $NNCP_NICE $tmp.xz $NNCP_SENDER:$name.xz
+rm $tmp.xz
+@end verbatim
+
+Now you can queue that node to send you some website's page, file or
+BitTorrents:
+
+@verbatim
+% echo http://www.nncpgo.org/Postfix.html |
+ nncp-exec remote.node warcer postfix-whole-page
+% echo http://www.nncpgo.org/Postfix.html |
+ nncp-exec remote.node wgeter postfix-html-page
+% echo \
+ http://www.nncpgo.org/download/nncp-0.11.tar.xz
+ http://www.nncpgo.org/download/nncp-0.11.tar.xz.sig |
+ nncp-exec remote.node aria2c
+@end verbatim
+
@node Git
@section Integration with Git