From: Sergey Matveev Date: Fri, 24 Jun 2022 13:08:07 +0000 (+0300) Subject: Mention crawl website crawler X-Git-Tag: v8.8.0^2~9 X-Git-Url: http://www.git.cypherpunks.ru/?p=nncp.git;a=commitdiff_plain;h=d708b1fb5ba9fef9ba5c6add645a0c74a2c2b27b Mention crawl website crawler --- diff --git a/doc/integration/warc.texi b/doc/integration/warc.texi index 4c58729..f449ed5 100644 --- a/doc/integration/warc.texi +++ b/doc/integration/warc.texi @@ -64,6 +64,10 @@ $ wget [--page-requisites] [--recursive] \ [@dots{}] http://www.example.com/ @end example +@pindex crawl +Or even more simpler @url{https://git.jordan.im/crawl/tree/README.md, crawl} +utility written on Go too. + @pindex tofuproxy That command will create @file{www.example.com-XXX.warc} web archive. It could produce specialized segmented