From d708b1fb5ba9fef9ba5c6add645a0c74a2c2b27b Mon Sep 17 00:00:00 2001 From: Sergey Matveev Date: Fri, 24 Jun 2022 16:08:07 +0300 Subject: [PATCH] Mention crawl website crawler --- doc/integration/warc.texi | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/doc/integration/warc.texi b/doc/integration/warc.texi index 4c58729..f449ed5 100644 --- a/doc/integration/warc.texi +++ b/doc/integration/warc.texi @@ -64,6 +64,10 @@ $ wget [--page-requisites] [--recursive] \ [@dots{}] http://www.example.com/ @end example +@pindex crawl +Or even more simpler @url{https://git.jordan.im/crawl/tree/README.md, crawl} +utility written on Go too. + @pindex tofuproxy That command will create @file{www.example.com-XXX.warc} web archive. It could produce specialized segmented -- 2.44.0