X-Git-Url: http://www.git.cypherpunks.ru/?p=goredo.git;a=blobdiff_plain;f=README;h=f64ff9d9c699e92227b142aec5d42f94b73901c8;hp=d7bae44f77e6f205a5b71d53a7060549810ac968;hb=ab0f45138762d5fdf412926f0582b3a70534b1b6;hpb=6f870fbf4abfc5a7fd7d0c31ab86fce502015c47 diff --git a/README b/README index d7bae44..f64ff9d 100644 --- a/README +++ b/README @@ -1,216 +1,4 @@ -*goredo* redo implementation on pure Go - -OVERVIEW *goredo-overview* - -This is pure Go implementation of DJB's redo (http://cr.yp.to/redo.html) -build system proposal. Originally it was just a rewrite of redo-c -(https://github.com/leahneukirchen/redo-c), but later most features of -apenwarr/redo (https://redo.readthedocs.io/en/latest/) were also -implemented. Why yet another implementation? It is feature full and has -better performance comparing to shell and Python implementation. - -It passes tests from dieweltistgarnichtso.net's redo-sh.tests and -implementation-neutral from apenwarr/redo. - +goredo -- redo implementation on pure Go. +See goredo.info and INSTALL for more information. goredo is free software: see the file COPYING for copying conditions. Home page: http://www.goredo.cypherpunks.ru/ - -INSTALL *goredo-install* - -Hopefully it should work on all POSIX systems. - > - $ go get go.cypherpunks.ru/goredo - $ goredo -symlinks - $ export PATH=`pwd`:$PATH - -If you have problems with *.golang.org's inability to verify -authenticity of go.cypherpunks.ru TLS connection, then you can disable -their usage by setting GOPRIVATE=go.cypherpunks.ru. If you still have -problems with the authenticity on your side, then build it manually: > - - $ git clone git://git.cypherpunks.ru/goredo.git - $ cd goredo - $ git tag -v v0.9.0 - $ git clone git://git.cypherpunks.ru/gorecfile.git - $ ( cd gorecfile ; git tag -v v0.4.0 ) - $ echo "replace go.cypherpunks.ru/recfile => `pwd`/gorecfile" >> go.mod - $ git clone git://git.cypherpunks.ru/gotai64n.git - $ ( cd gotai64n ; git tag -v v0.2.0 ) - $ echo "replace go.cypherpunks.ru/tai64n => `pwd`/gotai64n" >> go.mod - $ go build - $ ./goredo -symlinks - $ export PATH=`pwd`:$PATH - -NOTES *goredo-notes* - -* "all" target is default -* stdout is always captured, but no target is created if it was empty -* empty targets are considered always out of date -* .do's $3 is relative path to the file in target directory -* .do search goes up to / by default, but can be limited with either - $REDO_TOP_DIR environment variable, or by having .redo/top file in it -* target's completion messages are written after they finish -* executable .do is run as is, non-executable is run with /bin/sh -e[x] -* tracing (-x) can be obviously done only for non-executable .do -* parallizable build is done only during redo-ifchange for human - convenience: you can globally enable REDO_JOBS, but still do for - example: redo htmls infos index upload - -FEATURES *goredo-features* - -* explicit useful and convenient checks from apenwarr/redo: - * check that $1 was not touched during .do execution - * check that stdout and $3 are not written simultaneously - * check that generated target was not modified "externally" outside - the redo, preventing its overwriting, but continuing the build -* targets, dependency information and their directories are explicitly - synced (can be disabled, should work faster) -* file's change is detected by comparing its ctime and BLAKE2b hash -* files creation is umask-friendly (unlike mkstemp() used in redo-c) -* parallel build with jobs limit, optionally in infinite mode -* coloured messages (can be disabled) -* verbose debug messages, including out-of-date determination, PIDs, - lock and jobserver acquirings/releases -* displaying of each target's execution time -* each target's stderr can be prefixed with the PID -* optional statusline with currently running/waiting/done jobs -* target's stderr can be stored on the disk with TAI64N timestamp - prefixes for each line. To convert them to localtime you can use either - tai64nlocal utility from daemontools (http://cr.yp.to/daemontools.html), - or similar one: > - $ go get go.cypherpunks.ru/tai64n/cmd/tai64nlocal - -COMMANDS *goredo-commands* - -* redo-ifchange, redo-ifcreate, redo-always -* redo -- same as redo-ifchange, but forcefully and sequentially run - specified targets -* redo-log -- display TAI64N timestamped last stderr of the target -* redo-stamp -- record stamp dependency. Nothing more, dummy -* redo-cleanup -- removes either temporary, log files, or everything - related to goredo -* redo-whichdo -- .do search paths for specified target (similar to - apenwarr/redo): > - $ redo-whichdo x/y/a.b.o - x/y/a.b.o.do - x/y/default.b.o.do - x/y/default.o.do - x/y/default.do - x/default.b.o.do - x/default.o.do - x/default.do - default.b.o.do - default.o.do - default.do - ../default.b.o.do - ../default.o.do - ../default.do -* redo-dot -- dependency DOT graph generator. For example to visualize - your dependencies with GraphViz: > - $ redo target [...] # to assure that **/.redo/*.dep are filled up - $ redo-dot target [...] > whatever.dot - $ dot -Tpng whatever.dot > whatever.png # possibly add -Gsplines=ortho - -FAQ *goredo-faq* - - Hashing and stamping~ - -All targets are checksummed if their ctime differs from the previous -one. apenwarr/redo gives many reasons why every time checksumming is -bad, but in my opinion in practice all of them do not apply. - -* Aggregate targets and willing to be out-of-date ones just must not - produce empty output files. apenwarr/*, redo-c and goredo - implementations consider non existing file as an out-of-date target -* If you really wish to produce an empty target file, just touch $3 - -DJB's proposal with both stdout and $3 gives that ability to control -your desired behaviour. Those who does not capture stdout -- failed. -Those who creates an empty file if no stdout was written -- failed. - -redo is a tool to help people. Literally all targets can be safely -"redo-stamp < $3"-ed, reducing false positive out-of-dates. Of course, -with the correct stdout/$3 working and placing necessary results in $3, -instead of just silently feeding them in redo-stamp. - -redo implementations are already automatically record -ifchange on .do -files and -ifcreate on non-existing .do files. So why they can not -record redo-stamp the same way implicitly? No, Zen of Python does not -applicable there, because -ifchange/-ifcreate contradict it already. - -Modern cryptographic hash algorithms and CPUs are so fast, that even all -read and writes to or from hard drive arrays can be easily checksummed -and transparently compressed, as ZFS with LZ4 and Skein/BLAKE[23] -algorithms demonstrate us. - -goredo includes redo-stamp, that really records the stamp in the .dep -file, but it does not play any role later. It is stayed just for -compatibility. - - Can removed .do lead to permanent errors of its non existence?~ - -Yes, because dependency on it was recorded previously. Is it safe to -assume that .do-less target now is an ordinary source-file? I have no -confidence in such behaviour. So it is user's decision how to deal with -it, probably it was just his inaccuracy mistake. If you really want to -get rid of that dependency knowledge for foo/bar target, then just -remove foo/.redo/bar.dep. - - Does redo-always always rebuilds target?~ - -goredo, together with apenwarr/redo, rebuilds target once per run. -Always rebuilds during every redo invocation, but only once during it -building. http://news.dieweltistgarnichtso.net/bin/redo-sh.html#why-built-twice -has other opinion, that is why its redo-sh.tests/always_rebuild_1 will -fail. Rebuilding of always-ed targets even during the same build process -ruins any redo's usability in practice. - -For example if my .h file contains source code's version number, that is -git-describe's output and all my other files depends on that header, -then any redo-ifchange of .o will lead to git-describe execution, that -is rather heavy. Of course, because of either hashing or possible -redo-stamp-ing its dependants won't be rebuilt further, but build time -will be already ruined. If you need to rebuild TeX documents (case -mentioned in redo-sh's FAQ) until all references and numbers are ready, -then you must naturally expectedly explicitly use while cycle in your -.do, as apenwarr/redo already suggests. - -STATE *goredo-state* - -Dependency and build state is kept inside .redo subdirectory in each -directory related the build. Each corresponding target has its own, -recreated with every rebuild, .dep file. It is recfile -(https://www.gnu.org/software/recutils/), that could have various -dependency information (dep.rec with the schema included): > - - Build: 80143f04-bfff-4673-950c-081d712f573d - - Type: ifcreate - Target: foo.o.do - - Type: ifchange - Target: default.o.do - Ctime: 1605721341.253305000 - Hash: f4929732f96f11e6d4ebe94536b5edef426d00ed0146853e37a87f4295e18eda - - Type: always - - Type: stamp - Hash: 5bbdf635932cb16b9127e69b6f3872577efed338f0a4ab6f2c7ca3df6ce50cc9 - -USAGE *goredo-usage* - -Run any of the command above with the -help. - -LICENCE~ - -This program is free software: you can redistribute it and/or modify -it under the terms of the GNU General Public License as published by -the Free Software Foundation, version 3 of the License. - -This program is distributed in the hope that it will be useful, -but WITHOUT ANY WARRANTY; without even the implied warranty of -MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -GNU General Public License for more details. - - vim: filetype=help