X-Git-Url: http://www.git.cypherpunks.ru/?p=goredo.git;a=blobdiff_plain;f=README;h=a35eab3de85e5246da7c737de1718af811ba3109;hp=db2222ededb298583937ed792d695578debe370a;hb=14398260feaf14dac68b9bdb1c810ccba7d1e768;hpb=f15fe27971bcf37a06bb9542ea1907f11444515a diff --git a/README b/README index db2222e..a35eab3 100644 --- a/README +++ b/README @@ -8,65 +8,87 @@ build system proposal. Originally it was just a rewrite of redo-c apenwarr/redo (https://redo.readthedocs.io/en/latest/) were also implemented. Why yet another implementation? It is feature full and has better performance comparing to shell and Python implementation. + +It passes tests from dieweltistgarnichtso.net's redo-sh.tests and +implementation-neutral from apenwarr/redo. + goredo is free software: see the file COPYING for copying conditions. +Home page: http://www.goredo.cypherpunks.ru/ INSTALL *goredo-install* -Either: > +Hopefully it should work on all POSIX systems. + > $ go get go.cypherpunks.ru/goredo $ goredo -symlinks $ export PATH=`pwd`:$PATH -or: > + +If you have problems with *.golang.org's inability to verify +authenticity of go.cypherpunks.ru TLS connection, then you can disable +their usage by setting GOPRIVATE=go.cypherpunks.ru. If you still have +problems with the authenticity on your side, then build it manually: > + $ git clone git://git.cypherpunks.ru/goredo.git $ cd goredo + $ git tag -v v0.8.0 + $ git clone git://git.cypherpunks.ru/gorecfile.git + $ ( cd gorecfile ; git tag -v v0.4.0 ) + $ echo "replace go.cypherpunks.ru/recfile => `pwd`/gorecfile" >> go.mod + $ git clone git://git.cypherpunks.ru/gotai64n.git + $ ( cd gotai64n ; git tag -v v0.2.0 ) + $ echo "replace go.cypherpunks.ru/tai64n => `pwd`/gotai64n" >> go.mod $ go build $ ./goredo -symlinks $ export PATH=`pwd`:$PATH -< + NOTES *goredo-notes* -* "all" target is used by default +* "all" target is default * stdout is always captured, but no target is created if it was empty * empty targets are considered always out of date -* .do's arguments are relative paths +* .do's $3 is relative path to the file in target directory * .do search goes up to / by default, but can be limited with either - REDO_TOP_DIR environment variable, or by having .redo/top file in it -* executable .do is run as is -* shebangless .do is run with /bin/sh -e[x] + $REDO_TOP_DIR environment variable, or by having .redo/top file in it +* target's completion messages are written after they finish +* executable .do is run as is, non-executable is run with /bin/sh -e[x] +* tracing (-x) can be obviously done only for non-executable .do +* parallizable build is done only during redo-ifchange for human + convenience: you can globally enable REDO_JOBS, but still do for + example: redo htmls infos index upload FEATURES *goredo-features* -* explicit check that stdout and $3 are not written simultaneously -* explicit check (similar to apenwarr/redo's one) that generated target - was not modified "externally" outside the redo, preventing its - overwriting, but continuing the build +* explicit useful and convenient checks from apenwarr/redo: + * check that $1 was not touched during .do execution + * check that stdout and $3 are not written simultaneously + * check that generated target was not modified "externally" outside + the redo, preventing its overwriting, but continuing the build * targets, dependency information and their directories are explicitly synced (can be disabled, should work faster) -* file's change is detected by comparing its ctime and, if it differs, - its BLAKE2b hash. Hash checking is done to prevent false positives - (can be disabled, will work faster) +* file's change is detected by comparing its ctime and BLAKE2b hash * files creation is umask-friendly (unlike mkstemp() used in redo-c) +* parallel build with jobs limit, optionally in infinite mode * coloured messages (can be disabled) * verbose debug messages, including out-of-date determination, PIDs, - lock acquirings/releases -* parallel build with jobs limit, optionally in infinite mode -* optional display of each target's execution time -* each target's stderr can be displayed with the PID + lock and jobserver acquirings/releases +* displaying of each target's execution time +* each target's stderr can be prefixed with the PID +* optional statusline with currently running/waiting/done jobs * target's stderr can be stored on the disk with TAI64N timestamp - prefixes for each line (you can use tai64nlocal utility from - daemontools (http://cr.yp.to/daemontools/tai64nlocal.html) to convert - them to local time). Captures can be viewed any time later + prefixes for each line. To convert them to localtime you can use either + tai64nlocal utility from daemontools (http://cr.yp.to/daemontools.html), + or similar one: > + $ go get go.cypherpunks.ru/tai64n/cmd/tai64nlocal COMMANDS *goredo-commands* -* redo-ifchange, redo-ifcreate, redo-always (useful with redo-stamp) -* redo -- same as redo-ifchange, but forcefully and *sequentially* run +* redo-ifchange, redo-ifcreate, redo-always +* redo -- same as redo-ifchange, but forcefully and sequentially run specified targets * redo-log -- display TAI64N timestamped last stderr of the target -* redo-stamp -- record stamp dependency and consider it non out-of-date - if stamp equals to the previous one -* redo-cleanup -- removes either temporary, or log files, or - everything related to goredo +* redo-stamp -- record stamp dependency. Nothing more, dummy +* redo-cleanup -- removes either temporary, log files, or everything + related to goredo * redo-whichdo -- .do search paths for specified target (similar to apenwarr/redo): > $ redo-whichdo x/y/a.b.o @@ -82,12 +104,82 @@ COMMANDS *goredo-commands* default.do ../default.b.o.do ../default.o.do -< + ../default.do +* redo-dot -- dependency DOT graph generator. For example to visualize + your dependencies with GraphViz: > + $ redo target [...] # to assure that **/.redo/*.dep are filled up + $ redo-dot target [...] > whatever.dot + $ dot -Tpng whatever.dot > whatever.png # possibly add -Gsplines=ortho + +FAQ *goredo-faq* + + Hashing and stamping~ + +All targets are checksummed if their ctime differs from the previous +one. apenwarr/redo gives many reasons why every time checksumming is +bad, but in my opinion in practice all of them do not apply. + +* Aggregate targets and willing to be out-of-date ones just must not + produce empty output files. apenwarr/*, redo-c and goredo + implementations consider non existing file as an out-of-date target +* If you really wish to produce an empty target file, just touch $3 + +DJB's proposal with both stdout and $3 gives that ability to control +your desired behaviour. Those who does not capture stdout -- failed. +Those who creates an empty file if no stdout was written -- failed. + +redo is a tool to help people. Literally all targets can be safely +"redo-stamp < $3"-ed, reducing false positive out-of-dates. Of course, +with the correct stdout/$3 working and placing necessary results in $3, +instead of just silently feeding them in redo-stamp. + +redo implementations are already automatically record -ifchange on .do +files and -ifcreate on non-existing .do files. So why they can not +record redo-stamp the same way implicitly? No, Zen of Python does not +applicable there, because -ifchange/-ifcreate contradict it already. + +Modern cryptographic hash algorithms and CPUs are so fast, that even all +read and writes to or from hard drive arrays can be easily checksummed +and transparently compressed, as ZFS with LZ4 and Skein/BLAKE[23] +algorithms demonstrate us. + +goredo includes redo-stamp, that really records the stamp in the .dep +file, but it does not play any role later. It is stayed just for +compatibility. + + Can removed .do lead to permanent errors of its non existence?~ + +Yes, because dependency on it was recorded previously. Is it safe to +assume that .do-less target now is an ordinary source-file? I have no +confidence in such behaviour. So it is user's decision how to deal with +it, probably it was just his inaccuracy mistake. If you really want to +get rid of that dependency knowledge for foo/bar target, then just +remove foo/.redo/bar.dep. + + Does redo-always always rebuilds target?~ + +goredo, together with apenwarr/redo, rebuilds target once per run. +Always rebuilds during every redo invocation, but only once during it +building. http://news.dieweltistgarnichtso.net/bin/redo-sh.html#why-built-twice +has other opinion, that is why its redo-sh.tests/always_rebuild_1 will +fail. Rebuilding of always-ed targets even during the same build process +ruins any redo's usability in practice. + +For example if my .h file contains source code's version number, that is +git-describe's output and all my other files depends on that header, +then any redo-ifchange of .o will lead to git-describe execution, that +is rather heavy. Of course, because of either hashing or possible +redo-stamp-ing its dependants won't be rebuilt further, but build time +will be already ruined. If you need to rebuild TeX documents (case +mentioned in redo-sh's FAQ) until all references and numbers are ready, +then you must naturally expectedly explicitly use while cycle in your +.do, as apenwarr/redo already suggests. + STATE *goredo-state* Dependency and build state is kept inside .redo subdirectory in each -directory touched related the build. Each corresponding target has its -own, recreated with every rebuild, .dep file. It is recfile +directory related the build. Each corresponding target has its own, +recreated with every rebuild, .dep file. It is recfile (https://www.gnu.org/software/recutils/), that could have various dependency information (dep.rec with the schema included): > @@ -105,7 +197,11 @@ dependency information (dep.rec with the schema included): > Type: stamp Hash: 5bbdf635932cb16b9127e69b6f3872577efed338f0a4ab6f2c7ca3df6ce50cc9 -< + +USAGE *goredo-usage* + +Run any of the command above with the -help. + LICENCE~ This program is free software: you can redistribute it and/or modify