1 *goredo* redo implementation on pure Go
3 OVERVIEW *goredo-overview*
5 This is pure Go implementation of DJB's redo (http://cr.yp.to/redo.html)
6 build system proposal. Originally it was just a rewrite of redo-c
7 (https://github.com/leahneukirchen/redo-c), but later most features of
8 apenwarr/redo (https://redo.readthedocs.io/en/latest/) were also
9 implemented. Why yet another implementation? It is feature full and has
10 better performance comparing to shell and Python implementation.
11 goredo is free software: see the file COPYING for copying conditions.
13 INSTALL *goredo-install*
16 $ go get go.cypherpunks.ru/goredo
18 $ export PATH=`pwd`:$PATH
20 $ git clone git://git.cypherpunks.ru/goredo.git
24 $ export PATH=`pwd`:$PATH
28 * "all" target is used by default
29 * stdout is always captured, but no target is created if it was empty
30 * empty targets are considered always out of date
31 * .do's arguments are relative paths
32 * .do search goes up to / by default, but can be limited with either
33 REDO_TOP_DIR environment variable, or by having .redo/top file in it
34 * executable .do is run as is
35 * shebangless .do is run with /bin/sh -e[x]
36 * target's completion messages are written after they finish
38 FEATURES *goredo-features*
40 * explicit useful and convenient checks from apenwarr/redo:
41 * check that $1 was not touched during .do execution
42 * check that stdout and $3 are not written simultaneously
43 * check that generated target was not modified "externally" outside
44 the redo, preventing its overwriting, but continuing the build
45 * targets, dependency information and their directories are explicitly
46 synced (can be disabled, should work faster)
47 * file's change is detected by comparing its ctime and BLAKE2b hash
48 * files creation is umask-friendly (unlike mkstemp() used in redo-c)
49 * parallel build with jobs limit, optionally in infinite mode
50 * coloured messages (can be disabled)
51 * verbose debug messages, including out-of-date determination, PIDs,
52 lock and jobserver acquirings/releases
53 * displaying of each target's execution time
54 * each target's stderr can be displayed with the PID
55 * optional statusline with currently running/waiting/done jobs
56 * target's stderr can be stored on the disk with TAI64N timestamp
57 prefixes for each line. To convert them to localtime you can use either
58 tai64nlocal utility from daemontools (http://cr.yp.to/daemontools.html)
59 or make a symlink, to use built-in slower decoder: >
60 $ ln -s goredo tai64nlocal
62 COMMANDS *goredo-commands*
64 * redo-ifchange, redo-ifcreate, redo-always (useful with redo-stamp)
65 * redo -- same as redo-ifchange, but forcefully and *sequentially* run
67 * redo-log -- display TAI64N timestamped last stderr of the target
68 * redo-stamp -- record stamp dependency. Nothing more, just dummy
69 * redo-cleanup -- removes either temporary, log files, or everything
71 * redo-whichdo -- .do search paths for specified target (similar to
73 $ redo-whichdo x/y/a.b.o
86 * redo-dot -- dependency DOT graph generator. For example to visualize
87 your dependencies with GraphViz: >
88 $ redo target [...] # to assure that .redo is filled up
89 $ redo-dot target [...] > whatever.dot
90 $ dot -Tpng whatever.dot > whatever.png # possibly add -Gsplines=ortho
96 All targets are checksummed if their ctime differs from the previous
97 ones. apenwarr/redo gives many reasons why every time checksumming is
98 bad, but in my opinion in practice all of his reasons do not apply.
100 * Aggregate targets and willing to be out-of-date ones just must not
101 produce empty output files. apenwarr/*, redo-c and goredo
102 implementations consider non existing file as an out-of-date target
103 * If you really wish to produce an empty target file, just: touch $3
105 DJB's proposal with both stdout and $3 gives that ability to control
106 your desired behaviour. Those who does not capture stdout -- failed.
107 Those who creates an empty file if no stdout was written -- failed.
109 redo is a tool for helping people. Literally all targets can be safely
110 "redo-stamp < $3"-ed, reducing false positive out-of-dates. Of course,
111 with the correct working with stdout/$3 and placing necessary results in
112 $3, instead of just silently feeding them in redo-stamp.
114 redo implementations are already automatically records -ifchange on .do
115 files and -ifcreate on non-existing .do files. So why they can not
116 record redo-stamp the same way implicitly? No, Zen of Python does not
117 applicable there, because -ifchange/-ifcreate contradict it already.
119 Modern cryptographic hash algorithms and CPUs are so fast, that even all
120 read and writes to or from hard drive arrays can be easily checksummed
121 and transparently compressed, as ZFS with LZ4 and Skein/BLAKE[23]
122 algorithms demonstrate us.
124 goredo includes redo-stamp, that really records the stamp in the .dep
125 file, but it does not play any role later. It is stayed just for
128 Removed .do can lead to permanent errors of its non existence~
130 That is true, because dependency on it was recorded previously. Is it
131 safe to assume that .do-less target now is an ordinary source-file? I
132 have no confidence in such behaviour. So it is user's decision how to
133 deal with it, probably it was just his inaccuracy mistake. If you really
134 want to get rid of that dependency knowledge for foo/bar target, then
135 just remove foo/.redo/bar.dep.
139 Dependency and build state is kept inside .redo subdirectory in each
140 directory touched related the build. Each corresponding target has its
141 own, recreated with every rebuild, .dep file. It is recfile
142 (https://www.gnu.org/software/recutils/), that could have various
143 dependency information (dep.rec with the schema included): >
145 Build: 80143f04-bfff-4673-950c-081d712f573d
152 Ctime: 1605721341.253305000
153 Hash: f4929732f96f11e6d4ebe94536b5edef426d00ed0146853e37a87f4295e18eda
158 Hash: 5bbdf635932cb16b9127e69b6f3872577efed338f0a4ab6f2c7ca3df6ce50cc9
162 This program is free software: you can redistribute it and/or modify
163 it under the terms of the GNU General Public License as published by
164 the Free Software Foundation, version 3 of the License.
166 This program is distributed in the hope that it will be useful,
167 but WITHOUT ANY WARRANTY; without even the implied warranty of
168 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
169 GNU General Public License for more details.