Implement a process watcher to get exact artifact dependencies

today, finding "ancilliary" dependencies (e.g., headers included by c++ files) is done by scanning the sources in a pre-pass. this is slow.

a helpful modification to that is having the regular c preprocessor generate dependencies as a side effect of its operation (-M flags). however, this still suffers from a fundamental problem: we don't know about the dependencies until after compilation, which makes this method unsuitable for finding dependencies on generated files (the compile will simply break if the file was not generated yet). some build tools solve this problem by allowing to specify such dependencies manually, but this is cumbersome and error prone.

so what could we do differently?
the idea is to simply run the transformer ... under supervision.
when it checks for the presence of a file (unix stat() call), we check whether our build graph contains a file at that place. if not, or the artifact already exists and is up-to-date, we just let the syscall return as normal. but if the location matches an artifact which still needs building, we suspend the transformer, execute the steps necessary to create/update the artifact, and then resume the transformer.

this means that we get a perfect picture of the not explicitly listed dependencies without the transformer needing to cooperate in any way, and even without knowing anything about the transformed artifact's format/language to start with.
what's more, we can get to know all output artifacts of the transformer which are not really necessary, like log files, temp files (especially if it crashes), etc., which means we can make a perfect cleanup as well.

challenges:

• this obviously needs some kind of "virtual machine". the easiest approach seems to be using the dynamic linker (which obviously implies that we can support only dynamically linked transformers, but this doesn't appear to be a relevant limitation):
  • on Unix: http://en.wikipedia.org/wiki/LD_PRELOAD
  • on Windows: http://en.wikipedia.org/wiki/DLL_injection
• the suspended transformers still occupy memory, which can be a problem if many have to be suspended simultaneously (typical reason: many sources including the same generated file. less common: updating a dependency causes a suspension in turn (recursion))
• compilation caching and distribution require direct support in the build tool