# Using and Improving Your Old Makefiles

Makepp is designed to work without modification with existing makefiles. Its syntax is almost a strict superset of make's syntax, and although it uses different algorithms, almost all build rules defined by makefiles work without modification.

My suggestion for converting makefiles is to do the following:

1. Try your makefiles as they are. They will probably work without modification if you haven't been using any of the ugly hacks that people sometimes do to get around limitations of the traditional make. And you'll get automatic scanning for include files for free (provided your compiler command isn't too bizarre). (Note that makepp will insist on recompiling everything the first time you run it, because it doesn't know whether the files that are already there were made correctly.) Of course, you may want to make minor modifications like using cryptographic signature checking to avoid recompiling when a file has changed but the contents are the same except for whitespace and comments.
2. If makepp aborts and tells you to remove the `--noimplicit-load` command line option or the `--traditional-make` option, then just rerun it with those options. These make makepp's behavior emulate the behavior of the traditional unix make more precisely.
3. If makepp does not appear to be finding build rules, especially for files located in subdirectories, you may need to add the `--percent-subdirs` option to the command line.
4. A common problem is some things that are produced by the configure script rebuild themselves unnecessarily. This is because makepp is not willing to make the same assumptions as the standard make about whether a file is up to date (see the section on signature checking for details), except for things that it knows are involved in building makefiles. The proper way to get around this is to add the `: signature target_newer` to all rules that involve a configure procedure.
5. If things still do not work, you should probably look through the rest of this page, and at the list of incompatibilities with standard make. Even if your makefiles do work, there may be better ways to do some things; see below for some examples.
6. There are often a lot of things in legacy makefiles that don't cause a problem for makepp but aren't useful any more. You might want to discard anything in your makefile that attempts to determine include files automatically. Also discard any manual lists of include files. Such things are usually unnecessary since makepp can scan for include files automatically. This will probably get rid of a good part of the complexity of your makefile. You can also get rid of most invocations of recursive make, since makepp can usually figure out on its own which makefiles it needs to load. Again, this will probably reduce the complexity of your makefile substantially.

Makepp usually gives many warnings when using makefiles designed for the traditional unix make. This is because there are many unsafe practices in makefiles which I would like to discourage. The most common is not marking phony targets as such. Most makefiles (including makefiles produced by automake) don't bother to do this properly, even though GNU make does provide a mechanism (the dummy `.PHONY` target). You can disable most of these warning messages by adding `--nowarn` to the command line.

Here are the compilation options that I needed to compile some popular packages with makepp:

Package | Options | Notes
--- | --- | ---
KDE 2.0 (except kdemultimedia) | `RM=rm` | Earlier versions of KDE required no options. makepp does not predefine any symbols, which is why `RM` must be defined on the command line. KDE makefiles are controlled by automake, so many packages are likely to work the same way. (kdemultimedia doesn't compile with makepp because of some very sloppy constructs in one makefile, and also incompatibilities in the use of wildcards.)
perl | `--noimplicit-load --traditional-recursive-make` | Makefiles produced by MakeMaker cause many warnings because they often provide two rules to build the same target, something which makepp does not allow.
Qt 2.2.1 | `--noimplicit-load --traditional-recursive-make -m target_newer --percent-subdirs` | Library compiles fine; compilation of `uic` fails because of incompatible treatment of directories in pattern rules on three files.

## Common problems with legacy makefiles

makepp is designed to work with almost any makefile. In practice, it often doesn't work with complicated makefiles because they frequently assume something that makepp isn't willing to assume; makepp has higher standards for build correctness. There are also a couple of deliberate syntactical differences that could cause problems.

### Commands to rebuild the makefile

In my experience, the biggest problem in using makepp on a complicated existing makefile are commands which rebuild the makefile or dependencies of the makefile. In order to avoid unnecessary rebuilds, Makepp uses the same signature checking algorithm as make for updating makefiles (it only rebuilds if the target is older than a dependency, and doesn't worry if the command has changed). For example, this allows you to create the makefile with a configure procedure, and makepp in theory won't try to rebuild it.

However, there are still a number of different problems which can crop up related to rebuilding the makefile. Commands to rebuild the makefile often modify more targets than are explicitly listed, and as a result, the makefile may be rebuilt unnecessarily. The solution to this is to change the makefile to explicitly list as targets all files which are modified by a given action.

Another problem occurs when the makefile (or something that the makefile depends on, e.g., a file output by the configure procedure such as `config.status`) is a dependency of another file. Although makepp initially does not rebuild the makefile, it later tries to rebuild `config.status`. The traditional make would not bother to rebuild it, because it is newer than all of its dependencies. However, makepp rebuilds unless all the dependencies and the build command are exactly the same, which will never be true if it has never built the file before. Consequently, these files are unnecessarily rebuilt, which causes the makefile to be rebuilt as well on the next iteration.

The solution to this problem is to tell makepp to use a different signature checking algorithm for targets which are produced by the configure command.

If rebuilding the makefile is causing problems, you may want to run makepp with the `--noremake-makefiles` option. This won't suppress rebuilds of the makefile which are triggered by some other file depending on the makefile, however.

### Commands which assume make's signature checking algorithm

Since by default, makepp requires that the dependencies and targets and build command be exactly the same as on the last build, it will force a rebuild of anything that was not built under the control of make itself. This can cause a problem for a makefile which is assuming that the user has run some external program, or is testing for it. This is especially a problem with configure procedures which are expected to be run by the user. For example, Qt 2.1's configure procedure creates a file called `.buildopts`, and their makefile has a line that reads:

    all: other-things .buildopts

    .buildopts:
    	@echo "You haven't run configure yet."
    	@exit 1

makepp always tries to remake `.buildopts`, even if it was generated by the configure procedure, because makepp doesn't know for sure that it was properly made. As a result, the build always fails. To fix this, specify the signature checking algorithm like this:

    .buildopts:
    	: signature target_newer
    	@echo "You haven't run configure yet."
    	@exit 1

Note that this will work with the traditional make, too, because the `: signature` line is interpreted as a shell command which does nothing.

You can also specify the signature checking algorithm on the command line using the `-m target_newer` option.

### Rules which use different names for the same target

makepp recognizes any name for a file, though whatever tortuous path involving directories with symbolic, "..", ".", etc. For example, it will treat "./xyz" and "xyz" as identical. This is normally what you want to do, but other versions of make do not do this because they depend on matching the file name rather than matching the actual file. I have seen some makefiles that depend on this difference. There is no simple cure for this; you must rewrite the makefile so it doesn't make this assumption.

### Warning messages

makepp may give lots of annoying warning messages when run with a makefile that was designed for other implementations of make. You can turn these off using the `--nowarn` option. Some of these warning may be helpful, though, so you may want to look through them before turning them all off.

## Deprecated makefile idioms

There are a number of confusing or unsafe practices that I have seen in existing makefiles that are no longer necessary because there are better ways to achieve the same result with makepp. Here are some examples:

### Recursive make

Although makepp supports most recursive invocations of make, use of `$(MAKE)` is not recommended because there is a much better way to do it. Recursive invocations of make are dangerous because it is difficult and complex to specify accurately dependencies that span makefiles. For example, suppose `my_program` depends on `subdir1/libxyz.so`. It is usually the case that `subdir/libxyz.a` is produced by a different makefile than `my_program`. With the traditional make it is quite difficult to tell make that when it needs to build `subdir/libxyz.a` in order to make `my_program`, it should look in the other makefile for the rule. As a result, the usual procedure is to build everything in the subdirectories, and only then to build the targets at the top level. Thus a typical makefile looks something like this:

    SUBDIRS = subdir1 subdir2
    all: all-recursive my_program

    all-recursive:
    	for dir in $(SUBDIRS); do cd $$dir; $(MAKE) all; cd ..; done

    my_program: subdir1/libxyz.so
    	...

Although this is common, it is dangerous, because if someone types `make my_program`, then the libraries won't get rebuilt. There are ways to ensure that the libraries get rebuilt, but they greatly complicate the makefile and are not often used.

With makepp, you don't even need to worry about the problem at all. Simply take out the silly `all-recursive` phony target, and it will work fine. As soon as makepp sees that it needs to build `subdir1/libxyz.so`, it automatically looks in `subdir1` to see if a makefile is available. If there is one, it is automatically loaded, and therefore makepp will know about the rules for making `subdir1/libxyz.so`. So your makefile will just look like this:

    all: my_program

    my_program: subdir1/libxyz.so
    	...

Now isn't that much easier?

(You can also use the `load_makefile` to tell makepp explicitly to load a makefile, if makepp for some reason can't find the right makefile to load.)

Basically, if you're using recursive make, don't. Just discard all the complexity in your makefile with recursive make and use implicit makefile loading or `load_makefile` instead.

If you really don't want to modify your makefiles to avoid using recursive make, makepp will grudgingly accept recursive make invocations, but there are some caveats. See the section on incompatibilities for details.

### Multiple targets for a rule

It's fairly common for a single shell command to modify more than just one file. However, the traditional make does not provide any clean way of expressing this. For example, a yacc command frequently produces two output files at once. It would be natural to write the command this way:

    y.tab.c y.tab.h: parse.y
    	yacc -d parse.y

The problem is that the traditional make interprets this to mean that if it wants to build `y.tab.c`, it should invoke the yacc command, and if it wants to build `y.tab.h`, it should invoke it again. While this doesn't cause a real problem in this case, it can in some others.

So makefile writers have resorted to many devious hacks to get around this. One of them is the following:

    y.tab.c: y.tab.h

    y.tab.h: parse.y
    	yacc -d parse.y

It is not necessary to use such obfuscated rules (though they will work) with makepp. Just use the natural syntax above. Makepp interprets rules with multiple targets differently than the standard make. (An exception to this is if the rule has no dependencies; then the action is executed for each target, in order to conform with very common practice in makefiles.)

Another, even worse solution which is quite common in makefiles is simply not to list all of the targets of a given action. For example, the above could be written as:

    y.tab.c: parse.y
    	yacc -d parse.y

without even mentioning `y.tab.h`. This will cause problems. For example, makepp won't realize that the `yacc` command must be rerun in order to generate `y.tab.h`, so it may compile some files using the old `y.tab.h` before it gets around to running the yacc command. Always list all targets!!!!

Sometimes targets are deliberately not listed because they don't really change that often and cause unnecessary recompilation when they do. In the above exampel, it is true that `y.tab.h` could potentially be affected by changes to `parse.y`, in 99.9% of the cases it will be unchanged. And `y.tab.h` is likely to be included in lots of other files. With makepp, you can avoid the recompilation without the risk of an incorrect build. Simply use the MD5 signature checking for your C sources. This way, if the contents of the file `y.tab.h` hasn't changed (except for possibly comments), it won't trigger recompilation of anything that depends on it, even if the file date has changed.

### Repeated execution of one rule for many targets

This problem is the inverse of the one just described. For example, consider the following command (modified from the perl distribution):

    B.so ByteLoader.so $(other_targets) : miniperl $(other_dependencies)
    	sh ext/util/make_ext dynamic $@

Traditional make executes the command `sh ext/util/make_ext dynamic $@` twice, once each for `B.so` and `ByteLoader.so` (and once for each of the other dependencies). Makepp will in fact do this, because it is smart enough to realize that the shell command will only rebuild one of the targets. But generally speaking, for clarity and consistency, you should specify multiple targets for a rule only if a single invocation of the shell commands builds all the targets. Makepp has a different idiom for using the same rule for multiple targets:

    $(foreach) : miniperl $(other_dependencies)
    	: foreach B.so ByteLoader.so $(other_targets)
    	sh ext/util/make_ext dynamic $@

This idiom makes it clearer that the same rule is being applied to multiple targets.

### Depending on the makefile

The standard make does not remember what the commands were to build a target. Thus if you change the build command, it may be that the target needs to be rebuilt even if it's not out of date with respect to its dependencies. Sometimes people attempt to get around this by specifying the makefile itself as a dependency:

    CFLAGS = -g

    %.o: %.c Makefile
    	$(CC) $(CFLAGS) -c $< -o $@

This means that if you go and edit the makefile and change the value of `CFLAGS`, make will recompile every single C source file. However, this isn't a very effective way of doing it, because if at the shell prompt you simply type:

    make CFLAGS=-O

then the makefile will not have changed, but the build command still needs to be reexecuted. Furthermore, if you make some irrelevant change in the makefile, make will still recompile every single source file. Makepp has a much better solution to this problem: it remembers the whole build command. Thus it will notice that compilation options are different, no matter whether you change them by editing the makefile or by specifying variables on the command line. It also will only recompile files where the command has changed. There should be no reason to make a file depend on the makefile any more.

### Using the environment

Some makefiles use the environment to communicate things to commands. This is a very bad idea, because makepp does not record what the environment was when it executes the build command, so if you change the environment, it will not know that the build command needs to be reexecuted. (Neither will the traditional make, but it has much lower standards for correct builds.)

For example,

    TMAKEPATH = /somewhere/over/the/rainbow

    export TMAKEPATH

    Makefile: Makefile.pro
    	perl $(TMAKE) $< > $@

This command runs the tmake utility (from Troll Tech, the makers of Qt) to rebuild the makefile. There's nothing wrong with this in principle. But tmake gets an important input from the environment variable `TMAKEPATH`. If you change the line containing `TMAKEPATH`, makepp will not know to rerun the tmake command.

However, if you write the rule this way:

    TMAKEPATH = /somewhere/over/the/rainbow

    Makefile: Makefile.pro
    	TMAKEPATH=$(TMAKEPATH) perl $(TMAKE) $< > $@

i.e., you set the environment variable in the action, then if `TMAKEPATH` changes, makepp will realize that the build command needs to be redone.

Table of contents | Next (incompatibilities) | Previous (extending makepp)
Last modified: Fri Oct 27 13:07:16 PDT 2000