Just! Stop using Makefile

If you look at any software project, its codebase is rarely just the application itself. There's always a few other things in there: Tests (hopefully), linters, complicated setup scripts, incredibly complicated compile scripts. All of which take a lump of code and turn it into a full project. To make these work, a project will normally have a number of scripts, located potentially anywhere, which helps run these ancillary pieces in a reproducible and reliable way. Documenting the existence of said scripts is one thing, but maybe you want some shortcuts to make running them easier?

Some languages make it simple. Whilst I hate to admit it, Javascript, or more specifically npm actually does this reasonably well. An npm project defines its dependencies (and some extra metadata) in a package.json, but one of the optional keys is scripts, which lets users define some additional commands they can easily run with npm run <name>. No additional tooling is needed, which makes it incredibly accessible. The biggest downside is that each command has to be a single line (but yes, you can chain multiple commands together with &&.

What about everything else? Fortunately for the world, not everything is written in Javascript. When met with this issue, most people reach for Makefile. With a Makefile, you can define a number of short commands and fill them with shell commands you need to run in a single YAML-like file:

With this, you can just run make format-server, and it'll run your commands. Much easier than having blocks of commands in your README or needing to be copied from CI config. If you need more, just add more entries. Anyone with a copy of your project and make installed will have everything they need to get going. But here's the thing - make isn't really meant to be used in this way. Sure, you can use make this way, but you really shouldn't.

Makefile is designed for a very specific thing: making files. Languages like C and C++ don't really have real large-project build pipelines (that are first-party at least), so Makefile exists to define how each file gets compiled, in what order, and then how they're all linked together into a final binary. I've even used make in LaTeX projects, where it's necessary to compile a number of SVGs into images before being pulled into a resulting PDF. Sure, I could achieve that with bash, but make makes pipelines like that incredibly simple!

And of course, there's the secret weapon: Parallelism. If each SVG compilation doesn't depend on any other, why not run them in parallel for a performance boost. make -j specifies the number of tasks to run concurrently. Sure, for half a dozen SVGs running through inkscape, it's not going to make much difference. But if you're a kernel, or moderately large C project, you're going to need to leverage every core you can.

make lets you define "targets" (more on that later), which it can then run either directly by name or in a specific order if the task depends on others. As with everything, if you try and force a tool to do something it's not designed to, you start considering its "features" as "qwerks", because the get in your way.

Let's look at an example. If we want a shortcut to run a test script in the current directory using make, we might define a Makefile like:

If we were to run this, nothing will happen:

make makes files, after all. Because there's a file named the same as the task (test), and the task has no dependencies, make assumes it has nothing to do. When using make correctly, this is exactly what you want. If you're just running commands, it's unhelpful, annoying, and not obvious what's going on. This is where .PHONY comes in. By marking a task as "phony", make will ignore whether the file exists and run it every time. If we wanted our Makefile to work as expected, it'd need to look like this:

And so, if all your targets are marked .PHONY, it's a good sign you're trying to force make to do something it doesn't want to.

People probably reach for make for 2 reasons: It's available everywhere (and probably already installed), and it's all they know. Yes, even Windows users can use the same make commands as everyone else. Just because it's the tool you know, doesn't mean it's the best.

#Something else?

What about something else? Is there something else out there that can replace Makefile, without the need to force it to do something it wasn't designed to?

Yes, surprisingly enough. Actually, there's quite a lot of different tools, no matter your preferred tech stack. All of which are arguably better than make at certain things, but do fall short in others. Nothing is ever perfect, sadly.

First off, there's the obvious: A directory full of bash scripts. If you're going to be writing bash anyway for your commands, why not actually write bash? If all you're running is a sequence of commands, bash scripts are easy, versatile, and can be linted with tools like shellcheck. As a language, sure Bash does have some qwerks (I'm looking at you, esac), but even if you need conditionals, arguments, or other bits of control flow, Bash has you covered. You can do a lot with bash, even write Docker (well, bocker). But whilst you can call your bash scripts from others, it becomes easy to end up with a circular dependency, not to mention without a clear naming convention, the directory is prone to scope creep.

Of course, where would we be in any technical discussion without our good old friend "naming conventions". A directory of bash scripts is great, but what do you call them? Where should they live, scripts, helpers or somewhere else? That's where, of all people, GitHub came to the rescue, all the way back in 2015. GitHub defined "scripts to rule them all" - a standard for what scripts you need in a project and what to name them, so staff had a single interface for working with projects, no matter the technical implementation. It's great there's a standard, but I'm not sure how well it's caught on, nor even if it's still being used by GitHub.

Sure, you can do a lot with Bash, but what if you want something with tighter integration with your framework or language? If you need a slightly deeper integration with the project you're working in, you can always use a task runner written in the language you're working in. There's plenty of options out there (not that this is remotely a complete list):

• Python has Fabric, Invoke, and even Pike (an experimental tool I wrote)
• Ruby has Rake
• Go has Mage
• Elixir has Mix
• Javascript has Just (from Microsoft), Jake (no relation), and of course npm

If you want deep language integration, you're probably not using make anyway. But what if that's not what you need? What if you're happy with the solution make provides, you just want something without the qwerks?

#Enter just

just (not the one I just mentioned) aims to be what people often use make for: saving and running project-specific commands. Because it does that, rather than trying to "make" files, there isn't the concept of .PHONY, let alone the need for it. You define the commands you want to run as "recipes", and just do_it.

just is written in Rust, which means not only does it have no bugs (\s), but it's fast, lightweight, distributed as a single binary, and available on practically every platform and distribution (and in most package managers already). That means there's no polluting your development machine with a bunch of unnecessary extra dependencies, it's just a ~3MB binary blob you can stick on your $PATH and not have to worry about again.

just support all the features you'll need from make:

• Passing parameters into tasks
• Defining global variables
• Manage existing environment variables
• Hide the output of commands with @
• Task dependencies

But, it also has a few more. After all, it's designed for command running, so it has some niceties to optimise for that:

• Aliases
• Settings
• Automatically loading .env files
• Run something other than bash (either globally or per-task)
• First-party help text
• Template functions for easier variable manipulation
• Restricting tasks to certain platforms
• Named parameters, including accepting 0 or more arguments
• Run tasks after others, rather than before

All of those and more are documented in the project's README.

#Just an example

To see the power, let's take a look at a Makefile being used incorrectly. Sadly for me, it's a project I'm quite close to: Wagtail - The CMS framework behind my website. Here's its Makefile:

Wagtail's Makefile is fairly simple. It defines a few simple helper tasks for being able to format the project, without caring what tools are being used under the hood.

Sure, it's now trivial to run the various linters and tests for the Wagtail project, but the clue that Wagtail is using make incorrectly is the huge .PHONY line. Their current Makefile absolutely works, and has been just fine since it was added in early 2014, but what if we wanted some of the just benefits? Converting this to Just didn't take long, and it now looks like this:

This doesn't really look much better, but it is, it really is. Let me explain:

First off, there's documentation. As much as we don't like writing it, documentation is incredibly important, especially to people who haven't worked on the project before. Because we don't like writing it, it's a good idea to make it as easy as possible to do. In the past, we had to create a default task which echoes the help text manually. Sure, this gives us a lot of control, but it's more to deal with. With Just, all I need to do is add comments above each task (reminiscent of Rust) and it just does the rest. We still define a default task, but it just calls just --list, which is where all the magic happens.

Tasks get run in exactly the same way. To run the linters, run just lint, which in turn runs lint-server, lint-client and lint-docs. Whilst with make, you could run all of these at once with -j, just doesn't have that option. Sure, in this case it's less useful, but in the days of multi-core CPUs, parallelism is very useful. Instead we'll need to hope the underlying application can run itself over multiple cores.

Now, something the previous Makefile didn't do, but totally could have: Arguments. make absolutely has variables, both inline and reading from the environment, but just makes it far simpler to use if you want to pass more complex arguments. Take the test recipe I defined above. just test runs all the tests, much like make test used to. However, if I wanted to pass --keepdb in, so I don't need to run database migrations before each run, make makes that complicated. With just on the other hand, with the task above, it's just test --keepdb. No escaping, no magic, no nothing! If I wanted to require arguments be passed to just test, I can change the * to +, and everything works.

Another tiny additional feature is in that first line: set dotenv-load. As the name might imply, this tells just to automatically load a .env file in the current working directory. Most projects you'll work on will have some form of either secret or device specific functionality. For that, it's likely it'll be configured through environment variables, kept in a .env file outside of version control. This allows documentation

And of course, there's no .PHONY in sight. All recipes are just that: recipes!

Wagtail is just a simple example, and this is far from all the improvements we could make. Chances are if you've seen a makefile like it lying around, it might work a lot better as a justfile.

I also recently ported my infrastructure repository from a collection of bash scripts to a justfile. It's not the cleanest justfile, and could probably be improved, but it's more maintainable than it used to be. This website has been using just since day 1.

#Is there still a place for make?

Absolutely. I can't say this enough. This post is not a bashing on make for being pointless. make absolutely has a place, and in those places it's really the best tool for the job. If we look at the Linux Kernel, a giant pile of C, it's got a pretty big need for a Makefile. Which explains why even just its primary Makefile is over 2000 lines! If you have a complex pipeline which involves operating on files, potentially in parallel, and compiling them together into others, perhaps for further option, make is not only a great too, it might be the best tool out there for it.

If however, you're just using make because you want some shortcut comments, and it's the only thing you know about, then I highly recommend taking a look at just. make might look like the right tool when it's all you know, but once you find the correct tool, it's just objectively better (like anything else when compared to PHP). It's like trying to cut a hole with a drill. Sure, with a lot of willpower and stubbornness, you can cut a hole (don't ask me how I know). But as soon as you're presented with a saw, you'll never know how you lived without one.