Racket’s approach to macros is the latest point in an evolution that started in 1963 with Lisp’s macros. Building on those ideas, Racket’s macros have evolved so far that, to a modern macro programmer, macros are more helpfully understood as extending and manipulating the compiler’s front end than as a mechanism for textual substitution or syntactic abstraction.

Having a malleable compiler front end naturally enables succinct implementations of many domain-specific and embedded languages. A look at the Racket ecosystem reveals a wealth of examples. Scribble, a language for writing texts uses a LaTeX-inspired syntax and has been used to write thousands of pages of documentation, dozens of research papers, and at least two books. Redex, a language for writing and testing operational semantics, has helped numerous researchers debug their semantics and explore their ideas. Racket’s sister-language, Typed Racket, boasts numerous type-level innovations and full-fledged interoperability with untyped code. Beside these large efforts, Racket’s macros also have enabled extensions on the medium scale as well, being the basis for its pattern matcher, class system, contract system, family of for loops, and more. On the small scale, project-specific macros are common in Racket codebases, as Racket programmers can lift the language of discourse from general programming-language constructs to project-specific concerns, aiding program comprehension and shrinking codebase size.

In this talk, I’ll discuss the essential aspects of Racket’s macro system design, showing how they enable language-oriented programming and provide an intellectual foundation for understanding modern macros. These aspects all center on the idea of automatically managing scope and taking advantage of its automatic management.

Going beyond implementing languages, the data structures supporting automatic scope management have proven central to DrRacket (the Racket IDE), specifically its rename refactoring and its ability to navigate code-bases via uses and definitions of identifiers. Recently, Racketeers have begun to explore how more aspects of Racket’s macro system can support sophisticated IDE tooling for programming languages in the Racket ecosystem. I will try to paint a picture of where we hope to go with that work as well.