Course overview

Introduction

Understand the basics of an AST and how it relates to real-world code

What is an Abstract Syntax Tree? Visualizing Code Like a Compiler

Looking at an example code snippet and how it relates to the resulting AST

How to View Abstract Syntax Tree Code With AST Explorer

An overview of the tooling available in the web frontend (JavaScript) ecosystem that rely on ASTs

The Best JavaScript AST Tools - ESLint, Babel, Terser, and More

A quick overview of the basic environment setup used throughout this course

Environment setup

Understanding Abstract Syntax Trees (AST)

Exploring tools to convert (or parse) JavaScript into a real AST

How to Generate a JavaScript AST With Babel Plugins

Programmatically traverse an AST and visit arbitrary nodes

Traversing an AST With Babel Traverse

Add Type Safety and Prevent Runtime Errors With AST

Working with Abstract Syntax Trees

A practical example of needing to perform a code audit (static analysis) to understand the current state of a codebase

Practical code audits

Create a custom script to audit a codebase

How to Audit Your Code With AST Programming

Introduce a shared Button component to replace all existing button elements

How to Add a Component and Update an AST

A rough formula to determine when to use AST-based tooling

When to Use Abstract Syntax Tree Tooling to Refactor at Scale

Code Audits

Transforming code in place by mutating an AST

How to Mutate an AST and Automatically Replace Code Components

Using jscodeshift to make the same code transformations with less boilerplate

How to Create Codemods Using jscodeshift

Create tests for jscodeshift transforms for a faster feedback loop

How to Unit Test jscodeshift Transforms With ts-jest

Codemods

Implement a linting rule to prevent the code we just transformed from being reintroduced in the future

How to Ensure Codebase is Up to Date With Linting Rules

Use ESLint to create the same linting rule for the code with less boilerplate

How to Use ESLint to Create AST Rules With Babel Traverse

Testing custom ESLint linting rules to verify they work as expected

How to Test Custom ESLint Linting Rules With AST

Linting

In this course, we'll start with the fundamentals of abstract syntax trees (ASTs) and learn the basic mental models. This general AST knowledge can be translated to almost any tool that works with ASTs.

## Why this course?

Understanding and using ASTs unlocks the ability to make sweeping changes in a safe and reliable way in any size codebase.

## Course topics

Throughout this course, we'll have converted source code into ASTs, traversed, mutated, and generated ASTs. With these concepts we'll then explore several practical applications including things like code audits (static analysis), code transformations (codemods), and linting.

### Module 1

We'll learn the fundamentals of abstract syntax trees. 

- What is an AST?
- How to explore an AST
- Examples of JavaScript tools that work with ASTs

### Module 2

We'll learn how to work with ASTs.

- How to turn code into an AST
- How to programmatically navigate any AST 
- How to leverage TypeScript to prevent runtime errors

### Module 3

We'll learn how to statically analyze, or "audit" code to understand the state of the codebase using abstract syntax trees.

- An introduction to an example codebase and refactor
- Understanding the state of the current codebase
- When to use an AST-based tool versus doing something manually

### Module 4

We'll learn how to transform, or "codemod" code from one state to another using abstract syntax trees.

- How to make changes to an AST
- How to change ASTs with [jscodeshift](https://github.com/facebook/jscodeshift)
- How to test a code transform

### Module 5

We'll learn how to write rules, or "lint" code using abstract syntax trees.

- How to create rules for code
- How to create custom rules with [ESLint](https://eslint.org)
- How to test a rule

In this course, you'll learn the fundamentals of abstract syntax trees, what they are, how they work, and dive into several practical use cases of abstract syntax trees to maintain a JavaScript codebase.

Practical Abstract Syntax Trees

JavaScript tools that work with abstract syntax trees

How to parse, traverse, and generate abstract syntax trees

Practical skill set for maintaining large JavaScript codebases

As you look through the codebase, you'll notice several usages of a 

 element. Since this specific codebase is React, a common pattern is to create a single 

 component that can be reused in all these places. This makes it easier to change the internal implementation in one place.

Let's walk through using AST-based tooling to make this change.

To create this new component, we need some information about the current button usages:

While it might be tempting to manually look through and read all the files, let's try using AST-based tools to answer these questions.

AST-based tooling uses many of the same steps as you would if you were manually auditing the codebase to answer these questions. If doing this code audit manually, the first step would be to locate the relevant files. Then, you would need to find any 

 elements. Finally, you would look at the props (options) and styles to discover what the 

 component needs to support. A custom script using an AST can follow these same steps, but automate it.

# Create a new directory for the audit script
# Note: the paths below assume `flash-cards` is a sibling directory
mkdir audit

# Change into the new audit directory
cd audit

# Initialize a new package.json file
npm init -y

# Add the necessary dependencies (these will be covered shortly)
npm i @babel/parser @babel/traverse glob ts-node typescript
npm i -D @types/babel__traverse @types/glob

. The first step is to find all relevant files. For this codebase, that's all the files in 

import * as fs from "fs";
import * as glob from "glob";

// First, find all files relevant files. This glob pattern
// assumes this project is a sibling to the sample codebase.
const files = glob.sync("../flash-cards/src/components/**/*.js");

files.forEach((file) => {
  // Next, read the contents of each individual file.
  const contents = fs.readFileSync(file).toString();

  console.log(contents);
});

 package which allows using glob patterns (eg: 

 is used to synchronously find all files that match the pattern (the default is asynchronous). The result is an array of strings which are file paths matching the glob pattern.

Then, for each of these files, Node's built-in 

 file system package can be used to read the file's contents.

Running this current script will print the file contents of the relevant files.

This completes the first step of finding the relevant files. The next step is to find the 

 elements. This is where an AST becomes helpful. The previous parsing and traversal script can be adapted for this.

We first need to know which node(s) to traverse in the AST. This is where 

 comes in handy. Copy and paste the following relevant code snippet into AST Explorer.

<button
  type="button"
  className="button button--primary"
  onClick={handleStartOver}
>
  <span role="img">♻</span> Start over
</button>;

Note that we chose a small piece of code instead of the entire file. This makes it easier to explore. While we chose this specific snippet, any of the button uses could have been chosen.

Clicking directly on the opening button element tag in the AST Explorer's editor should highlight the 

 node which contains the element's name: 

 property. This property contains an array of all the attributes (props) associated with this element.

 nodes will all appear similar, but inspect the 

 properties of each. These will contain nodes that represent the name of the prop and the value.

---
title: How to Audit Your Code With AST Programming
description: Create a custom script to audit a codebase
privateVideoUrl: "https://fullstack.wistia.com/medias/f64o98htdw"
code: https://gitlab.com/fullstackio/books/newline-course-apps/spencer-miskoviak-practical-abstract-syntax-trees-app/-/tree/main/code/module_04/lesson_04.02/App?ref_type=heads
---

# Performing a code audit

As you look through the codebase, you'll notice several usages of a `button` element. Since this specific codebase is React, a common pattern is to create a single `Button` component that can be reused in all these places. This makes it easier to change the internal implementation in one place.

Let's walk through using AST-based tooling to make this change.

## Understanding the existing codebase

To create this new component, we need some information about the current button usages:

-   what kind of styles (variants) need to be supported?
-   what kind of props (options) need to be supported?

While it might be tempting to manually look through and read all the files, let's try using AST-based tools to answer these questions.

## Creating a custom script

AST-based tooling uses many of the same steps as you would if you were manually auditing the codebase to answer these questions. If doing this code audit manually, the first step would be to locate the relevant files. Then, you would need to find any `button` elements. Finally, you would look at the props (options) and styles to discover what the `Button` component needs to support. A custom script using an AST can follow these same steps, but automate it.

Let's start by creating a new project.

```bash
# Create a new directory for the audit script
# Note: the paths below assume `flash-cards` is a sibling directory
mkdir audit

# Change into the new audit directory
cd audit

# Initialize a new package.json file
npm init -y

# Add the necessary dependencies (these will be covered shortly)
npm i @babel/parser @babel/traverse glob ts-node typescript
npm i -D @types/babel__traverse @types/glob
```

### Search for relevant files

Then, create a new file named `audit-00.ts`. The first step is to find all relevant files. For this codebase, that's all the files in `src/components`.

```typescript { actualFilename=./protected/audit/audit-00.ts endChar=407 endLine=14 startChar=0 startLine=1 statedFilename=audit-00.ts url=https&#x3A;//gitlab.com/fullstackio/books/practical-ast-and-codemods/module_04/lesson_04.02/protected/audit/audit-00.ts }
import * as fs from "fs";
import * as glob from "glob";

// First, find all files relevant files. This glob pattern
// assumes this project is a sibling to the sample codebase.
const files = glob.sync("../flash-cards/src/components/**/*.js");

files.forEach((file) => {
  // Next, read the contents of each individual file.
  const contents = fs.readFileSync(file).toString();

  console.log(contents);
});
```

This script is using the [`glob`](https://www.npmjs.com/package/glob) package which allows using glob patterns (eg: `*` and `**`) to find files. Here, `glob.sync` is used to synchronously find all files that match the pattern (the default is asynchronous). The result is an array of strings which are file paths matching the glob pattern.

Then, for each of these files, Node's built-in [`fs`](https://nodejs.org/api/fs.html) file system package can be used to read the file's contents.

Running this current script will print the file contents of the relevant files.

```bash
npx ts-node ./audit-00.ts
```

### Search for relevant elements

This completes the first step of finding the relevant files. The next step is to find the `button` elements. This is where an AST becomes helpful. The previous parsing and traversal script can be adapted for this.

We first need to know which node(s) to traverse in the AST. This is where [AST Explorer](https://astexplorer.net) comes in handy. Copy and paste the following relevant code snippet into AST Explorer.

```typescript { actualFilename=./protected/button-code-snippet.tsx endChar=141 endLine=8 startChar=0 startLine=1 statedFilename=button-snippet.js url=https&#x3A;//gitlab.com/fullstackio/books/practical-ast-and-codemods/module_04/lesson_04.02/protected/button-code-snippet.tsx }
<button
  type="button"
  className="button button--primary"
  onClick={handleStartOver}
>
  <span role="img">♻</span> Start over
</button>;
```

:::note

Note that we chose a small piece of code instead of the entire file. This makes it easier to explore. While we chose this specific snippet, any of the button uses could have been chosen.

:::

Clicking directly on the opening button element tag in the AST Explorer's editor should highlight the `JSXIdentifier` node which contains the element's name: `button`. Its parent is a `JSXOpeningElement` with an `attributes` property. This property contains an array of all the attributes (props) associated with this element.

![AST Explorer with JSXAttribute nodes](https://d2uusema5elisf.cloudfront.net/courses/practical-abstract-syntax-trees/module_04/lesson_04.02/public/assets/ast-explorer-jsx-attributes.png)

The `JSXAttribute` nodes will all appear similar, but inspect the `name` and `value` properties of each. These will contain nodes that represent the name of the prop and the value.

These `JSXAttribute` nodes define the names of the props used so we can see which element attributes and functionality needs to be supported. Additionally, we can inspect the class names used in the `className` prop to determine which kind of styles need to be supported.

Now that we know how to find only elements named `button` let's update the script to use `@babel/parser` to generate an AST for the file contents, then `@babel/traverse` to target only the `button` element nodes.

```typescript { actualFilename=./protected/audit/audit-01.ts endChar=1054 endLine=33 startChar=0 startLine=1 statedFilename=audit-01.ts url=https&#x3A;//gitlab.com/fullstackio/books/practical-ast-and-codemods/module_04/lesson_04.02/protected/audit/audit-01.ts }
import { parse } from "@babel/parser";
import traverse from "@babel/traverse";
import * as fs from "fs";
import * as glob from "glob";

const files = glob.sync("../flash-cards/src/components/**/*.js");

files.forEach((file) => {
  const contents = fs.readFileSync(file).toString();

  // Parse the file contents source code string into an AST.
  const ast = parse(contents, {
    // Now, more configuration options need to be set since
    // the files are now modules with imports and exports.
    // https://babeljs.io/docs/en/babel-parser#options
    sourceType: "module",
    // Additionally, JSX is not a part of JavaScript, so 
    // plugins are required to handle the JSX syntax.
    // https://babeljs.io/docs/en/plugins
    plugins: ["jsx"],
  });

  traverse(ast, {
    // Find all JSXOpeningElements, and filter to only those
    // with the name "button" and log those nodes.
    JSXOpeningElement({ node }) {
      if (node.name.type === "JSXIdentifier" && node.name.name === "button") {
        console.log(node);
      }
    },
  });
});
```

Just as in the previous lesson, this script uses `@babel/parser` to convert the file contents into an AST. It requires additional configuration options since the file contents are now modules and use [JSX](https://reactjs.org/docs/introducing-jsx.html) with React. The exact plugins will depend on the codebase and syntax being used. If babel is already a part of the codebase, the same plugins can likely be used.

Spencer Miskoviak

As you looked through the codebase, you likely noticed several usages of button

Since this specific codebase is React, a common pattern is to create a single

button component that can be reused in all these places.

This makes it easier to change the internal implementation in one place and

The first step in this process is understanding the existing codebase. To

create this new component, we need some information about the current button us

What kind of styles or variants need to be supported?

What kind of props or options or functionality needs to be supported?

In a smaller codebase like this, it might be tempting to manually look through

But let's try using AST-based tooling to answer these questions.

AST-based tooling uses many of the same steps as you would, as if you were

manually auditing the codebase to answer these questions.

If doing this code audit manually, the first step would be to locate the

Then, you would need to find any button elements.

And finally, you would look at the props or options and styles to discover what

A custom script using AST can follow these same steps, but automate it.

First, create a new directory for the audit script.

Ideally, this directory will be a sibling to the flashcards directory, since it

Then, change into the new audit directory and initialize a new package.json

Finally, we'll add the necessary dependencies.

For this audit script, we'll use the Babel parser and Babel traverse package.

We'll use glob, which we'll cover in a moment, in TS node and TypeScript for

In this case, we need the types for the Babel traverse package, because the

Babel parser package already includes first party types, as well as the types

Let's start by importing from FS the node file system package and glob from the

The first step is to find all relevant files.

For this codebase, that's all files in the source components directory.

As mentioned earlier, let's assume that the flashcards directory is a sibling

If this isn't the case, use a slightly different path to match that.

Then, we'll use the glob package to synchronously find all of the files that

This double star will match any number of directories.

This single star.js will match all of our component files.

So this method will return all of the files in our codebase that match this gl

ob pattern, and these are the files that we want to search.

Next, let's loop over each of these files and read the file contents.

Here, we're using the file system read file sync method to read the file

For now, we're printing that to the terminal.

Running the script will print the file contents of all of the relevant files in

This completes the first step of finding all of the relevant files.

The next step is to find the button elements.

The previous parsing and traversal script can be adapted for this.

We first need to know which nodes to traverse in the AST.

This is where AST Explorer comes in handy.

Open one of the files in the flashcards codebase and copy one of the button

Note that we chose a small piece of code instead of the entire file.

This makes it easier to explore the generated tree.

While we chose this specific snippet, any of the button uses could have been

Clicking directly on the opening button element tag in the AST Explorer's

editor should highlight the JSX identifier node, which contains the elements

Its parent is a JSX opening element with an attributes property.

This property contains an array of all the attributes or props associated with

The JSX attribute nodes will all appear similar, but inspect the name and value

These will contain nodes that represent the name of the prop and the value.

These JSX attribute nodes define the names of the props used so we can see

which element attributes and functionality needs to be supported.

Additionally, we can inspect the class names used in the class name prop to

determine which kind of styles need to be supported.

Now that we know how to find the only element named button, let's update the

script to use Babel parser to generate an AST for the file contents, then Babel

traverse to target only the button element nodes.

Let's start by importing the parse and traverse functions.

Now let's use the parse function to parse the contents of the file into an AST.

It will also require a little bit of additional configuration because now we're

parsing files with JSX, so we'll need to add the JSX plugin.

Additionally, all of these files are now modules, so we'll also need to define

Now that we have an AST, we can use the traverse package to find JSX opening

To check that this node's name is the JSX identifier node, and that this node's

You'll start to notice the benefit of TypeScript, providing some type ahead in

Type Safety while traversing these nodes.

Just as in the previous lesson, the script is using Babel parser to convert the