Continuous Integration

Overview

NUbots uses GitHub Actions for its continuous integration pipelines. The pipelines are triggered automatically when pull requests are made, updated, or merged. The pipelines build and test the code on GitHub's cloud, with logs and results available on the GitHub website, associated with the relevant pull request.

Branch Workflow

This section assumes basic knowledge of Git. If you need a git refresher, see the Git guide.

This CI process works in combination with the Git Feature Branch workflow we use for development:

1. Developers branch off main, creating a new branch with a name of the form <lastname>/<purpose-of-branch>. For example:

   Copy
   # Make sure main is up to date before creating your branch
   git fetch
   git checkout main
   git pull
   
   # Create and check out the branch off of main
   git checkout -b biddulph/ball-detector

2. Once development on the branch is complete, or reaches a point where it can be reviewed, the developer makes a pull request (PR) on GitHub. Creating the pull request triggers GitHub action agents to check the code. GitHub automatically creates separate build agents for different parts of the PR (build, check code format, test, etc).

3. Each agent completes a set of tasks specified in a workflow file, found inside the .github/workflows directory of the codebase. These steps usually involve cloning the codebase, checking out the relevant commit, pulling down the docker image (rebuilding it if it has changed), then checking some aspect of the code.

4. GitHub logs the results of each job and displays the results on the webpage for the pull request. The logs are viewable in the 'actions' tab of the PR.

5. Once the build passes, the PR is reviewed by other members of the team. Once the PR has been approved and all required checks pass, it can be merged into main.

6. Merging the PR into main then triggers another CI build. This time, in addition to building the code, a separate job is created to rebuild the main docker images and push them to DockerHub. This is the main way that our docker images are kept up to date.

CI Overview PRCI Overview Main

The Build Pipeline

GitHub Actions are configured in workflow files, written in YAML. To add, remove, or change build jobs, we edit the relevant workflow file in the NUbots repo. The GitHub Actions documentation should be your first stop when you want to make changes.

Workflow Outline

Note that the source of truth for what a workflow actually does is the given workflow file. It defines precisely what happens.

When a pull request is created or its branch is modified, checks are run on the code to make sure that it builds and maintains standards of correctness. First, an action checks which parts of the codebase have been modified, which informs which checks should be run. For instance, changes to C++ code require the C++ code to be built, and its formatting checked.

Next, a docker image tag is built and pushed to DockerHub for the PR. This is cached from the PR image, if it already exists, or the main branch image if it does not. The caching means that only what needs to be rebuilt in the image is rebuilt, to the point that if the image hasn't changed, the PR image tag simply acts as an alias for the image it cached from.

Finally, all other checks are run conditionally, based on whether the files which those checks apply to have changed. These checks are run inside the Docker container associated with the PR image.

The results from each of these steps are displayed on the pull request page on GitHub (see below).

Each of these steps happens when a pull request is merged into the main branch, however, in this case the results are displayed on the main page of the repository, rather than the pull request page.

Additionally, when pull requests are merged into main, docker images are built, tagged, and pushed to be the new standard images we use - for example, the generic image used for most of our development. This actions workflow is the primary way that those standard images are updated.

Monitoring and Troubleshooting Builds

GitHub Actions have live-updating logs of each build, which you can use to monitor the build progress or troubleshoot errors when a build fails.

Failed builds and pipeline steps can be diagnosed by viewing their logs. To view them, scroll to the build checks at the bottom of the PR page and click the Details link on the failed check.

Failed steps usually require changes to your pull request. Pushing your updated branch to GitHub automatically triggers a fresh build, with no extra steps required. The build status of your pull request will automatically update with the results.