GitHub, the leading platform for software development and collaboration, has recently announced that it has adopted GitHub Actions for a part of its continuous integration system. GitHub Actions is a feature that allows users to create custom workflows that automate various tasks, such as testing, building, deploying, and releasing software. By using GitHub Actions, GitHub has improved how it ships software in a scalable and effective way.
What are GitHub Actions and Runners?
GitHub Actions are composed of two main components: workflows and runners. Workflows are YAML files that define the steps and conditions for running a specific task. Runners are the machines that execute the workflows. Runners can be either hosted by GitHub or self-hosted by the users. Hosted runners are provisioned by GitHub and have a fixed set of hardware and software specifications. Self-hosted runners are managed by the users and can have any configuration and environment.
One of the advantages of self-hosted runners is that they can access private resources that are not available to hosted runners, such as artifact storage, application metadata services, and other internal services. However, self-hosted runners also require more maintenance and security measures from the users.
How GitHub Uses Actions and Runners for GitHub.com
GitHub has been using GitHub Actions for some of its internal projects since 2019, but it was not until recently that it started to use it for a part of its continuous integration system for GitHub.com. According to a blog post by GitHub engineers, they leveraged the new Actions larger runners to run 15,000 CI jobs across 150,000 cores. Larger runners are a type of self-hosted runners that can handle more intensive workloads and have more memory and CPU resources than hosted runners.
To use larger runners, GitHub had to create a gateway that allows them to access some of their private infrastructure while preserving security. The gateway works by minting an OIDC token in GitHub Actions, passing the token to a remote access gateway that validates the token, and then proxying the request to the private service residing in a private network.
GitHub also used some other features provided by GitHub Actions, such as reusable workflows and reusing previous workflow outcomes. Reusable workflows are meant to provide a mechanism to manage a collection of workflows in a single repository so they can be reused in other repositories. This was key to make the process of adding CI to a repository seamless. Reusing previous workflow outcomes is an optimization mechanism that prevents running a workflow when the file contents of a repository are exactly the same as they were in a previous run of the workflow. This happens when the Git tree ID for the current commit is the same as the previous commit. In such cases, running the workflow would produce exactly the same outcome, so instead of running it again, the results of the previous run are reused.
What are the Benefits of Using Actions and Runners for GitHub.com
By using GitHub Actions and larger runners for their continuous integration system, GitHub has achieved several benefits, such as:
- Faster feedback loop: By using larger runners, GitHub was able to reduce the average time for running CI jobs from 30 minutes to 10 minutes.
- Higher reliability: By using self-hosted runners, GitHub was able to avoid issues related to network latency, throttling, or outages that could affect hosted runners.
- Better scalability: By using GitHub Actions, GitHub was able to scale up or down their CI capacity based on their needs and demand.
- Improved developer experience: By using reusable workflows and reusing previous outcomes, GitHub was able to simplify and streamline their CI process and make it easier for developers to add new repositories or update existing ones.
How You Can Use Actions and Runners for Your Own Projects
If you are interested in using GitHub Actions and larger runners for your own projects, you can follow these steps:
- Create a GitHub account if you don’t have one already.
- Install Helm 3 on your machine if you don’t have it already.
- Create or join a Kubernetes cluster where you want to deploy your larger runners.
- Install the Actions Runner Controller (ARC) on your cluster using Helm. ARC is a Kubernetes operator that orchestrates and scales self-hosted runners for GitHub Actions.
- Generate a personal access token or create a GitHub App to authenticate ARC to the GitHub API.
- Configure your runner scale set using ARC. A runner scale set is a group of larger runners that share the same configuration and can scale up or down automatically based on the number of workflows running in your repository, organization, or enterprise.
- Create or update your workflows to use your runner scale set as the value of runs-on.
- Enjoy faster and more reliable CI with GitHub Actions and larger runners.
For more details and instructions on how to use ARC, you can refer to Quickstart for Actions Runner Controller in the GitHub documentation.
