Many of Civis Platform’s features, such as Container Scripts, have built-in git integrations that allows you to automatically download the latest version of your code from a GitHub repository.

In some cases, however, you may want to be able to directly run git commands. Here are a few example situations where that may be relevant:

• If you have a complex project that needs access to code in multiple GitHub repositories
• If you want to directly run git commands to create new branches, etc.
• If you have code in a non-GitHub repository, such as a Bitbucket repository.

GitHub Repositories

To use the git command-line interface to access a GitHub repository, follow these steps:

1. Create a GitHub personal access token (PAT) via https://github.com/settings/tokens, as discussed in GitHub's documentation. There are two types: classic tokens and fine-grained tokens. Fine-grained tokens are generally recommended since they provide more specific options for access control, but they are currently in Beta.
2. Store your personal access token in a custom credential in the Credentials page in Civis Platform, as discussed here.
3. Create a new container script (e.g., by clicking “CODE” in the top navigation bar and then “Container”).
4. Add a required parameter of the “Custom Credential” type, as described here.
5. Select your custom credential with the personal access token as the argument to the script parameter.
6. Add a git command to the container script’s “Command” text box, using an environment variable to supply the access token to git. If you named the parameter “GITHUB_TOKEN”, then the environment variable “GITHUB_TOKEN_PASSWORD” will have the GitHub personal access token as its value. Here is an example command for cloning a private repository: git clone https://${GITHUB_TOKEN_PASSWORD}@github.com/my_org/private_repo.git

An Alternative for GitHub Fine-grained Personal Access Tokens

In some cases, fine-grained personal access tokens may not work as described above. Instead of providing the GITHUB_TOKEN_PASSWORD environment variable in the URL, you may need to authenticate using GitHub’s command-line interface tool. To do so, the container script command can include the following:

The example above makes a couple assumptions:

• That your custom credential is named "GITHUB_TOKEN".
• That the Docker image has apt-get, which is usually true for Debian and Ubuntu images such as civisanalytics/datascience-python.

Non-GitHub Repositories

In a Container Script, you can check out code from a non-GitHub remote repository such as Atlassian BitBucket or GitLab. For public repositories, you should be able to use git pull commands directly in your script. For private repositories, you’ll need to follow service-specific instructions to authenticate in order to use git commands in your script. For Bitbucket, see their documentation about Bitbucket App passwords. For Gitlab, see their documentation about GitLab Personal access tokens.

As described above for GitHub, any tokens or other secrets needed for authentication may be securely stored as Custom Credentials in Civis Platform and then accessed as environment variables in your script.

A Note About Directories

If you want to clone a repo into a particular directory, then you’ll need to change the current working directory before running the git clone command. If the directory doesn’t exist, you’ll need to create it first with the mkdir command.

For example, if you want the repository to be at /app/private_repo, then you would run “mkdir -p /app” and "cd /app" before your "git clone" command.

Related Resources

• This page describes how to use GitHub tokens in a Python notebook.
• This page describes how to use GitHub tokens in an R notebook.