Version Control: Notebooks

Civis Platform supports Git in Jupyter Notebooks through the Notebook terminal as well as the version control side panel. When a Notebook is running, the Notebook terminal must be used to connect to Git; when a Notebook is not running, the version control side panel must be used to connect to Git.

This is a walkthrough of how to configure a Platform Notebook to use Git through the Notebook terminal. If you use Bitbucket, you will need to use the version control side panel. For instructions on how to integrate Notebooks with Git using the version control side panel visit this link.

Connecting Platform to Github/BitBucket

First, add a Git repository to Platform through your user profile. This must be a Github or BitBucket repository if you are interacting with Git through the Notebook terminal. 

Click on the “Git Repos” tab at the top of the page. You’ll navigate to a page that shows a list of repositories that you have bookmarked to your profile. You can add new repositories by clicking the "Add Repository" button.

Next, go to the Platform notebook where you would like to establish a Git integration.

• To bring an existing notebook from Git into Platform, create a new Platform notebook with the server stopped.
• To commit an existing Platform notebook to Git, open the existing Platform notebook with the server stopped.

In order to integrate git with notebooks, you need to use image tag 1.5 (python 3) or 1.3.0 (python 2/R)  or later.

Before starting the server, add a Git repository to your notebook by filling out the three Git-relevant fields.

Repository: The repository field is a drop-down of the list of repositories that you have bookmarked in your profile.

Branch: The branch field is the specific branch, tag, or commit hash. The reference you use must already exist in your repository.

Path: The path field is the path to a Notebook in your Git repository. If you are connecting this Notebook to your repository for the first time, you can type the path of the notebook file you would like to be created. Please note that the file must have an .ipynb extension, or your Notebook will fail to start.

When you have configured all of your Notebook settings, select "Manage Version", and choose the version that you want to use in the job. As you edit and save your Notebook, you’ll see an “uncommitted changes” Notification appear at the top of the Notebook.

Committing and Pushing Changes to Git

Notebooks changes are committed and pushed to Git through the terminal window in your Notebook. Executing Git commands in the terminal will use the authentication from the repository you set up in your profile to push changes to your repository. You can access the terminal by clicking on “Uncommitted Changes.” You can also access the terminal by clicking on the terminal icon in the Notebook:

To orient yourself within the terminal, first type “git status” at the command prompt.

You’ll see the branch you are working in as well as the name of the Notebook that has been modified and needs to be committed to Git. Note that the html preview from your most recent save is also available -- we do not recommend including this file in your commit.

Next, the following steps are required to commit your changes to Git:

• git add <filename> -- stage your changes for commit
• git commit -m “<<commit message>>” -- commit your changes and specify a commit message
• git push -- publish your commits to this notebook’s repository and branch

Requirements and Custom Packages

In addition to the Custom Packages option for adding packages to the Docker image, you also have the option of using a requirements.txt file from your Git repository. When you launch a notebook that has a Git repository specified, Platform will search your repository for a requirements.txt file. For example, if you provide a path to a Notebook at /notebooks/clustering/k_means.ipynb, Platform will search up the tree until it finds a requirements.txt file. If you don’t have a requirements.txt file in your repository, you can add packages to the “custom packages” list in the packages pane. However, we only use one OR the other. If you list any custom packages in the packages pane, the requirements.txt file will be ignored.

Autosaving vs Committing

Platform will continuously save a copy of the Notebook file as a backup in case you forget to commit your changes back to your Git repository. When you restart your Notebook, and the copy we have saved in Platform is not empty, we will launch the Notebook that is autosaved in Platform. If you want to use the version that is committed to Git, you can use the terminal to access a past commit.