Python Notebooks

Civis Platform supports Jupyter notebooks for R and Python. This is a walkthrough of how to use Python notebooks in Platform. 

Civis offers an open-source Python API client that allows users to interact with the Platform from Notebooks and Python Scripts, or from their local console. While you are free to write Python code that does not use the client, we recommend using it to streamline your workflows. Click here for a sample notebook on how to use the Civis Python API Client.

Python Notebooks run in Python 3. To create a new notebook, at the top of the Platform interface, go to 'CODE" and choose 'Python 3'. 

Notebook Settings

After choosing your language, you can configure your settings before running any code. 

The above shows the default configuration for a Python notebook. To see which packages are already loaded in the Notebook, click "View Packages". You can load any additional packages after starting the notebook by running the following command:

pip install <package_name>

To add a credential to your notebook, select a credential that you have already stored in Platform from the dropdown, or click 'Add a New Credential'. The credential will be stored as two environment variables: 'credentialname_USERNAME' and 'credentialname_PASSWORD'. For example, to access a credential called 'tester', you would run the following commands (note that all letters in the environment variable are capitalized):

username = os.environ["TESTER_USERNAME"]
password = os.environ["TESTER_PASSWORD"]

When you're ready to start your notebook, click "Start Server" (note that this may take a few minutes). 

Please Note: once your notebook has been started, you may be unable to edit the notebook settings. To get around this, stop your notebook by selecting "Shutdown Server" and then go into the settings.

Jupyter has its own internal help documentation--for a tutorial on how to use Jupyter notebooks, go to the "Help" tab and select "User Interface Tour".

When you are finished running your notebook, you MUST save the notebook and click "Shutdown Server" in the top-righthand corner of the notebook. If you do not save the notebook before shutting it down, all of your changes will be lost. If you do not shut down the notebook before exiting, it will continue to run and consume computing resources. If a notebook is idle (i.e. no processes in the notebook are running) for a continuous 3 hours, the notebook will automatically shut down.

Magics

To load the magic commands, use the following in ipython or a Jupyter notebook:

%load_ext civis_jupyter_ext

You can also autoload the magic commands every time a notebook is opened by adding:

c.InteractiveShellApp.extensions = ['civis_jupyter_ext']

to your ~/.ipython/profile_default/ipython_config.py.

SQL Queries

To get a table preview, use the cell magic like this:

%%civisquery my-database
select * from dummy.table limit 10;

To return a DataFrame for further processing, use the line magic like this:

df = %civisquery my-database select * from dummy.table;

Additional Documentation

To learn more about Civis' Python API client, click here. Click here for an example Python notebook with a walkthrough of how to use Civis' modeling tool, CivisML.