Civis users can write their own Python 3 code via Scripts. From the top navigation menu, select Code, and then Scripts. Next, click on New Script in the top-right corner of the page.
If you do not see this option under scripts and would like to use Python scripts, please contact support@civisanalytics.com to have it added.
Import the Client
To make using Python within Civis simpler, Civis offers an open-source Civis API Python client that allows users to interact with the Platform from Python Scripts in Platform, or from their local console. While you are free to write Python code that does not use the client, we recommend using it to streamline your workflows. The civis namespace contains tools for typical workflows aimed at optimizing user efficiency. To install the Civis Python Client, simply run the following in your terminal environment:
pip install civis
In order to use the civis package in your scripts, you will need to run the following line at the beginning of each script:
import civis
Test Configuration
It is a good idea to test that you have properly imported the python client by running a simple script. Test the following to ensure you are configured correctly:
import civis
client = civis.APIClient()
me = client.users.list_me()
print("My username is " + me.username)
Once you run this script, you can check the Run Logs to see if the script ran successfully. If done properly, in the log you should see your print statement. See below:
You are now configured with the Civis Client and are able to use Python 3 code to interact with the Platform. For additional documentation on the Civis client, please reference the Python User Guide.
Accessing Redshift
A common workflow is to manage data in a Pandas dataframe. To extract a table from Redshift into a dataframe, you can use the following code:
import civis
df = civis.io.read_civis(table="my_schema.my_table",
database="database",
use_pandas=True)
Accessing Parameters
If you want to make use of parameters within your Python script, they will be stored as environment variables. You can reference these using the command os.environ['PARAMETER_NAME'].
my_table = os.environ['TABLE_ID']
Default Environment Variables
In addition to the environment variables generated by parameters, Scripts run in Civis automatically inherit several environment variables at run time. The relevant variables provided by Civis are:
Environment Variable | Description |
CIVIS_JOB_ID | The ID of the Script |
CIVIS_RUN_ID | The ID of the Run for this Script |
CIVIS_API_KEY | The API Key generated at run time that you can use to make calls to the Civis API during this script run |
Scheduling Your Script
If you'd like to set your script to run on a schedule, click on the clock icon in the top-right corner of the page. From there you can have the export run on a schedule of your choosing, or else incorporate it into a workflow. For more information about job automation, see the Automate and Workflow help documentation.
Packages
Available Packages
A set of Python packages are automatically installed and available for use on all python scripts. To view what is available, see: https://github.com/civisanalytics/datascience-python/blob/master/requirements-full.txt.
Installing Additional Packages
To install any additional packages, we recommend switching to a container script and adding additional packages in the command line.
Docker Images
New Python Scripts use the latest available version of the public civisanalytics/datascience-python Docker image. See this page for information about Civis Docker images.
Variables in Notification Emails
You can embed values produced by your script in the script's success notification email. To leverage this feature, upload any values or tables you want to include in your email as a JSONValue run output named "email_outputs".
Open the Notify pane (paper airplane icon) and include {{variable}} in the Email Body text box where you want to input the corresponding value. If you are embedding a Markdown table, you need to include line breaks before and after the {{table}}.
See here for a code example that provides query_to_variable and query_to_markdown_table functions, and demonstrates how to upload tables and variables to a JSON run output.
Comments
0 comments
Please sign in to leave a comment.