Overview

When we talk about Services at Civis, we’re referring to persistent applications that can be accessed over the web by consumers. If you have built or used a Shiny app hosted on Civis Platform, you are already familiar with services at Civis.

The same infrastructure that makes Shiny apps possible can be used to enable a much broader array of web services. One type of data science advanced deployment that we enable is the deployed model. Deployed models are trained machine learning models that serve predictions in real time via a web API.

For example, imagine that you have a retail website and your data science team has built a model to recommend products to customers as they arrive to your website. To make the predictions from this model accessible to your website, you could deploy the model. As a potential customer arrives on your site, you could have your servers (or javascript running in the customer’s browser) request a recommendation for that customer from the deployed model.

To deploy a model or any other web app, you must encapsulate it inside of a Docker container. When you instruct Civis Platform to deploy your app, Platform grabs your container and creates a Kubernetes pod for it. You can think of the pod as an isolated compute environment where just your app and its supporting services live. Once the container holding your web app is running, web requests are passed to your app through an intermediary web server, automatically provisioned and controlled by Kubernetes.

A broad range of bespoke web services can be deployed in Civis Platform, thanks to several advanced features.

Environment Variables

Environment variables inside of the container running your service can be set via the environmentVariables attribute of the patch services/{id} API endpoint. The environment variables that you set will be passed into the container and accessible to your service. While any variable may be set, there are a few reserved environment variables which may be used to configure the service:

• CIVIS_SERVICE_PORT - used if your service responds to a non-default port.  The default port is 3838.
• CIVIS_SERVICE_HEALTH_PATH - used to check if your service is healthy, defaults to the root / endpoint. See “Health Check and Port Configuration” below for more details. 
• CIVIS_SERVICE_VERSION - automatically provided to the service, but may be overridden.  See “Versioning” below for more information.
• CIVIS_SERVICE_LOG_PREFIX - customizes destination of analytics logs in Spectrum.
• CIVIS_PERMISSION_SET_ID - ID of attached permission set, if relevant. See “Permission Sets” below for more details.

Health Check and Port Configuration

Platform will check for a 200 HTTP code on the root ("/") endpoint of your service at CIVIS_SERVICE_PORT to check that it is alive. If you'd like a different endpoint to indicate that the service is ok, you can set the CIVIS_SERVICE_HEALTH_PATH environment variable on the service.

Versioning

By default, Civis assigns a version number to each service deployment and increases that number by 1 every time the service is redeployed. This value is passed into the model as the following environment variable: CIVIS_SERVICE_VERSION. Users have the option to set this environment variable. The CIVIS_SERVICE_VERSION must be between 1 and 32 characters and must start and end with alphanumeric characters. Optionally, it can have dashes or underscores.

Permission Sets

Permission sets allow you to control access to different parts of your app for different users. From the service’s configuration page, you can link an existing permission set or create a new one. The ID of this permission set will be provided in the app's environment as CIVIS_PERMISSION_SET_ID. Please see the Permission Set documentation for more details.

Service Tokens

User Tokens

After authenticating a user, Platform passes several values to your service as query parameters to help you identify the requestor. These values are:

• civisuserid - The ID of the user accessing your service
• civisusername - The username of the user accessing your service
• civisuserorgid - The organization ID of the user organization accessing your service

Note that if a user includes these values as part of their request, they will be overwritten before reaching your service.

Machine Tokens

If your service does not require user information, you can use machine tokens for authentication instead. These service tokens function the same as user tokens, but do not carry user information. This makes them useful for public service URLs.

Service Token Authentication

Platform services are available at the public endpoints shown at the bottom of the service page. When accessing these services through a report or preview, cookie-based authentication is used transparently. To access service endpoints from outside of Platform, you can create Service Tokens. By default, service tokens expire in 24 hours.  You may customize a token’s expiration date when you create it.  

If your token expires, you must create a new one to continue accessing your service outside of Platform. 

Creating Service Tokens

You can create a service token from either the API or the UI.

API: Use the post /services/{id}/tokens endpoint. To set a custom expiration date for your token, use the expiresIn parameter.

UI: Navigate to your Service’s detail page and open the “Advanced” tab at the bottom of the Service Settings.

Create a descriptive name for your token, select a non-default Expiration Date if desired, and click “Generate Token”.

Using Service Tokens

To access your service using a Service Token, you may use either an Authorization header or the civis_service_token query param.  For example, in curl you would execute

To mitigate the risk of a token being compromised, we recommend rotating tokens on a regular cadence, say, monthly. If you are no longer using a token, or suspect it may have been compromised, you can revoke its access via the Services UI or the delete /services/{id}/tokens/{token_id} API endpoint.

Public URLs

You can also use service tokens as part of an embeddable, public service URL. By filling out token information and then clicking on the ‘Generate…’ button, a URL will be automatically generated and copied to your clipboard. You can share this link with colleagues or embed it in a different webpage, and it will last for as long as the token is valid. By clicking this button, you are also generating a machine token for authenticating consumers of the public URL, which will appear in the list of tokens.

Please note that the public URL will only work if the service is already running. For this reason, On Demand scheduling is not a good fit for services with public links. Instead, we recommend setting them to be Always On. Alternatively, if you know when users will be accessing your dashboard, then you can schedule it for Specific Times.

Autoscaling

Autoscaling may be enabled by defining the maxReplicas field on a service via the API. When maxReplicas is defined, the service will automatically scale up and down as CPU load on the existing replica set heightens or drops. The autoscaler will target 75% CPU usage across all replicas in the deployment. The replicas attribute is the minimum number of replicas to be deployed.

When using autoscaling, it is important to ensure that the service can use all of the resources configured for a single replica. That is, if you set CPU > 1000m (1000 millicores = 1 CPU), your app must be able to use more than one CPU, otherwise it will not scale. However, it is not recommended to set CPU < 550 unless your service is expected to be handling large amounts of traffic (≥50 requests per second) before maxing out its CPU.

Autoscaling is not recommended for Shiny apps, as they make use of session state which is stored in memory and may not be preserved for users when downscaling.