Notebooks, Services, and Scripts Troubleshooting

Common Problems:

My Notebook is taking a long time to load:

1. You are using a custom docker image.
• Civis preloads their default docker images on the resources running a Notebook. If a user brings his or her own docker image, the image will need to be pulled once the Notebook is started, resulting in a longer startup time.
2. Your organization is currently using all its allocated resources.
• If this is the case, a user may see a warning banner that states his or her organization is currently at its resource limit.
• Each organization has a max number of resources it is allowed to use at one time. Users can monitor who in their organization is using resources by visiting their organization’s Platform Usage Overview.
• Once a resource becomes available (due to another user shutting down their Notebook or Service, for example), a user will be able to start their Notebook. 

*If you are interested in increasing your resource max, please contact your client success analyst. 

My Notebook shut down unexpectedly:

1. The log history indicates a 137 exit code. This indicates that your container received a SIGKILL (kill -9) signal. These may be sent for a variety of reasons, including but not limited to:
   • Exceeding the Notebook’s memory allocation request. This could happen when trying to download or process large amounts of data. The memory stats provided in the logs may not be accurate if you see a 137 exit code. It is possible your Notebook exceeded the memory allocation request, despite the logs showing you have memory available. 
   • Failure to respond to health checks. This indicates that the Jupyter server has crashed and is no longer communicating with Platform. Restarting the notebook server should allow you to pick up where you left off
2. Notebooks have a maximum continuous uptime of 3 days. After 3 days of running, our system will shut down your notebook in order to conserve resource usage and maintain infrastructure health.

My Notebook “Disconnected from Kernel:"

1. You are reading a large dataset or file into your Notebook. If this is the case, the Notebook may have run out of memory (despite misleading error messages in the run history logs). You may also see a  “Your Notebook is failing to respond to health checks” message in the run history logs. Increase the memory of your Notebook (under settings) and rerun your Notebook.

My Script Failed:

1. The log history indicates a 137 exit code. This indicates that your script received a SIGKILL (kill -9) signal. These may be sent to the script for a variety of reasons, including but not limited to:
   • Exceeding the container’s memory allocation request. This could happen when trying to download or process large amounts of data. The memory stats provided in the logs may not be accurate if you see a 137 exit code. It is possible your Script exceeded the memory allocation request, despite the logs showing you have memory available. 
2. Scripts have a max runtime of 5 days. After 5 days of running, our system may terminate your script.