Periodic Entrypoint Tutorial

Periodic entrypoints (also known as “batch”, “cron”, or “scheduled”) are tasks meant to be run repeatedly at a set interval, rather than exposed as an endpoint on a persistent web server. Generating the code to create one is similar to creating an extension:

Make sure you are on Caliper SDK version 2.37.0 or greater
Type q2 create_entrypoint at the Command Line Interface (CLI)
When prompted, fill in a name: for this tutorial, we’ll choose “LogTheTime”
When prompted for periodic job support, select “Y”

This will append “LogTheTime” to the INSTALLED_ENTRYPOINTS list in settings.py, as well as generate a new file called LogTheTime/entrypoint.py.

Let’s open that file and examine what was generated for us.

EntryPoint Class

In the entrypoint.py file, you will find a class called EntryPoint that inherits from PeriodicJobEntrypoint. It is important not to alter this name– the classname EntryPoint informs the SDK that it is to be registered and and available through the CLI, as shown below:

$ q2 -h
...
LogTheTime          No help text defined
...

Neato! We can update the name by changing the name class property, and update the help text by changing the help class property.

Let’s update help to be something, well, helpful.

    class EntryPoint(PeriodicJobEntrypoint):
        name = 'LogTheTime'
        help = 'Logs the current time'
    ...

Now try the CLI command again:

$ q2 -h
...
LogTheTime          Logs the current time
...

Assuming you have the CLI completion scripts installed (if not, try running q2 install_completion to make your life easier!) your entrypoint is now available via completion, so typing q2 Log<tab> should complete the command for you.

Methods

PeriodicEntrypoints provide three main hooks for configuring your repeatable task.

run

The run method is called when your entrypoint is invoked. For our task, this should show the current time.

First, import Python’s datetime module at the top of the file:

import datetime

Then we can replace the current ‘Doing work’ placeholder line in the run method with the following:

    async def run(self):
        current_time = datetime.now()
        return current_time

Now if we were to run our entrypoint:

$ q2 LogTheTime

2020-07-28T14:04:16.921 q2_sdk.general  INFO     job LogTheTime-LogTheTime began execution at 2020-07-28T19:04
2020-07-28T14:04:16.921 q2_sdk.general  INFO     job LogTheTime-LogTheTime successfully executed.

Nothing visible happens yet except the default logging behavior. The run method can be extended to do anything the Caliper SDK is capable of– calling HQ, interacting with 3rd party APIs, generating reports, and so on.

on_success

The on_success method is called if the run method finishes without raising an exception. It’s useful for cleanup tasks, such as sending a confirmation email. Let’s use it to print the value we calculated in run:

    async def on_success(self, run_result):
        self.logger.info(f'Success! The current time is {run_result}')

Now when running our entrypoint:

$ q2 LogTheTime

2020-07-28T14:49:02.082 q2_sdk.general  INFO     job LogTheTime-LogTheTime began execution at 2020-07-28T19:49
2020-07-28T14:49:02.082 q2_sdk.general  INFO     Success! The current time is 2020-07-28 14:49:02.082477
2020-07-28T14:49:02.082 q2_sdk.general  INFO     job LogTheTime-LogTheTime successfully executed.

on_failure

The on_failure method is called if the run method raises an exception. It is useful for cleanup tasks as well. Let’s update our run method to raise an Exception for demonstration purposes:

    async def run(self):
        current_time = datetime.now()
        raise Exception
        return current_time

And update our on_failure method to log some information:

    async def on_failure(self, err: Exception, failure_dict: dict):
        self.logger.error(f"Current time was: {failure_dict['local_vars']['current_time']}")

Running our entrypoint with this forced failure prints the following:

$ q2 LogTheTime

...
2020-07-28T15:31:01.697 q2_sdk.general  INFO     {'self': {'logger': <Logger q2_sdk.general (INFO)>, ...
2020-07-28T15:31:01.698 q2_sdk.general  ERROR    Current time was: 2020-07-28 15:31:01.567868
...

If you need to take any action after a failure before exiting, this is the place.

Note that the program state is logged by default on failure. If you find your job failing but did not explicitly log some useful information, examining this data can help track down the problem.

Scheduling

Now that you’ve built your over-engineered clock, it’s time to hand it over and get it running in the Q2 datacenter. Requesting this is the same process you are likely already familiar with, described in detail in Review and Deployment. Be sure to put the desired scheduling in the ‘Anything else we should know?’ section of the deployment request.