Networking and Web Requests

The lifecycle of a typical Caliper SDK web request is something like this:

Q2 Online -> HQ -> Extension -> HQ -> Online

HQ will always POST data related to the current online user to your extension, but what if you need more information not present in this POST?

What if you need to use that information to look up a transaction date in your core?

Or query a third-party service, such as the Twitter API?

You’re going to need to make a web request. With the Caliper SDK, we’ve broken web requests into three categories.

Communication with the Q2 Database
Communication with the your Core
Communication with a third-party service

Q2 Database

The information you require might be present in Q2’s database, but not present in the standard online user data returned to the extension. We do not communicate directly with the database, but instead use the relevant HQ API endpoint.

HQ communicates using either SOAP or JSON. We talk a great deal about HQ operations in the section HQ API, but to make this a bit easier to manage, we’ve created a Python wrapper around many useful HQ endpoints. There is a generic form of the wrapper included in the Caliper SDK. To use it, you can simply import the desired module. For instance:

from q2_sdk.hq.hq_api.q2_api import ExecuteStoredProcedure

If the generic hq_api bindings are not appropriate for some reason, you can generate your own and alter them to your liking for the desired module:

$ q2 generate_hq_api -a ExecuteStoredProcedure

If you’ve been through the Caliper Tutorials, you’ve already used this method to make a web request. Here’s a sample:

from q2_sdk.hq.hq_api.q2_api import GetUserAccountList
params_obj = GetUserAccountList.ParamsObj(
    self.logger,
    user_name,
    7,
    True,
    'D*|L*|C*'
)
hq_response = await GetUserAccountList.execute(params_obj)

This code will query our database using an HQ endpoint called GetUserAccountList, which, as you might expect, returns a list of user account information.

Core

Connecting to your core is more involved. Testing against a core introduces the potential for interfering with production resources, moving real money, and other concerns that go along with using a part of the stack not meant for development use.

Helping our users safely develop code to make direct core requests is a serious and ongoing challenge for the Caliper team. Our carefully considered approach is outlined in the Custom Coreflows section of our guides. Please read through that section for more information.

Third-party Services

Perhaps you need to talk to Twitter, or you have created your own REST API that you would like to call from an Caliper extension. The Caliper SDK fully supports arbitrary third-party web requests. We built our request tools on top of the popular Requests library. However, Requests was not originally designed to support asynchronous requests, so we created Q2 Requests to ensure the full benefits of asynchronous behavior are realized.

Here’s an example of how to use q2_requests:

from q2_sdk.core import q2_requests
response = await q2_requests.get('https://www.q2ebanking.com/')

It’s that simple. A method exists for each commonly used http method: GET, POST, PUT, DELETE, OPTIONS, PATCH, and HEAD.

As a further example, to POST a request with data, use the post method and add the payload as a keyword argument:

from q2_sdk.core import q2_requests
response = await q2_requests.post('https://www.q2ebanking.com/', data={'hello': 'world'})

In addition, the examples from the Requests documentation page are still valid, just be sure to put an await keyword before the call.

Important

Q2 blocks all outbound traffic in our datacenter by default. To enable a url through our firewall, a networking request must be made through Salesforce. If you are an SDK Client, please contact Q2 Support to whitelist any url you are planning to use in staging or production.

The settings file contains a variable called OUTBOUND_WHITELIST to help remind you of this. It is possible to suppress this warning by passing verify_whitelist=False to your q2_request function call.

For instructions on whitelisting a URL with Q2 Support, refer to the Whistlist Help Page