HTTP Request

HTTP requests can be sent to a specific URL using various HTTP methods.

Use a HTTP Request Action to interact with REST APIs and web applications. For example: find all tweets mentioning specific keywords; update a JIRA ticket based on incoming Events; create a Pager Duty incident; check VirusTotal for an MD5 hash.

Features​ 

  • Send requests on receipt of incoming Events, or run on a schedule.

  • Send requests using 'GET', 'POST', 'PUT', 'PATCH' and 'DELETE' methods.

  • Enable/disable SSL verification. TLS 1.2 and 1.3 are supported.

  • Optionally specify HTTP headers, including user agent.

  • Include information from incoming Events in HTTP requests

  • Request response will be emitted as a new Event

Configuration Options​ 

  • url: Specify where the request should be sent. Include the URI scheme ('http' or 'https').

  • basic_auth: (Optional) Specify HTTP basic auth parameters: "username:password", or ["username", "password"].

  • content_type: (Optional) Specify the content type to use with the request. When set, Tines will perform smart validation of HTTP request payload to fit the content type. If this is not desired behavior, set a content type header using the headers option instead. Shorthands are provided for the following, common content types:

    • 'application/json; charset=utf-8': json

    • 'text/xml; charset=utf-8': xml

    • 'application/x-www-form-urlencoded': form

    • 'multipart/form-data': data

  • custom_certificate_authority_name: (Optional) Specify the name of the custom certificate authority to use for this request. Defaults to 'default', if present

  • customize_output: (Optional) Control the event data this action produces. Note that event and LOCAL data are not available here.

  • disable_redirect_follow: (Optional) Disables automatic following of redirects during HTTP requests. When false, only the results of the last request, as a result of the last redirect are shown. False by default.

  • disable_ssl_verification: (Optional) Set to 'true' to disable ssl verification.

  • disable_url_encoding: (Optional) Disable automatic URL encoding of non-URL-safe characters in the URL and query parameters. False by default.

  • emit_failure_event: (Optional) Control when events are emitted for error conditions without setting up failure paths.

    • Always: Emit events for any logged error. If retry_on_status is configured, a single failure event will be emitted after all retries have been exhausted.

    • Never: Never emit events for logged errors (story run will stop).

    • Error response only (HTTP Request actions only): Emit events for HTTP status code errors (404, 500, etc.) but not transient connection failures (timeouts, DNS errors). This is the default behavior for HTTP Request actions. See also retry_on_status and log_error_on_status.

  • headers: (Optional) Specify a hash of headers to send with the request.

  • https_only: (Optional) Require that all requests use the HTTPS protocol, including redirects.

  • local_values: (Optional) A place for performing upfront calculations, which can then be re-used from the LOCAL object.

  • log_error_if: Log an error if the given formula is true, based on the output event data.

  • log_error_on_status: [0, 400-499, 500-599] by default. Specify the array of status codes that should cause an error to be logged. If the HTTP response received by the action has one of these codes, then an error will be logged. Note, if the action is configured to both log_error_on_status and retry_on_status for a status code, it will first retry and log that it failed but is retrying, but no error will be logged until all retries are exhausted.

    • Each array element can be either a single status code (e.g. 400), or a range of status codes (e.g. 400-499). Ranges are inclusive of starting and ending values.

  • method: (Optional) Specify the HTTP method to use, i.e.: get, post, put, patch, or delete. Defaults to post.

  • mutual_tls: (Optional) Credentials to use mutual TLS for the request. Must be an object with the following keys:

    • root_certificate: The root certificate for the certificate authority (CA) responsible for signatures.

    • client_certificate: The certificate issued by the CA for this client.

    • client_private_key: The private key for the client certificate. For convenience, this can also be an interpolated Mutual TLS credential containing the required information.

  • use_ntlm_authentication: (Optional) For connecting to older Windows-based systems. When enabled, set the username and password using basic auth.

  • pagination: (Optional) Automatically paginate through the results of the request.

    • has_more: Boolean condition to indicate if there are more results to fetch. The first request will be made regardless of the condition value. Subsequent requests will only be made while this condition is true AND the request count is below the limit of 100.

      • An example of a has_more condition might be a formula that checks if the current_page is less than total_pages in the response.

      • PAGINATE variables, which hold data about the pagination run, might be helpful to use in the formula.

    • If output as single event is true, outputs all page responses as a single event once the last request has completed.

      • The maximum size limit for each page is 10 MB.

      • The combined event is still subject to the 100 MB size limit.

      • When an error occurs, such as an HTTP error status code or a connection error, the pagination run will stop short and emit an event with responses up to and including the error response.

  • params_encoder: (Optional) When set to 'flat' allows duplicate query parameters and when 'nested' duplicate params are not permitted. When 'nested', query params included in the payload are added to the query using array syntax eg. /example?key[]=value_1&key[]=value_2. Defaults to 'nested'.

  • parse_response_as: (Optional) Choose how responses are converted into event data, the default is to infer this from the response's Content-type header. Available options are:

    • text: Treat the response as plain text

    • json: Attempt to parse the response as JSON

    • xml: Attempt to parse the response as XML

    • file: Store the response as a file object with base64 encoded contents

  • payload: (Optional) Specify key-value parameters to include in the body of the request. Use wrapped JSONPaths to include data from incoming Events.

  • http_proxy_url: (Optional) Direct your requests through a specified proxy URL.

  • query_params: (Optional) Specify key-value query parameters to be appended to the URL of the request. Query params can be used with both flat and nested query param formats, and with or without url encoding. Use wrapped JSONPaths to include data from incoming Events.

  • redirect_authorization: (Optional) Retain the authorization header when following a redirect to a different hostname.

  • retries: (Optional) Specify the maximum number of retries. This does not affect the interval of retries, only the number that are performed. Defaults to 25. Enabling retries on the action will not trigger retries to run, it only sets the number of retries to be made. retry_on_status must be enabled.

  • retry_on_status: (Optional) Specify the array of status codes that should cause a retry. If the HTTP response received by the action has one of these codes, then it will be retried.

    • Each array element can be either a single status code (e.g. 400), or a range of status codes (e.g. 400-499). Ranges are inclusive of starting and ending values.

    • The retry schedule consists of 25 retries with exponential back-off plus random "jitter", starting at 5 seconds after the initial failure and gradually increasing to 10 minutes after the most recent failure, ie. [5, 10, 20, 40, 80, 160, 320, 600, 600, ...]. The jitter added is of random duration and up to (10 * (retry_count + 1)) seconds. Total back-off time over 25 retries is approx. 3h 20mins, [5 * (2**retry_count), 10 * 60].min + (rand(10) * (retry_count + 1)). If emit_failure_event is configured, a failure event will not be emitted for each unsuccessful retry. A single failure event will be emitted after all retries have been exhausted.

  • send_get_payload_as_body: (Optional) For HTTP GET requests, rather than sending the payload as URL params (e.g. https://example.com/search?parameter=value), send as the request body.

  • timeout: (Optional) Specify the timeout of the HTTP request in seconds. Defaults to 30 seconds.

  • use_tunnel: (Optional) If you have one or more Tines Tunnels configured, you can choose to send requests from this Action through a Tunnel.

  • rules: (Optional) Specify the rules to determine if this action executes.

  • must_match: (Optional) Specify the number of rules that must match for the action to execute.

Emitted Events​ 

Events emitted by the HTTP Request Action will include the 'body', 'headers' and response 'code' from the returned response. For example:

{
  "body": "ok",
  "headers": {
    "Date": "Mon, 1 Jan 2018 10:10:00 UTC",
    "Content-Type": "text/html; charset=utf-8",
    "Transfer-Encoding": "chunked",
    "Connection": "keep-alive",
    "Set-Cookie": "__cfduid=df0297dac2e4057e71e36fb67009723e91519037460; expires=Tue, 01-Jan-19 10:10:00 UTC; path=/; domain=.example.com; HttpOnly",
    "Via": "1.1 vegur",
    "Strict-Transport-Security": "max-age=15552000",
    "X-Content-Type-Options": "nosniff"
  },
  "status": 200,
  "meta": {
    "response_time": 0.08232
  }
}

Example Configuration Options​ 

The below samples use the postman-echo.com utility.

Send a simple GET request:

{
  "url": "https://postman-echo.com/get?foo1=bar1&foo2=bar2'",
  "method": "get"
}

Send a POST request with data from an incoming Event:

{
  "url": "https://postman-echo.com/post",
  "content_type": "json",
  "method": "post",
  "payload": {
    "user": "alice",
    "title": "<<person.title>>",
    "age": "85"
  },
  "headers": {}
}

Retry a request on 429 & 5xx errors, and log errors for other 4xx errors:

{
  "url": "https://postman-echo.com/post",
  "content_type": "json",
  "method": "post",
  "payload": {
    "user": "alice"
  },
  "retry_on_status": ["429", "500-599"],
  "log_error_on_status": ["400-428", "430-499"]
}

Send a request to a service that requires Basic authentication (password is accessed using the CREDENTIAL formulas key), include a custom header:

{
  "url": "https://postman-echo.com/basic-auth",
  "method": "get",
  "headers": {
    "X-Tines-Request": "123456"
  },
  "basic_auth": "postman:<<CREDENTIAL.postman>>"
}

Submit a file emitted as an attachment from an IMAP action to Virustotal using the data method:

{
  "url": "https://www.virustotal.com/vtapi/v2/file/scan",
  "content_type": "data",
  "method": "post",
  "payload": {
    "file": {
      "contents": "<<BASE64_DECODE(get_email_with_attachment.attachments[0].base64encodedcontents)>>",
      "filename": "<<get_email_with_attachment.attachments[0].filename>>"
    },
    "apikey": "<<CREDENTIAL.virustotal>>"
  },
  "headers": {}
}

Note, in the above example, the file key underneath payload can be called anything and is used as the form name argument in the body of the request e.g. image or document. The action will find any such object with the child keys contents and filename set.

Additional parameters that do not need to be included in the HTTP boundary framing should be included as query parameters in the URL instead of the payload of the request e.g. https://slack.com/api/files.upload?channels=<<channel_id>>&initial_comment=<<comment>>&filetype=<<filetype>>.

Egress IP addresses 

The egress IP is the IP address that external servers or services see as the source of an HTTP Request action's request. To see the egress IP addresses for your tenant, visit https://<tenant-domain>/info.

Scheduling 

A HTTP Request can be scheduled to run at a specific time, interval & timezone. This can be configured using the Scheduling menu as demonstrated below:

This can also be injected via the JSON payload for the Action using the schedule object. See example:

{
  "schedule":[{"cron":"0 8 * * *","timezone":"Europe/Dublin"}]
}

Rules 

Rules allow you to specify criteria that must be met in order for an event to be emitted. The Tines HTTP Request action can parse the event data from upstream action to meet these criteria. An event is created only when a rule is matched, and the matching rule is included in the meta object of the event. If no rule is matched, a log line is emitted.

In the following example, we are checking if a message from an upstream event transaction action matches a string.

{
  "path": "4d4217f7e94467ee3dec675189566f9a",
  "secret": "a409cd0d15a079ee69873aaa5b2e3d04",
  "verbs": "get,post",
  "must_match": 1,
  "rules": [
    {
      "type": "field==value",
      "value": "Error detected",
      "path": "<<event_transformation_action.message>>"
    },
  ]
}
▲  Demo
Was this helpful?