Calling a loop task to filter and construct final json result in Ansible

Sometimes you need to :

• iterate a payload
• call an API for each item
• filtering the json result using json_query
• returning a final result with all the filtered result

With this result you can iterate for another API.

This article talk about how to do that using Ansible, json_query filter

Firstly the structure 2 yaml files, the main.yml that call in a loop the tools/post_api.yml

main.yml

---
- hosts:
    localhost
  vars:
    input_payload:
      -
        uuid: uuid_1
        attribute1:
          - item1_attribute1_value1
          - item1_attribute1_value2
          - item1_attribute1_value3
        attribute2: item1_attribute2_value1
      -
        uuid: uuid_2
        attribute1:
          - item2_attribute1_value1
          - item2_attribute1_value2
          - item2_attribute1_value3
        attribute2: item2_attribute2_value1
      -
        uuid: uuid_3
        attribute1:
          - item3_attribute1_value1
          - item3_attribute1_value2
          - item3_attribute1_value3
        attribute2: item3_attribute2_value1
  
  tasks:

  - name: Call a task looping the payload
    include_tasks: tools/post_api.yml
    loop: "{{ input_payload }}"
    loop_control:
      loop_var: outer_item
    register: get_api_wrapper_result

  - name: Show the final_result
    debug:
      var: final_result

tools/post_api.yml

---

# Api call will return for first element
#
#{
#   "some_key": "some_value"
#   "callback_url_1": "https://item1_attribute2_value1.domain.com/callback_url/value1"
#   "callback_url_2": "https://item1_attribute2_value1.domain.com/callback_url/value2"
#   "callback_url_3": "https://item1_attribute2_value1.domain.com/callback_url/value3"
#   "another_key": "another_value"
# }

- name: Call an api for each item
  uri:
    url: "http://5rqgz.mocklab.io/json"
    method: POST
    headers:
      Accept: "application/json"
      Content-Type: "application/json"
    body: "{{ outer_item }}"
    body_format: json
    return_content: yes
    status_code: 200,201
  register: api_call_result

- debug:
    var: api_call_result.json

- name: Construct the final result
  set_fact:
    "final_result": "{{ final_result | default([]) | union([ api_call_result | json_query(jmesquery) ]) }}"
  vars:
    jmesquery: "{uuid: '{{ outer_item.uuid }}', first_url: json.callback_url_1, third_url: json.callback_url_3}"

Final result

{
    "final_result": [
        {
            "first_url": "https://item1_attribute2_value1.domain.com/callback_url/item1_attribute1_value1",
            "third_url": "https://item1_attribute2_value1.domain.com/callback_url/item1_attribute1_value3",
            "uuid": "uuid_1"
        },
        {
            "first_url": "https://item2_attribute2_value1.domain.com/callback_url/item2_attribute1_value1",
            "third_url": "https://item2_attribute2_value1.domain.com/callback_url/item2_attribute1_value3",
            "uuid": "uuid_2"
        },
        {
            "first_url": "https://item3_attribute2_value1.domain.com/callback_url/item3_attribute1_value1",
            "third_url": "https://item3_attribute2_value1.domain.com/callback_url/item3_attribute1_value3",
            "uuid": "uuid_3"
        }
    ]
}

Explanation

For each item in the payload it calls an API (I used https://app.mocklab.io/ to simulate an API) that will return a content depending of the input payload.

I need to use the result of all the loops to call another API but I don’t need all the content so I want to filter it and I want to identify each result with an uuid from the initial loop.

main.yml

• input_payload is very classic array payload with an uuid and key/values
• I call a tools task looping on the input_payload and using an outer_item if we need to loop in the tools task

tools/post_api.yml

• I call an API that return a json with values for the outer_item and register the result for the item in api_call_result

{
  "some_key": "some_value"
  "callback_url_1": "https://item1_attribute2_value1.domain.com/callback_url/value1"
  "callback_url_2": "https://item1_attribute2_value1.domain.com/callback_url/value2"
  "callback_url_3": "https://item1_attribute2_value1.domain.com/callback_url/value3"
  "another_key": "another_value"
}

 

• I use a set_fact to construct a variable that will contains the union of all the API call

• final_result | default([]) for first item I need to initiate the final_result as an empty array

• union([ api_call_result | json_query(jmesquery) ]) I make an union with previous value of final_result and a filtered api_call_result

• I filter the json using jmespath  setting  uuid with a variable from the outer_item and filtering/renaming 2 of the third url returned by the API

mesquery: "{uuid: '{{ outer_item.uuid }}', first_url: json.callback_url_1, third_url: json.callback_url_3}"

 

At the end in the main.yml file after calling my task post_api.yml I have a final_result that I can use to interact with another task/API/action.

I used this method to create multiple items from a big input_payload, the API used return urls to an Hashicorp Vault for each item, so I have to call again another API to retrieve the content of the Vault.