{"id":214,"date":"2021-05-25T21:45:55","date_gmt":"2021-05-25T19:45:55","guid":{"rendered":"https:\/\/deleforterie.com\/wordpress\/?p=214"},"modified":"2023-01-18T08:53:11","modified_gmt":"2023-01-18T07:53:11","slug":"calling-a-loop-task-to-filter-and-construct-final-json-result-in-ansible","status":"publish","type":"post","link":"https:\/\/deleforterie.com\/wordpress\/index.php\/2021\/05\/25\/calling-a-loop-task-to-filter-and-construct-final-json-result-in-ansible\/","title":{"rendered":"Calling a loop task to filter and construct final json result in Ansible"},"content":{"rendered":"\n<p>Sometimes you need to :<\/p>\n<ul>\n<li>iterate a payload<\/li>\n<li>call an API for each item<\/li>\n<li>filtering the json result using json_query<\/li>\n<li>returning a final result with all the filtered result<\/li>\n<\/ul>\n<p>With this result you can iterate for another API.<\/p>\n<p>This article talk about how to do that using Ansible, json_query filter<!--more--><\/p>\n<p>Firstly the structure 2 yaml files, the <code>main.yml<\/code> that call in a loop the <code>tools\/post_api.yml<\/code><\/p>\n<p><strong>main.yml<\/strong><\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"YAML\">---\n- hosts:\n    localhost\n  vars:\n    input_payload:\n      -\n        uuid: uuid_1\n        attribute1:\n          - item1_attribute1_value1\n          - item1_attribute1_value2\n          - item1_attribute1_value3\n        attribute2: item1_attribute2_value1\n      -\n        uuid: uuid_2\n        attribute1:\n          - item2_attribute1_value1\n          - item2_attribute1_value2\n          - item2_attribute1_value3\n        attribute2: item2_attribute2_value1\n      -\n        uuid: uuid_3\n        attribute1:\n          - item3_attribute1_value1\n          - item3_attribute1_value2\n          - item3_attribute1_value3\n        attribute2: item3_attribute2_value1\n  \n  tasks:\n\n  - name: Call a task looping the payload\n    include_tasks: tools\/post_api.yml\n    loop: \"{{ input_payload }}\"\n    loop_control:\n      loop_var: outer_item\n    register: get_api_wrapper_result\n\n  - name: Show the final_result\n    debug:\n      var: final_result<\/pre>\n<p><strong>tools\/post_api.yml<\/strong><\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"YAML\">---\n\n# Api call will return for first element\n#\n#{\n#   \"some_key\": \"some_value\"\n#   \"callback_url_1\": \"https:\/\/item1_attribute2_value1.domain.com\/callback_url\/value1\"\n#   \"callback_url_2\": \"https:\/\/item1_attribute2_value1.domain.com\/callback_url\/value2\"\n#   \"callback_url_3\": \"https:\/\/item1_attribute2_value1.domain.com\/callback_url\/value3\"\n#   \"another_key\": \"another_value\"\n# }\n\n- name: Call an api for each item\n  uri:\n    url: \"http:\/\/5rqgz.mocklab.io\/json\"\n    method: POST\n    headers:\n      Accept: \"application\/json\"\n      Content-Type: \"application\/json\"\n    body: \"{{ outer_item }}\"\n    body_format: json\n    return_content: yes\n    status_code: 200,201\n  register: api_call_result\n\n- debug:\n    var: api_call_result.json\n\n- name: Construct the final result\n  set_fact:\n    \"final_result\": \"{{ final_result | default([]) | union([ api_call_result | json_query(jmesquery) ]) }}\"\n  vars:\n    jmesquery: \"{uuid: '{{ outer_item.uuid }}', first_url: json.callback_url_1, third_url: json.callback_url_3}\"\n<\/pre>\n<p><strong>Final result<\/strong><\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"JSON\">{\n    \"final_result\": [\n        {\n            \"first_url\": \"https:\/\/item1_attribute2_value1.domain.com\/callback_url\/item1_attribute1_value1\",\n            \"third_url\": \"https:\/\/item1_attribute2_value1.domain.com\/callback_url\/item1_attribute1_value3\",\n            \"uuid\": \"uuid_1\"\n        },\n        {\n            \"first_url\": \"https:\/\/item2_attribute2_value1.domain.com\/callback_url\/item2_attribute1_value1\",\n            \"third_url\": \"https:\/\/item2_attribute2_value1.domain.com\/callback_url\/item2_attribute1_value3\",\n            \"uuid\": \"uuid_2\"\n        },\n        {\n            \"first_url\": \"https:\/\/item3_attribute2_value1.domain.com\/callback_url\/item3_attribute1_value1\",\n            \"third_url\": \"https:\/\/item3_attribute2_value1.domain.com\/callback_url\/item3_attribute1_value3\",\n            \"uuid\": \"uuid_3\"\n        }\n    ]\n}<\/pre>\n<p><strong>Explanation<\/strong><\/p>\n<p>For each item in the payload it calls an API (I used <a href=\"https:\/\/app.mocklab.io\/\">https:\/\/app.mocklab.io\/<\/a> to simulate an API) that will return a content depending of the input payload.<\/p>\n<p>I need to use the result of all the loops to call another API but I don&#8217;t need all the content so I want to filter it and I want to identify each result with an uuid from the initial loop.<\/p>\n<p><strong>main.yml<\/strong><\/p>\n<ul>\n<li>input_payload is very classic array payload with an uuid and key\/values<\/li>\n<li>I call a tools task looping on the input_payload and using an outer_item if we need to loop in the tools task<\/li>\n<\/ul>\n<p><strong>tools\/post_api.yml<\/strong><\/p>\n<ul>\n<li>I call an API that return a json with values for the outer_item and register the result for the item in <code>api_call_result<\/code><\/li>\n<\/ul>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"JSON\">{\n  \"some_key\": \"some_value\"\n  \"callback_url_1\": \"https:\/\/item1_attribute2_value1.domain.com\/callback_url\/value1\"\n  \"callback_url_2\": \"https:\/\/item1_attribute2_value1.domain.com\/callback_url\/value2\"\n  \"callback_url_3\": \"https:\/\/item1_attribute2_value1.domain.com\/callback_url\/value3\"\n  \"another_key\": \"another_value\"\n}<\/pre>\n<p>\u00a0<\/p>\n<ul>\n<li>I use a set_fact to construct a variable that will contains the union of all the API call<\/li>\n<li>\n<p><code>final_result | default([])<\/code> for first item I need to initiate the <code>final_result<\/code> as an empty array<\/p>\n<\/li>\n<li>\n<p><code>union([ api_call_result | json_query(jmesquery) ])<\/code> I make an union with previous value of <code>final_result<\/code> and a filtered <code>api_call_result<\/code><\/p>\n<\/li>\n<li>I filter the json using <a href=\"https:\/\/jmespath.org\/\">jmespath<\/a>\u00a0 setting\u00a0 <code>uuid<\/code> with a variable from the <code>outer_item<\/code> and filtering\/renaming 2 of the third url returned by the API<\/li>\n<\/ul>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"YAML\">mesquery: \"{uuid: '{{ outer_item.uuid }}', first_url: json.callback_url_1, third_url: json.callback_url_3}\"<\/pre>\n<p>\u00a0<\/p>\n<p>At the end in the main.yml file after calling my task post_api.yml I have a final_result that I can use to interact with another task\/API\/action.<\/p>\n<p>I used this method to create multiple items from a big input_payload, the API used return urls to an Hashicorp Vault for each item, so I have to call again another API to retrieve the content of the Vault.<\/p>\n<p>\u00a0<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Sometimes you need to : iterate a payload call an API for each item filtering the json result using json_query returning a final result with all the filtered result With this result you can iterate for another API. This article talk about how to do that using Ansible, json_query filter<\/p>\n","protected":false},"author":2,"featured_media":155,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"footnotes":""},"categories":[6],"tags":[26,28,27,29,30,31],"class_list":["post-214","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ansible","tag-ansible","tag-jmespath","tag-json","tag-json_query","tag-loop","tag-union"],"_links":{"self":[{"href":"https:\/\/deleforterie.com\/wordpress\/index.php\/wp-json\/wp\/v2\/posts\/214","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/deleforterie.com\/wordpress\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/deleforterie.com\/wordpress\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/deleforterie.com\/wordpress\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/deleforterie.com\/wordpress\/index.php\/wp-json\/wp\/v2\/comments?post=214"}],"version-history":[{"count":8,"href":"https:\/\/deleforterie.com\/wordpress\/index.php\/wp-json\/wp\/v2\/posts\/214\/revisions"}],"predecessor-version":[{"id":222,"href":"https:\/\/deleforterie.com\/wordpress\/index.php\/wp-json\/wp\/v2\/posts\/214\/revisions\/222"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/deleforterie.com\/wordpress\/index.php\/wp-json\/wp\/v2\/media\/155"}],"wp:attachment":[{"href":"https:\/\/deleforterie.com\/wordpress\/index.php\/wp-json\/wp\/v2\/media?parent=214"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/deleforterie.com\/wordpress\/index.php\/wp-json\/wp\/v2\/categories?post=214"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/deleforterie.com\/wordpress\/index.php\/wp-json\/wp\/v2\/tags?post=214"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}