In data processing, two of the most common collection data types are array (list) and map (HashMap). The key difference between a list and a map is how they are accessed. A list is accessed by an integer positional index, such as list. However, map is accessed by a key, such as map.getValue(“key1”) or simply “map.key1”, syntax may vary based on the context.
This post discusses the use case where a Mule flow needs to cross-reference a pair of related array and a map. We’ll use Dataweave as the example to demonstrate the use case.
The subtle difference between two similar Json data sets
Because this post uses Json format to describe the sample data, let’s clarify some subtle differences between map and list in Json format.
In the following examples, both json data sets describe the same collection of courses with ID and name:
{“course1″:”Introduction to Mule”, “course2″:”Advanced Mule”}
[{“course1″:”Introduction to Mule”}, {“course2″:”Advanced Mule”}]
However, if they are directly converted to java object inside a Mule flow, the first collection should be directly translated as a java map. A pseudo access syntax will be like courseMap[‘course1’], or courseMap.course1. The 2nd one should be converted to a java List (of Map). The pseudo access syntax would be like courseList[0][‘course1’], or courseList[0].course1.
Now, let’s define the hypothetical problem.
Definition of the hypothetical problem
We have two input json data sets #1 and #2. We need to generate the output structure #3 in Json.
#1-a collection of course IDs and names represented as a map:
{“course1″:”Introduction to Mule”, “course2″:”Advanced Mule”}
#2 – a raw grade report (list) for 2018, it looks like:
[{“year”:”2018″, “studentName”: “john”, “cid”: “course1”, “grade”: “A”},
{“year”:”2018″, “studentName”: “john”, “cid”: “course2”, “grade”: “B”},
{“year”:”2018″, “studentName”: “joe”, “cid”: “course1”, “grade”:”C”}]
#3 – We want to generate a new grade report with course names and some header info like:
{“reportDate”: “2018-02-15”,
“schoolYear”: “2018”,
“schoolName”: “This is a Mule school”,
“grades”: [
{
“student”: “john”,
“course”: “Introuction to Mule”,
“grade”: “A”
},
{
“student”: “john”,
“course”: “Advanced Mule”,
“grade”: “B”
},
{
“student”: “joe”,
“course”: “Introuction to Mule”,
“grade”: “C”
}
]
}
Dataweave script explained
The full source code can be found at the end of this post. Let’s analyze the meat part of the project first – the Dataweave script.
Assume we store #1 in a flow variable “courseMap”, put #2 the grade list in the payload, also add flow variable “schoolNameVar”, here is the DW script will do the trick.
Notes for the script:
Line 4 and line 15 – indicate the output will be a single element, not an array
Line 5 – grab the current time and convert it to date string
Line 6 – if the array shares a common value in each element, you can reference the 1st in the output header
Line 7 – how to access flowVars
Line 8 to 14 – the output contains an array called “grades”
Line 9 – map is a lambda function that takes an input array (payload in this case), “map” has two implied arguments “$” and “$$”. So “map” really implies “map($, $$)”, where $ represent each element in the input array, $$ is index of the element inside the array.
You can add explicit arguments like “map(oneGradeRec, idx)”, then line 11, 12 will be like:
Course: flowVars.courseMap[oneGradeRec.cid],
Grade: oneGradeRec.grade
Complete source code:
In this implementation, we dynamically create a HashMap, then we set payload a Json array as text string, and use “object to json” to convert payload to JsonData before invoke DW.
Please keep your eyes on the two high-level data structures while reading the source code:
- we first created a flow variable “courseMap”. It is a LinkedHashMap java object (I believe a HashMap would work just fine for this case). We initialized the variable with the map values
- then we set payload as a json string, it is logically is an array, internally represented as a JsonData object after “json-to-object” conversion.
<flow name=“mainFlow”>
<http:listener config-ref=“HTTP_Listener_Configuration” path=“/proxy” doc:name=“HTTP”/>
<set-variable variableName=“courseMap” value=“#[new java.util.LinkedHashMap()]” doc:name=“courseMap”/>
<set-payload value=“#[((java.util.HashMap)courseMap).put(“course1”, “Introuction to Mule”)]” doc:name=“c1”/>
<set-payload value=“#[((java.util.HashMap)courseMap).put(“course2”, “Advanced Mule”)]” doc:name=“c2”/>
<logger message=“c2=#[courseMap[“course2”]] or c2=#[courseMap.course2]” level=“INFO” doc:name=“show map, #[courseMap[“course2”]] or #[courseMap.course2]”/>
<set-payload value=“[{“year”:”2018”, “studentName”: “john”, “cid”: “course1”, “grade”: “A”},
{“year”:”2018”, “studentName”: “john”, “cid”: “course2”, “grade”: “B”},
{“year”:”2018”, “studentName”: “joe”, “cid”: “course1”, “grade”:”C”}]” doc:name=“Set DB Grade Records as Json string”/>
<json:json-to-object-transformer doc:name=“JSON to Object”/>
<logger level=“INFO” doc:name=“by default, converts to generate org.mule.module.json.JsonData “/>
<set-variable variableName=“schoolNameVar” value=“This is a Mule school” doc:name=“schoolNameVar”/>
<dw:transform-message doc:name=“Transform Message” metadata:id=“fecb7fe3-7b5f-4fba-bc3a-da5f30d3bb6e”>
<dw:set-payload><![CDATA[%dw 1.0
%output application/json
—
{
reportDate: now as :string {format: “yyyy-MM-dd”},
schoolYear: payload[0].year,
schoolName: flowVars.schoolNameVar,
grades:
payload map ({
student: $.studentName,
course: flowVars.courseMap[$.cid],
grade: $.grade
})
}
]]></dw:set-payload>
</dw:transform-message>
<logger message=“json input is converted by DW as hashMap=#[payload]” level=“INFO” doc:name=“Use DW, json input is converted as hashMap”/>
</flow>
