Processing JSON with Mule Transformers

To see more on Mule Json data processing, please follow this newer blog post: https://blogs.perficient.com/integrate/2018/03/02/json-data-processing-mule-transformers-dataweave/

When a JSON object comes into Mule application, it’s very common it’s represented as a JSON formatted string. For example, when it’s part of HTTP post string or part of the message received from an Amazon SQS queue.

In order to process the JSON formatted string effectively, we need to convert the it into some kind of object other than Java String. A natural candidate is map or array of maps depends on the JSON data string. It may appear to be a trivial task since Mule comes with the “Object to JSON Transformer.” However, a test will quickly prove this transformer doesn’t have the magic power to convert a JSON string into anything. The result is still a string, not directly parse-able by Mule. What ended up working for me is the Dataweave with some preparations beforehand.

I’ll walk through a full example to show how it is done and what are the various nuances make the transformation work (or not work).

The example has 3 sub flows. At the beginning of each flow, we set the payload with some JSON string like: {“foo”: “111”, “bar”: “222”}. Then we try to convert the string into map and parse it.

We’ll have a high level description of each sub flow. Then we’ll analyze the logger output.

The first sub flow is “no-obj2json-no-mime”. This one works. After setting the JSON string, “Object to JSON” is called before DW. The data is successfully parsed.

Choosing a Global Software Development Partner to Accelerate Your Digital Strategy

To be successful and outpace the competition, you need a software development partner that excels in exactly the type of digital projects you are now faced with accelerating, and in the most cost effective and optimized way possible.

Get the Guide

The 2nd sub flow is “no-obj2son-with-mine-settings”. This one works too. However, as the flow name indicates, there is no “Object to JSON” transformer, but there is “application/json” mine setting on the “set payload” processor.

Finally, “no-obj2json-no-mine”. This one doesn’t work. The result after DW is not parse-able. (change the mine setting on “set payload” like in 2nd flow, re-run the program, you will see this flow works as well).

Here is the logger output:

with obj2json xformer: orginal msg type=SimpleDataType{type=java.lang.String, mimeType='*/*', encoding='null'}
with obj2json xformer: aft obj2json msg type=SimpleDataType{type=java.lang.String, mimeType='application/json', encoding='UTF-8'}
with obj2json xformer: aft obj2json, dw2java, msg type=SimpleDataType{type=java.util.LinkedHashMap, mimeType='*/*', encoding='null'}
with obj2json xformer: table name=xxx, key=[yyy]

with-mine-settings: orginal msg type=SimpleDataType{type=java.lang.String, mimeType='application/json', encoding='null'}
with-mine-settings:  dw2java, msg type=SimpleDataType{type=java.util.LinkedHashMap, mimeType='*/*', encoding='null'}
with-mine-settings: table name=xxx, key=[yyy]

no-obj2json-nomine, orginal msg type=SimpleDataType{type=java.lang.String, mimeType='*/*', encoding='null'}
***: MimeType was not resolved '*/*' delegating to Java.
no-obj2json-nomine, , aft obj2json, dw2java, msg type=SimpleDataType{type=java.lang.String, mimeType='*/*', encoding='null'}
no-obj2json-nomine, foo=null, bar=null

In the first flow, the payload type initially is a string with mineType “*/*”. After “Object to JS ON”, it is still a string, but mineType=”application/json”. After DW, the data type become a map. Finally, it shows the json is parsed successfully.

Armed with this “mineType” discovery, I created the 2nd flow “no-obj2json-with-mine-setting”. Here I skipped the “Object to Json” transformer, but I set mineType on the “set payload” operation. As the result shows, this works as well.

The last flow “no-obj2json-no-mine” doesn’t have “Object to json”, and it doesn’t set mineType either. It doesn’t work.

Here is my theory, in order for Dataweave to transform JSON formatted string, we need to inform DW the input is “application/json”. I vaguely remember if you set the DW input metadata as JSON, it will work as well. That’s something you can test on your own. Have fun!

Here is the full source code:

<?xml version=”1.0″ encoding=”UTF-8″?>

<mule xmlns:dw=”http://www.mulesoft.org/schema/mule/ee/dw” xmlns:json=”http://www.mulesoft.org/schema/mule/json” xmlns:http=”http://www.mulesoft.org/schema/mule/http” xmlns=”http://www.mulesoft.org/schema/mule/core” xmlns:doc=”http://www.mulesoft.org/schema/mule/documentation”
xmlns:spring=”http://www.springframework.org/schema/beans”
xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance”
xsi:schemaLocation=”http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/json http://www.mulesoft.org/schema/mule/json/current/mule-json.xsd
http://www.mulesoft.org/schema/mule/ee/dw http://www.mulesoft.org/schema/mule/ee/dw/current/dw.xsd”>
<http:listener-config name=”HTTP_Listener_Configuration” host=”0.0.0.0″ port=”8081″ doc:name=”HTTP Listener Configuration”/>
<sub-flow name=”with-obj2json-xformer”>
<set-payload value=”{&quot;tbl_name&quot;: &quot;xxx&quot;, &quot;key&quot;: [&quot;yyy&quot;], &quot;rec&quot;: {&quot;foo&quot;: &quot;111&quot;, &quot;bar&quot;: &quot;222&quot;} }” doc:name=”Set Payload”/>
<logger message=”with obj2json xformer: orginal msg type=#[message.dataType]” level=”INFO” doc:name=”msg.dataType”/>
<json:object-to-json-transformer doc:name=”Object to JSON”/>
<logger message=”with obj2json xformer: aft obj2json msg type=#[message.dataType]” level=”INFO” doc:name=”msg.dataType1″/>
<dw:transform-message doc:name=”DW”>
<dw:set-payload><![CDATA[%dw 1.0
%output application/java
—
payload]]></dw:set-payload>
</dw:transform-message>
<logger message=”with obj2json xformer: aft obj2json, dw2java, msg type=#[message.dataType]” level=”INFO” doc:name=”msg.dataType2″/>
<logger message=”with obj2json xformer: table name=#[payload.tbl_name], key=#[payload.key]” level=”INFO” doc:name=”parse-1″/>
</sub-flow>
<sub-flow name=”no-obj2json-with-mine-setting”>
<set-payload value=”{&quot;tbl_name&quot;: &quot;xxx&quot;, &quot;key&quot;: [&quot;yyy&quot;], &quot;rec&quot;: {&quot;foo&quot;: &quot;111&quot;, &quot;bar&quot;: &quot;222&quot;} }” mimeType=”application/json” doc:name=”set payload”/>
<logger message=”with-mine-settings: orginal msg type=#[message.dataType]” level=”INFO” doc:name=”msg.dataType”/>
<dw:transform-message doc:name=”DW”>
<dw:set-payload><![CDATA[%dw 1.0
%output application/java
—
payload]]></dw:set-payload>
</dw:transform-message>
<logger message=”with-mine-settings: dw2java, msg type=#[message.dataType]” level=”INFO” doc:name=”msg.dataType1″/>
<logger message=”with-mine-settings: table name=#[payload.tbl_name], key=#[payload.key]” level=”INFO” doc:name=”parse-2″/>
</sub-flow>
<sub-flow name=”no-obj2json-no-mime”>
<set-payload value=”{&quot;foo&quot;: &quot;111&quot;, &quot;bar&quot;: &quot;222&quot;}” doc:name=”Set Payload”/>
<logger message=”no-obj2json-nomine, orginal msg type=#[message.dataType]” level=”INFO” doc:name=”msg.dataType”/>
<dw:transform-message doc:name=”DW”>
<dw:set-payload><![CDATA[%dw 1.0
%output application/java
—
payload]]></dw:set-payload>
</dw:transform-message>
<logger message=”no-obj2json-nomine, , aft obj2json, dw2java, msg type=#[message.dataType]” level=”INFO” doc:name=”msg.dataType1″/>
<logger message=”no-obj2json-nomine, foo=#[payload.foo], bar=#[payload.bar]” level=”INFO” doc:name=”parse-3″/>
</sub-flow>
<flow name=”jsonFlow”>
<http:listener config-ref=”HTTP_Listener_Configuration” path=”/json” doc:name=”HTTP”/>
<flow-ref name=”with-obj2json-xformer” doc:name=”with-obj2json-xformer”/>
<flow-ref name=”no-obj2json-with-mine-setting” doc:name=”no-obj2json-with-mime-setting”/>
<flow-ref name=”no-obj2json-no-mime” doc:name=”no-obj2json-no-mine”/>
</flow>
</mule>