The project was to upgrade and maintain the catalog system from PIM to Sitecore:
Earlier catalog data, like category, product, and SKUs were coming to the commerce system from the PIM server.
It’s a system that was running for quite some time under a different vendor with code that we inherited. Things are the way they are because the data is in a certain format that cannot be changed, so we need to work around it to get it to what we want
Infrastructure to accomplish the catalog system upgrade from PIM to Sitecore:
To achieve the upgradation of the catalog system, the following architecture is in place:
PIM server->Sitecore server->Integration server->Commerce server->commerce database persistency.
Old catalog data is transferred from the PIM server to the Sitecore server.
Sitecore server: Publish the new category, attributes, products, and SKU data and send it to the integration server.
Integration server: Receives the catalog data from the Sitecore server, formatting this data according to commerce provided request, and sends it to commerce server over MQ in XML format.
Commerce Server: Accepts catalog data from the integration server. It processes, transforms, and stores the information into the WebSphere Commerce database.
Below are some issues and resolutions we did:
- The client was sending attribute group feed for category separately, due to the one extra MQ message that was required. Hence, we told the club attribute group along with category structure feed only. Due to this, we were able to save one extra MQ message and no separate code was required for attribute group, parsing, processing and saving into the database.
- We are having some attributes for products and items display some information about them on site like brand, volume, capacity, etc. We faced the issue with these attributes for products and items. Sitecore was sending the attribute details for product only not for items, but our old code for frontend was expecting the attributes at item level also. Therefore, we told to send the attributes at product and item level both. So, we were able to associate them appropriately.
- As we discussed in point 2 we have to show some description of the product and items, for that, we are receiving some attributes and their values in product feed from Sitecore.
For each attribute, there is a value identifier associate with it in the commerce database, which is unique in nature. In the commerce database against that value identifier, we are storing value for the attribute, which we get from Sitecore feed.
The issue was, Sitecore was not sending a value identifier for attributes, that is why uniqueness was not maintained in the database, and whenever the feed was coming, in database new value identifier was created. It was an overload to the database. Then we asked the Sitecore integration team to send a value identifier to us.
For that, we asked them to make the value as a value identifier and send it to us, so that the uniqueness of value maintains in a database. Value identifier field length in the database is limited to 256 characters, that is why we asked them to trim value to 250 characters and make it as value identifier.
Example snippet,
<td>
<id>ART_16</id>
<name>Generic Name Products</name>
<unit></unit>
<values>
<value id=”BLOWER | BLOWER, HAND HELD“>BLOWER | BLOWER, HAND HELD</value>
</values>
</td>
Bold text is value identifier.
- In the old PIM system, if the client wanted to delete some products or items, they were not sending them in the next feed. After transforming the new feed, commerce was comparing it with existing products and items in the commerce database. If the new feed was not having those products and items, they were deleted from the commerce database. But this process was error-prone. During Sitecore integration, we told the client to send a flag at the product and item level which they wanted to delete. So, we can delete those products and items from the sales catalog. They are now sending the below flags value in Sitecore feed at the product level and item level respectively,
<productNeedToDelete>1</productNeedToDelete>
<itemNeedToDelete>1</itemNeedToDelete>
- Issues were related to images for products and items. Sitecore was sending images at the item level, not at the product level. They wanted to use item images as product images. The problem was, our old code, which we inherited from the previous vendor, was pulling images for the product separately and item separately. This meant, we asked the Sitecore integration team to send images for products and items both.
This helped us later also when they wanted different images for products and items.
