How to Remove Special Characters Using Routines in Datastage

To implement complex business logic, Datastage designer allows us to create custom defined routines to implement it. A parallel routine provides us features to use external functionality written in C/C++ code to use in Datastage. Even though Datastage has most of the essential functions available, routines are very helpful to create custom functions for a very specific logic. In this blog, I will show you the implementation of routines to remove special characters.

Prerequisites:

1. C++ code to remove special characters

  2.       C++ Compilers- eg GCC in UNIX

  3.       Change the settings in DS Administration.

InfoSphere Datastage Administration –> Properties –> General –> Environment –> Compiler –> APT_COMPILER/APT_COMPILEOPT

Steps to create Routines:

1.       Compile the C++ function in UNIX and create the object file. Make sure to compile using the same compile command which you have in the Datastage administrator for your project. Please note: we need to compile the code without MAIN() Function

              g++ -fpic -c spec_char_new.c

To create a shared object

         g++ -shared -o spec_char_new.so  spec_char_new.o

2.       Create a parallel routine in Datastage:

Choosing a Global Software Development Partner to Accelerate Your Digital Strategy

To be successful and outpace the competition, you need a software development partner that excels in exactly the type of digital projects you are now faced with accelerating, and in the most cost effective and optimized way possible.

Get the Guide

                  File>New>Routines>Parallel Routine

                  Fill all the required values as:

                  Routine Name:  Any name with just alphanumeric characters only.

                  External subroutine name: Name of the C function which we want to invoke

                  Type: External Function

                  Object Type: Library /Object

                  Return Type: Return type of the C function (For string function, specify as char*)

                  Library path: Library name with complete path

Usage:

Here I have used a Sequential file as input and output. We need a transformer stage to use a parallel routine to call the external function in job.

In the Transformer stage, Right Click > DS_Routines>< Routine Name> call the routine
E.g : specialchar(%InStr%)

Sample Input:

Output:

We can reuse the same routine anywhere in the project to remove special character by calling it in a transformer.