How to Remove Special Characters Using Routines in Datastage

To implement complex business logic, Datastage designer allows us to create custom defined routines to implement it. A parallel routine provides us features to use external functionality written in C/C++ code to use in Datastage. Even though Datastage has most of the essential functions available, routines are very helpful to create custom functions for a very specific logic. In this blog, I will show you the implementation of routines to remove special characters.

Prerequisites:

1. C++ code to remove special characters

  2.       C++ Compilers- eg GCC in UNIX

  3.       Change the settings in DS Administration.

InfoSphere Datastage Administration –> Properties –> General –> Environment –> Compiler –> APT_COMPILER/APT_COMPILEOPT

Steps to create Routines:

1.       Compile the C++ function in UNIX and create the object file. Make sure to compile using the same compile command which you have in the Datastage administrator for your project. Please note: we need to compile the code without MAIN() Function

              g++ -fpic -c spec_char_new.c

To create a shared object

         g++ -shared -o spec_char_new.so  spec_char_new.o

2.       Create a parallel routine in Datastage:

Revolutionize Your Business With Generative AI

From product design and software development to virtual agents, content creation, and reporting, GenAI is transforming business. Our AI experts help you unlock GenAI’s full potential and drive growth.

Let’s Get Started

                  File>New>Routines>Parallel Routine

                  Fill all the required values as:

                  Routine Name:  Any name with just alphanumeric characters only.

                  External subroutine name: Name of the C function which we want to invoke

                  Type: External Function

                  Object Type: Library /Object

                  Return Type: Return type of the C function (For string function, specify as char*)

                  Library path: Library name with complete path

Usage:

Here I have used a Sequential file as input and output. We need a transformer stage to use a parallel routine to call the external function in job.

In the Transformer stage, Right Click > DS_Routines>< Routine Name> call the routine
E.g : specialchar(%InStr%)

Sample Input:

Output:

We can reuse the same routine anywhere in the project to remove special character by calling it in a transformer.