In this blog I wanted to take some time to describe certain “behaviors” of Cognos TM1 TurboIntegrator processes.
Access to Processes
As of version 10.2 (or as of this writing), TM1 Server lists all processes (in alphabetical order) under the consolidation “Processes”. The visibility of processes can be controlled by implementing TM1 security. TM1 groups can be set to have READ, WRITE or NONE/BLANK access to individual processes. Security must be set for each group and for each process individually. Process security (by group) is loaded and maintained in the TM1 control cube “}ProcessSecurity”.
Groups with NONE/BLANK access will not have visibility to the process.
Groups with READ access will have visibility and have the ability to right-click on the process and select Run. This executes the process.
Groups with WRITE access will have visibility and have the ability to either:
- Right-click on the process and select Run (to execute the process) or
- Double-click on the process and edit/run the process. Double-clicking on a process enters the client into “read-time” (or edit mode) for that process.
- All TM1 Administrators have (by default) WRITE access to all processes giving them full edit and execution ability for all processes.
Processes and Chore Execution
As a (best practice) rule, all processes that exist within a production environment should be executed as part of a TM1 Chore.
Access to Chores follows the same tactic as processes: Groups are assigned READ, WRITE or NONE/BLANK access. Chore access is loaded and maintained in the TM1 control cube “}ChoreSecurity”.
Groups with NONE/BLANK access will not have visibility to the chore.
Groups with READ access will have visibility and have the ability to right-click on the chore and select Run. This executes the chore.
Groups with WRITE access will have visibility and have the ability to either:
- Right-click on the chore and select Run (to execute the chore) or
- Double-click on the chore and edit/run the chore (using the Chore Setup Wizard).
- All TM1 Administrators have (by default) WRITE access to all chores giving them full edit and execution ability for all chores.
Individual Processes
At times, it may be necessary to execute (run) individual processes without executing them “through” an associated TM1 chore. To do this it is recommended to use the “select and right-click” method (rather than double-clicking and editing to run the process). The following are the correct steps:
- From TM1 Server explorer, expand “Processes”.
- Locate the process to be executed and select/highlight it (by (single) clicking on it).
- Right-click on the process name and select Run (see below):
If the selected process includes run-time parameters, TM1 will prompt you for the required values by displaying the “Parameters dialog” (shown below). Multiple parameters will be listed sequentially/vertically on the same dialog.
Typically, the prompt will display a Prompt Question (text) describing each parameter and space for you to enter a value for that parameter (labeled Default Value). Certain processes may have included a default value for certain (or all) parameters and if so, those values will appear in the Default Value space. Default values may be changed if desired by typing over the value shown.
Processing and Data
TM1 TurboIntegrator processes include a “Data Source” tab where the Datasource Type option is set (ODBC, Text, ODBO, IBM Cognos TM1, SAP, IBM Cognos Package or None). When any option other than “None” is selected, the idea is that the TM1 process will attempt to “connect” to the specified datasource and then read that data. If the data source is inaccessible for any reason (does not exist or is restricted by security) a “compilation” error will occur.
Run-Time or Read-Time
When working with TurboIntegrator processes, you need to understand the concept of run-time vs. read-time.
Run-time is defined as the time when TM1 is running (executing) a process. During run time the client session (clients may in some environments be setup with the ability to have multiple sessions active) that initiated (the running of) the process is “locked” by TM1 server until that process completes. During run-time, processing or run-time errors may occur. Best practice recommends that scripting logic within the process be in place do prevent the occurrence of the most common run-time errors (and would perform the appropriate steps for remediating the errors) and that TM1 message logging be appropriately leveraged.
TM1 writes run-time errors to a log file. The number of minor errors that will be written to the TM1ProcessError.log file during process execution is by default set to 1000 (this value can be changed for an individual process).
When a process error log file is generated, TM1 assigns a unique name that lets you readily identify which TurboIntegrator process generated the error file and the time at which the file was created. File names are assigned using the convention TM1ProcessError_time stamp_process name.log. In this convention, time stamp is the time (expressed as yyyymmddhhmmss GMT) at which the file was generated and process name is the name of the TurboIntegrator process that caused the errors.
There may be multiple TM1ProcessError.log files associated with the server session that crashed. All TM1ProcessError.log files are stored in the server data directory.
Read–time is defined as the time when a TM1 client who has WRITE access to a process is “reading” the scripting within that process. In other words, a client has double-clicked on a selected process name and has the process “open” on the desktop. The intention of the client may be:
- Perform a review of the scripting within the process
- Modify the scripting or option settings within the process
- Run the process (although best practice recommends that processes NOT be run/executed from within edit or “ready-time” mode)
If changes are made to a process during read-time, those changes are not automatically (or auto) saved. To save the changes the following must occur:
- The client clicks the Save icon or clicks File, then Save or Save As on the TurboIntegrator process menu
- The client closes the TurboIntegrator process. TM1 will prompt the client with the “Save Changes to Process Definition?” dialog. The clicks “Yes”.
- The client clicks the Run icon or the File then Run menu. Before TM1 runs a process it saves the latest version of the process (which would include the changes). Note: When TM1 saves a process it performs a syntax check (or compilation) of the settings/options of the process as well as the scripting in the process. If there are errors, TM1 will prompt the client and not run the process.
Compilation Errors
During read-time, compilation errors may occur. Compilation errors are defined as “errors generated by TM1 based upon its perceived inability to execute a process to completion successful”. These errors are generated when TM1 compiles (checks for syntax errors in the process scripting or settings/options of the process) a process.
TM1 performs a compilation of the process when:
- A client attempts to save the process
- A client attempts to run a process
- A client begins editing or enters read-time for a process (double-clicks on the process name and “opens” the process as described above).
The most common compilation error encountered during read-time of a TurboIntegrator process is the “cannot find” error. When the datasource type option (described above) is set to any option other than NONE, TM1 will attempt to verify the datasource named in the Datasource Name field on the Datasource tab of the process upon opening the process. These means that if you have set the value to an ODBC datasource TM1 will attempt to connect to that datasource named, or if the setting is Text, TM1 will validate the fully qualified filename, etc. if TM1 cannot connect to the source, an error will occur. This error can most always be considered a warning as best practice recommends that specific datasource values be set within the script and therefore validated during execution of the Prolog in the process. Scripting should include logic to verify the datasource and perform remediation actions if the datasource cannot be verified. Typically, “Cannot find” errors that occur upon opening a process are meaningless.
An Example
A large percentage of processes will work with data already in TM1. Therefore the Datasource Type would be set to IBM Cognos TM1 and the Data Source Name field will then show the explicit cube view or dimension subset to be read.
In this example, a process uses a specific view of the cube named Forecasting. The view is specific to the requirements of the process in that it filters the data in the cube view by certain parameters (for example by version, time or region). During development of the process, the developer started with a temporary view named: “Forecasting All” which included all data for the version named FORECAST in the current time period. At run-time the process filters the view by region (provided by a process parameter). Using this view the developer was able to preview a sample of the cube data and set all of the Variable Tab parameters easily. Once these values were set, script was created in the Prolog section of the process, based upon best practice recommendations, to verify and filter the view as required. This script in fact checks for the existence of a view and if it does not exist, creates it. If the view does exist, it filters the view (using subsets) to meet the runtime requirements of the process.
Note: after the first time the process is run, the required view will exist.
Once the process is created, compiled and tested, the developer would delete/remove the temporary view from the FORECAST cube (as that view is no longer needed). Since the process (via scripting) creates and/or modifies its own view. Once the temporary view is removed, each time the TI process is opened for edit, a “Cannot find” warning would occur (shown below).
To resolve this warning/error, you can simply click OK and proceed to either review and/or edit the process or execution/run it. You may be tempted to resolve this issue by changing the value in the Datasource Name field to a view name that will always exist within TM1; however views can be modified or removed in a variety of ways including other processes, manually by TM1 clients with appropriate access, or unanticipated events. Best practice recommends processes use scripting to set datasource options at runtime to ensure successful execution of the process.
Cheers!
