At Perficient, our SupportNet team uses a variety of tools for analyzing logs across the Oracle applications. There are a number of different file formats, which often makes parsing logs troublesome. We have adopted use of regular expressions to help parse the variety of formats that we have to routinely analyze.
“A regular expression (regex or regexp for short) is a special text string for describing a search pattern. You can think of regular expressions as wildcards on steroids.”1
LogMX is a commercial software package from LightlySoft which provides a nifty interface for parsing log files. The tool utilizes regular expressions for parsing logs so you can essentially parse any text log provided you spend the time necessary developing the format for the regular expression.
The tool is very handy for parsing the various Oracle Diagnostic Log (ODL) formatted logs as well as application logs from JVMs running under WebLogic. When a file is opened in LogMX, it is quickly analyzed to determine if any of your defined parsers (named regex) match the format of the file. When if finds a parser, it very quickly parses the file into columns and rows in accordance with the regular expression. For example, the regular expression “\[(.*?)\](.*?)/(.*?)/(.*?)/(.*?)/(.*?)/(.*?)\((.*?)\)\n(.*?)” will provide a parsed Essbase application log into this:
Guide to Oracle Cloud: 5 Steps to Ensure a Successful Move to the Cloud
Explore key considerations, integrating the cloud with legacy applications and challenges of current cloud implementations.
Row background colors are defined based on the severity level of the error. From here, I can right-click any cell and filter the result set to all rows with the same or with different values. For example, I can filter on the MessageID and display all calc script completion messages:
The message level is indexed so the tool can very quickly prune the display to only a specific severity of message, i.e. just a severity of ERROR.
Filtering is very strong and custom filters can be saved. I keep a white list of MessageID values that I know are not bad and add a lot of junk to the log for me to analyze. Instead of scrolling past these errors, I filter them out and am left with messages I am either unfamiliar with or know to be bad. This filtering saves us weeks of effort over the period of a year.
LogMX is a slick tool for postmortem analysis and even for proactive analysis of logs. I also consider it a “gateway” tool to more robust enterprise log monitoring systems such as SumoLogic and Splunk. What you learn about regular expressions in LogMX can easily be applied to many of the enterprise grade log monitoring systems.
- “Welcome to Regular-Expressions.info.” Regular-Expressions.info, http://www.regular-expressions.info, Accessed 3 May 2017.
