EventCombMT

EventCombMT is a multi-threaded tool that will parse event logs from many servers at the same time.

It allows you to specify any or all of these criteria:

  • Individual Event IDs
  • Multiple Event IDs
  • A range of Event IDs
  • An Event Source
  • Event Text
  • A date range that events must match

Features:

  • Multithreaded for very fast performance! Every server gets its own thread.
  • SIDs are cached for faster searching of security logs.
  • Flexible Searching parameters
  • 6 ways to add servers to search
  • Built in Searches (Account Lockout and Several FRS searches)
  • Adjustable Thread count to control bandwidth
  • DLL Caching
  • Save and load searches
  • Save results to a database for more refined searches or log merging.

Potential Uses:

  • Find the last time all your servers rebooted
  • Track down account lockouts
  • Troubleshoot FRS issues
  • Determine DC Health (look for problems before they become serious)


Domain: Text box that lists the domain for your user account, as well as the domain from which list of domain controllers will be populated.

Servers/Logs: Select the servers/logs you would like to search.

Right click in this window do add servers.

Use File\Open Saved Logs to add Saved Event Logs to this list.

Output Directory: Default is c:\temp and it will be created if it doesn’t exist. Choose Set Output directory from the File menu to change this. This setting should be remembered between sessions.

Event IDs: Enter the specific IDs you wish to search for. Enter IDs separated by spaces, up to 10 events may be entered.

>=ID=<: Use these 2 boxes to enter a range of Event IDs to search for. The range is inclusive.

Source: The source is pulled from the registry. It is only available if exactly one log is selected.

Text: Enter the text that must be in the event to capture it. Do not use logic (and, or), or quotes. It is case insensitive.

Status: Lists information pertaining to the current search. This information is also saved to the file EventCombMT.txt file.

Choose Log File to Search:

Allows you to select which log files to search.

Available logs files are:

    • Directory Service
    • Application
    • File Replication Service
    • System
    • DNS
    • Security

NOTE: Logs will be disabled if you do not have the capability to open them normally (Event Viewer.) You may be able to edit the registry to enable parsing of disabled logs.

Event Types: Allows you to choose which types of events you would like to search.

You can choose from:

Error

Informational

Warning

Success Audit

Failure Audit

Success (very rare. Event Viewer shows these as Informational.)

Get All Events Matching Above Criteria: Checking this box short-circuits any other search criteria. For instance, if you check Warning and Application, you will capture all Warning events in the Application log. This will also disable all search criteria since it will not be used. This applies even if data was previously enter in the criteria boxes and is visible. If the criteria controls are disabled they will not count towards the search.

Threads: Use this slider to control how many threads are running. It is active even when a search is running.

Default: 25 threads

NOTE: If there are 50 threads running and you slide it to 40, there will still be 50 threads running, it doesn’t pause or kill running threads. However, no new threads will be spun until the count drops to 39.

Threads Running: Displays a list of the servers whose searches are running now.

Statistics:

Completed: Progress bar that show the percentage of servers finished. The progress bar is updated when the search has completed. The progress bar does not indicate the progress of a log file, only the server. If you are searching 5 log files per server, all 5 log files must complete before incrementing the bar.

Threads Running: The number of currently running threads. The name of each thread is also shown, either servers or log files.

Total Records searched: The number of events that have been examined.

Matching Events Found Total: The number of events that have matched the

search criteria.

Records per seconds: The average number of records per seconds that were parsed (saved or discarded.)

Total Bytes Read: The number of bytes reads. As the number grows, it will switch to Mbytes and then GBytes.

Search: Starts the search. If you are missing some criteria you will be asked to correct it.

Quit:

When searching: Quits the search. It may take several minutes for all the threads to exit.

When not searching Quits the application.


File

Launch Event Viewer

Opens the Event Viewer MMC

Open Log Directory

Opens the currently specified output directory in explorer. When a search is completed it opens automatically.

Exit

Exits the application


Options

Cache SIDS
Network lookup of SIDs is very expensive, SIDs are cached to prevent multiple looks for the same SID.

Cached SIDs are displayed in the main dialog box
Default: Enabled

Auto Fail Unavailable Servers
Skips servers that are unreachable, for instance, they could be off line, need alternate credentials, or lacking a specific right (security logs.) If you are searching log files, they could be corrupt.
Default: Enabled

Cache DLLs
At least one DLL must be loaded with LoadLibrary for EACH event log that needs to get parsed. Instead of unloading each one after I am done, I save the address and unload them at the end of the search. This could be disabled to reduce memory utilization.

Cached DLLs are displayed in the main dialog box
Default: Enabled

Resolve Hostname in 675 Records
A 675 Security event contains the IP address of the client that logged the error. If this option is checked 675 events call gethostbyaddr() to determine the hostname given that IP address.
Please Note: gethostbyadrr() is very slow, especially if the address cannot be resolved (running a saved log query offline, for instance.) Also if the clients are DCHP, this information may not be correct, however the IP address given would also be expired.

Default: Disabled

Only Get One Matching Event
This option was added to allow you to easily determine the last time your servers rebooted.
It could also be used to quickly determine if your servers are encountering severe problems that require immediate attention and you don’t want to wait for a detailed search to finish. Examples might be, Drive Array Errors, DS Corruption Errors.
Default: Disabled

Throttle CPU
Slows the application down. This can be used to reduce the network bandwidth that is used.

Popup Warnings for Full Event Logs
If an event log is full, your searches might not find the events you are looking for, depending on the options you selected, then could have been overwritten, or never logged. This option alerts you to that.
Default: Enabled

Notify if Criteria Found
If an event is found and written to a log a dialog box is shown containing the information.
This option is primarily meant for errors that you do not expect to find but would be critical if they were.
Only one popup will be displayed per server per log file.

Capture Event ID Statistics
Keep track of how many times an event is found in the log. This information is put at the end of the log file.
Click here to see an example.

Save Files As CSV Files

By default files are saved as .txt files. If this option is selected the extension is changed to .csv so that they can easily be opened in Excel.

Use Alternate Credentials

This allows you to use alternate credentials to connect to other servers. Since this utility connects to servers in so many different ways (LDAP, RPC, SMB) the use of Alternate Credentials is not recommended.

If should work fine for LDAP and SMB, however it does not work for RPC at this time. If you are using Alternate Credentials, the RPC FRS tests will not be available to you.

You can select alternate credentials before starting, or it will be displayed if an access denied error is returned when populating the Server List. You can not choose alternate credentials once you start searching.

Below is the dialog box that is show when you choose alternate credentials. You will have the option to save the credentials in the registry.

NOTE: The password is only somewhat encrypted and should only be stored in the registry for convenience.

Create Files for Failed / Empty Searches

By default if there are no parsed events the text file is deleted when the search is done. If this option is chosen a file will be created for all servers/logs that were searched. It will contain either of these lines:

C:\eventcombmt\Server-System_LOG.txt contains no parsed events.

Or

System Log on <notfound> not available. GetLastError was 131. Error text was: The RPC server is unavailable.

Data

Data formatting requires much more memory and space than the raw data. For instance a Dr. Watson record that is 120k, will be almost 650k when formatted into text.

Print Data As Words
This is a optional portion of an event record. If it exists it will be printed in this form: 0000: c0020017
If the data contains an error code, Word form is probably more useful.

Print Data As Bytes
This is a optional portion of an event record. If it exists it will be printed in this form: 0000: 17 00 02 c0 ...À

Slow Parameter Parsing

A very small group of Event ID's reuse the parameters that are passed into the event text. During normal parameter parsing when an insert is found, additional instances of that insert are not looked for. Almost all events can be parsed in normal mode, without a problem.

Unformatted Example:
The browser service was unable to retrieve a list of servers from the browser master %1 on the network %2.
Browser master: %1
Network: %2
This event may be caused by a temporary loss of network connectivity. If this message appears again, verify that the server is still connected to the network. The return code is in the Data text box.

Normal Parameter Parsing Example:
8021,WARNING,BROWSER,Sat Feb 16 08:51:47 2002,,The browser service was unable to retrieve a list of servers from the browser master %1 on the network %2.
Browser master: \\WSHBONE
Network: \Device\NetBT_Tcpip_{F2E0E1B1-88B5-415D-8985-556FA263106A}
This event may be caused by a temporary loss of network connectivity. If this message appears again, verify that the server is still connected to the network. The return code is in the Data text box.

Slow Parameter Parsing Example:
8021,WARNING,BROWSER,Sat Feb 16 08:51:47 2002,,The browser service was unable to retrieve a list of servers from the browser master \\WSHBONE on the network \Device\NetBT_Tcpip_{F2E0E1B1-88B5-415D-8985-556FA263106A}.
Browser master: \\WSHBONE
Network: \Device\NetBT_Tcpip_{F2E0E1B1-88B5-415D-8985-556FA263106A}
This event may be caused by a temporary loss of network connectivity. If this message appears again, verify that the server is still connected to the network. The return code is in the Data text box.

Write All Results to Database

This option will also write matched events out to an Access database called EventCombMT.mdb in the output directory. All text files will also be written as well.

All events will be written to the same table (EventCombMT) this will allow you to merge event logs (cluster servers) and get a better picture of DC health by being able to compare events from the same time period.

If an existing database is detected you will be given the option to merge the events into the database, create a new one, or cancel. Text files outputs are always overwritten.

The table format is as follows:

EventCombMT
Server | LogFile | EventID | RecordType | EventSource | EventDate | User | EventText
REGALTANG | System | 10005 | ERROR | DCOM | Tue Mar 05 15:42:11 2002 | No User | DCOM got error "The service cannot be started, either because it is disabled or because it has no enabled devices associated with it. " attempting to start the service IISADMIN with arguments "" in order to run the server: {A9E69610-B80D-11D0-B9B9-00A0C922E750}

NOTE: The threads for writing to the database must be serialized and will run much slower than writing to just text files.

Decode Event 1000 Flags

Userenv Event 1000 may contain a flags field.
Checking this option will cause this number to be parsed.

Default: Off

Example:

Without Decode:
1000,ERROR,Userenv,Fri Jun 29 14:59:56 2001,NT AUTHORITY\SYSTEM, The Group Policy client-side extension Security was passed flags (17) and returned a failure status code of (3).

With Decode:

1000,ERROR,Userenv,Fri Jun 29 14:59:56 2001,NT AUTHORITY\SYSTEM, The Group Policy client-side extension Security was passed flags (17) and returned a failure status code of (3). Flag Decode: Apply machine policy rather than user policy. Background refresh of policy.

Possible Flags:
GPO_INFO_FLAG_MACHINE              0x00000001  // Apply machine policy rather than user policy
GPO_INFO_FLAG_BACKGROUND           0x00000010  // Background refresh of policy (ok to do slow stuff)
GPO_INFO_FLAG_SLOWLINK             0x00000020  // Policy is being applied across a slow link
GPO_INFO_FLAG_VERBOSE              0x00000040  // Verbose output to the Eventlog
GPO_INFO_FLAG_NOCHANGES            0x00000080  // No changes were detected to the Group Policy Objects
GPO_INFO_FLAG_LINKTRANSITION       0x00000100  // A change in link speed was detected between previous policy
                                               // application and current policy application
GPO_INFO_FLAG_LOGRSOP_TRANSITION   0x00000200  // A Change in Rsop Logging was detected between previous policy
                                               // application and current policy application.
GPO_INFO_FLAG_FORCED_REFRESH       0x00000400  // Forced Refresh is being applied.
GPO_INFO_FLAG_SAFEMODE_BOOT        0x00000800  // windows safe mode boot flag
GPO_INFO_FLAG_ASYNC_FOREGROUND     0x00001000  // Asynchronous foreground refresh of policy
Find Longest Time Gap
This option will find the longest gap between records for a given event log. It will show the time of the record before the
longest gap in the event log.
Example:
The longest gap between all scanned records occurred at Tue Apr 30 09:50:54 2002 and was 91 days, 0 hours, 2 minutes, 44 seconds.
when the event records where:
40961,WARNING,LSASRV,Tue Jul 30 09:53:38 2002,No User,The Security System could not establish a secured connection with the server cifs/cheywest1. No authentication protocol was available.
3019,WARNING,MRxSmb,Tue Apr 30 09:50:54 2002,No User,The redirector failed to determine the connection type.
NOTE: This will find the gap for all the records scanned, not output to the text file or database. You could show a gap that does not
have a corresponding entry in the output file.
Default: Off
Set Date Range
  This allows you to set the range that events must be in before being included in the result set.
The default for both the FROM and the TO ranges are the current time.
Once a date range is set it remains in affect until it is changed or reset. To reset the date range, open the dialog again and check Reset date range to find all events. Changing either time or date will clear the checkbox to reset the date range.
NOTE:
The times cannot be the same.
The the FROM time must be earlier than the TO time.
You cannot enter a date earlier than 1980 or later than 2035.

Set Output Directory

Sets the output directory. By default it is c:\temp. If the directory doesn't exist it will be created. Previous path is saved.

Open Saved Logs

Allows you to perform all actions on a saved event log files.

Enter the path to your saved logs in the dialog box, you can also browse for a path. Previous path is saved


Command Line Options

NOTE: It may take a minute to parse the command line options and for the GUI to start.

These commands set fields that are exposed through the GUI.

Load a Saved Search

To load a search that you previously saved use:

/load:<previously saved search>

NOTE: if /load is specified no other parameters are parsed, except for /start.

DCs

To add all DCs in your domain to the list of servers to search use:
/dc

To add DCs from another domain use:
/dc:<domain name>

Example: /dc:redmond

Servers (from file)

To add servers from a text file use:
/file:<path to file>

Example: /file:"C:\program files\reskit\server.txt"

Servers (from command line)

To add server from the command line use:

/s:<server name>

Events

To specify events to search for use:
/evt:”string of events”

Example: /evt:"644 528 639"

Event Types

To specify the types of events to collect use:
/et:weisafasu
OR
/et:all

The different types are:

w - Warning
e - Error
i - Informational
sa - Success Audit
fa - Failure Audit
su - Success

Use all to search for all types

Event Logs

To specify event logs types use:

/log:sysappsecdsfrsdns
OR
/log:all

The log types are:

sys = System
app = Application
sec = Security
ds = Directory Services
frs = FRS
dns = DNS

Output Directory:

To specify the output directory use:

/outdir:”path to where output files should be written”

example: /outdir:"c:\program files\reskit\"

NOTE: Do not specify a filename. The path should include the trailing '\'.

Threads

To specify the number of threads use:

/t:<number>

NOTE: The default is 25.

Event Source

To specify the Event Source use:

/Source:”Source of event message”

Example: /source:netlogon

NOTE: When using the GUI the list of sources is pulled from the registry. When populated from the command line there is no validation checking. You could choose a source and a log/event combination that is not possible.

Event Text

To specify the text that needs to be in the event use:

/text:"text to match"

NOTE: Only use quotes for CMD.EXE's argument parsing. Do not include quotes, or logical expressions (AND, NOT, OR) in your search criteria, unless you are actually searching for that phrase. The search is case insensitive.

Date Range

To specify the date range use:

/after: to set the starting point for events

/before: to set the ending point for events.

NOTE: Both parameters take a date in the form of MMDDYYYYHHMMSS, or Month, Day, Year, Hour, Minute, Second. The time/date format needs to be exactly 14 characters. It cannot be a year before 1980 or after 2035. Both parameters must be used together.

Example: /after:05012002123000 /before:05052002123000

This resolves to:
Find Events After: Wed May 01 12:30:00 2002
Find Events Before: Sun May 05 12:30:00 2002

All Events

To override /text, /source, /time, /unit and /evt use:

/getallevents.

This is useful when you want to dump an entire event log to a text file.

These commands are only used when searching from the command line.

/nologfile use this to skip creating a log file. This might be useful if you are parsing all the text files that were created and wanted to skip EventCombMT.txt

/start Use /start to automatically start searching.

NOTE: Using /start will cause MessageBoxes to be thrown in the event of errors with parameters. If your parameters are incorrect and you are not using /start, the GUI should catch any problems when you click Search.

/help Using /help (/? or ?) shows this page.


Built In Searches

    • FRS
      • D2/D4 Check

1. Gets All DCs

2. Selects all DCs for searching

3. Selects the FRS Log

4. Selects Error and Warning Events

5. Sets the Specific IDs to 13520 13560 13561 13514

      • Staging Area Full

6. Gets All DCs

7. Selects all DCs for searching

8. Selects the FRS Log

9. Selects Warning Events

10. Sets the Specific IDs to 13522

      • FRS Health (Share,'8 &'9, DNS, ServerReference)

1. Gets All DCs

2. Selects all DCs for searching

3. Selects the FRS Log

4. Selects Warning Events

5. Sets the Specific IDs to 13508 and 13509

This search also displays and additional dialog box:

On this dialog box you can choose which FRS tests to run, the first test is always run. (The slider)

The first test (slider) is always run with this test, it attempts to match 13508s and 13509s, and display and error in the log file if the time difference is more than the number of minute you chose.

Default is 60 minutes (one hour.)

Test Failed Servers for Connectivity, SYSVOL and NETLOGON

Calls NetShareEnum and enters into the log if the SYSVOL and

NETLOGON share were found.

Default: Enabled

Test Failed Servers DNS A Record

Connects to your default DNS server (bypassing the cache) and prints the

returned address in the log.

Default: Enabled

Test Failed Servers for ServerReference

Connects to the PDC and displays its ServerReference attribute for the

failing server, flags an error if the SR attrib doesn’t contain the ServerName.

Also displays the SR attrib from the server having trouble connecting, and

compares it to the SR attrib on the PDC. Flags an error if they do

not match.

Default: Enabled

Test Failed Servers for RPC connectivity (and Get Polling Intervals)

Connects to the FRS RPC service listed as having not replicated. This

verifies name resolution, connectivity, RPC health, and FRS

service status on the remote machine. If successful displays the

Polling Intervals from the remote server.

Default: Enabled

Provide Verbose FRS information on each Domain Controller (50+k of text)

Dumps lots of data on each FRS config on each machine. Click here to see an example.

After choosing RFS health tests, the following dialog is displayed.

All other searches write to an individual log file, however the FRS Health test write to a single file which must be coordinated. The threads must wait for this resource and that can cause the application to appear hung. This is complicated by the fact that some of the calls made by these tests take a long time to fail if the server is unavailable. In particular are NetShareEnum and The RPC binding calls. RPC takes up to a minute (or more) to fail if the server is not available.

The status list box will display the following information when trying these tests so that you can tell it isn’t hung.

  • Calling NetShareEnum for <DC>. Please wait... or
  • Connecting to the FRS RPC service for <DC>. Please wait...

The FRS report is saved to a file called: FRS_Health_Analysis.txt in the logs directory. Click here to see an example file along with explanation.


Account Lockouts

Populates search criteria based on normal canned searches:

1. Gets All DCs

2. Selects all DCs for searching

3. Selects the Security Log

4. Selects Success and Failure Audits.

5. Sets the Specific IDs to 529 539 644


Help

Support

Displays this message.

Please report any problems encountered, however an individual response cannot be guaranteed.

Help

Should open this file. EventCombMT.chm.

About

Displays copyright and version information, and the names of the testers are also included. The date in the lower right corner is the last modified time of the source files.