Process Monitoring for the Curious and Paranoid

It's been months since I had time for any of this, but I've been thinking for a long time about what I would discover if I were to monitor process creation with some sort of balloon notification. Between coming to bed late one night, some scraps of time here and there to document it, and a day home to polish it off while my sick daughter naps, here's a useful tool. I want to emphasize, it's hacky, but for a busy father's casual/opportunistic research, it's enough to play jazz.

Objective

I would like to expediently answer a few questions, including:

• Is a new process the reason why my mouse pointer changed to the wait icon?
• Was a new process responsible for my computer slowing down?
• How often do new processes start, anyway?
• What are some commonly executed processes that I haven't noticed yet?
• Does this process run any sub-processes?
• Is there any process associated with that pop-up, or is it an already-running process?

Poring over my event logs is the wrong answer because eventvwr is slow to pop up and navigate, so when I am experiencing slowness, it doesn't allow me to get up-to-the-moment answers. Also, it can be tedious and time consuming to go back and find the right event, and my boss doesn't pay me to stare at event logs. And then how do I know that this event occurred at the same time as the phenomenon I'm observing?

What I want is a way to casually take note of interesting process creation events throughout the day without really spending time on it.

Alternative Solutions

I've had a few options rolling around in my head for a while:

• Instance creation event query on Win32_Process creation - Around 2005 I experimented with this and found that it cannot catch short-lived processes because they are created and destroyed between polling intervals which must last, at minimum, one second.
• Win32_ProcessTrace - I started out with this, but alas, they do not contain full image name information, so I needed to query the OS for further information, and again, short-lived processes result in information loss.
• Monitoring event logs - Event ID 4688 provides image names, but advanced configuration is required to obtain full command lines. Alternatively, SysInternals' Sysmon logs this information by default. WMI or other methods could be used to notify on event creation.
• The Windows kernel exports PsSetCreateProcessNotifyRoutineEx, which provides access to a convenient PS_CREATE_INFO structure containing the full image name and command line. Alas, this requires either purchasing and protecting an expensive driver signing certificate, or leaving a kernel code execution vulnerability unpatched so as to inject a driver as described in the whitepaper I published in February.

Implementation

Unfortunately, mucking with drivers is not lazy enough for me. Since short-lived processes are important (they are commonly used as part of post-exploitation / recon), a Win32_Process instance creation query won't work. For ease of use, I've created a first draft solution by Frankensteining two C# StackOverflow answers together to use systray balloon notifications with WMI's Win32_ProcessTrace. I put this on the Internet so I could compile it and use it to see what was going on with my work computer.

Here's the gist of it

It's lazy, but for casual/opportunistic research, it's enough to play jazz. It doesn't capture command-line arguments and doesn't always capture the full image name, because it just uses Win32_ProcessTrace and then the .NET System.Diagnostics classes to get process information after the fact.

Alas, it bothers me not to have full image names or command-line arguments. The best source of information I know of in userspace is event logs, but I had trouble getting the info I needed on advanced logging configuration for my Windows 8.1 box, I just installed Sysmon. Now what?

Another gist

As it turns out, it is necessary to modify the registry and restart the Windows Management Instrumentation service (and its dependent services) to make this work. I added a Microsoft-Windows-Sysmon/Operational key to HKLM\SYSTEM\CurrentControlSet\Services\EventLog and restarted the winmgmt service, and it all came together.

This gist has a detailed console view along with the systray notification to prevent me from having to necessarily open eventvwr to see more details. Here's how it looks:

Observations

Here are a few startling events and associated discoveries sure to send a chill down your spine, all from tracking down process activity during my journey:

• netsh.exe just ran. Is this some post-exploitation alteration of my firewall rules? No, a certain VPN client executes netsh.exe to get its work done. This was not the only software I caught doing that.
• Windows Remote Assistance COM Server (raserver.exe) executed and terminated immediately. What interfaces does this provide? Could this be post-exploitation enabling of remote assistance for future access? No. It's a scheduled task that triggers upon group policy updates so remote assistance knows to update its configuration.
• reg.exe just got run by cmd.exe. Holy schnikes, now I'm truly pwned. Is the parent process a backdoor executing persistence or other post-exploitation commands? Nope. It's just some endpoint management software that IT confirms they deploy and manage.
• Added 9/7/16: Heart rate increases as I read C:\Windows\system32\rundll32.exe C:\Windows\system32\inetcpl.cpl,ClearMyTracksByProcess Flags:525568 WinX:0 WinY:0 IEFrame:0000000000000000. Then I remember I just closed an IE in-private window. Take a look at the parent process command line and see: "C:\Program Files\Internet Explorer\IEXPLORE.EXE" -private. WHEW!

Value

So, as you can see, sometimes situational awareness is not all it's cracked up to be! As most DFIR people and hunters are aware, there is plenty of noise just sitting there waiting to alarm you.

Even so, I think this tool could be useful for noticing anomalous process observables such as:

• Post-exploitation commands a la RTFM, e.g. whoami, net.exe, netsh.exe, and so on
• svchost.exe executing from within your user profile
• Ransomware deleting shadow copies using vssadmin.exe

This tool can increase your awareness of what applications are responsible for certain behaviors, such as the Get Windows 10 user prompts that everybody loved so much. It can also raise your awareness of cases where security policies do not appear to be doing their job, such as application whitelisting. If you're a paranoid or curious power user, this may all be useful to you. In case it is, here are those gists again:

• ptray 1: Win32_ProcessTrace with augmented info from .NET FCL
• ptray 2: Win32_NTLogEvent instance creation against SysInternals Sysmon (requires registry change)

If I polish this up into something nicer, I'll try to update this article with the link.

Poor Man's Persistence Monitoring

I've been interested in detecting badness on my computer since I started using computers to begin with. To that end, there is an interesting SysInternals tool called autoruns that can go through a large number of potential persistence points and indicate anomalies. The output of the graphical tool is daunting, but the command line tool, autorunsc (documented along with autoruns), provides an opportunity to automate the detection of changes in these potential persistence points, allowing you to identify new potential badness on your 'puter. This amounts to a basic level of configuration auditing, with a special focus on configuration settings that are relevant to malware persistence. Here's how to do it.

The command-line autorunsc tool allows you to audit all persistence entries that it knows about, print its output as a comma-separated variable (CSV) file, show file hashes, and verify digital signatures. Using this, you can capture and examine a system baseline configuration, and use differential analysis to identify new anomalies on a recurring basis. The autorunsc command-line options to do all this can be found by running autorunsc with the /? switch. Here is an excerpt of the output showing the relevant arguments:

     -a         Show all entries.
     -c         Print output as CSV.
     -f         Show file hashes.
     -v         Verify digital signatures.

Since autorunsc outputs to the console, you can use output redirection to save this to a file within a command script:

autorunsc.exe -acfv > output.csv

Then you can use the Windows FC.EXE utility (I suppose FC stands for "file compare") to compare this with a past capture. Then you can use the IF ERRORLEVEL command to determine whether fc detected a difference.

For this to be effective, you must first capture and audit a baseline. This means triaging the files and registry settings in the first set of output. Easy checks for suspicious binaries are to look for the "(Verified)" label and confirm that the binary is signed by an author you trust, or to look up hashes on VirusTotal. Registry settings like KnownDLLs should only point to DLLs that are part of Windows. Once your baseline configuration is verified, you can perform a much quicker differential analysis (with FC.EXE as mentioned above) to detect changes.

If a difference is detected, you can use a utility such as gvim, diff, or WinDiff, to examine the differences. Note that if you use gvim for Windows, you may need to use a utility such as Cygwin's iconv.exe to convert the UCS-2 output of autorunsc.exe into UTF-8. This is what I do, and it looks like this:

I have a script that does this and presents changes using gvim. I placed this in my Startup program group and also run it periodically if my computer has not been rebooted in a long while. The script compares both the output from autorunsc and the contents of HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon\Notify, because that key is occasionally written to, but appears to maintain the same contents each time. In practice, something almost always changes across reboots, but it is not very difficult to locate and audit each item.

One byproduct of using this script has been an increased awareness of the changes that are made when software updates are installed. For instance, after an iTunes for Windows update (don't ask), I found out that Apple registers a WinSock namespace for Bonjour, pointing to a digitally signed DLL produced by Apple. Another byproduct of this investigation over time is that it gives me a gradual tour of the persistence points in Windows.

In case you would find it useful as the basis for your own persistence checks, my script can be had here:

https://github.com/strictlymike/persistmon

Modify it to your liking. Happy un-hacking ;-)

Everything is broken

In some cases doing malware triage, you may find yourself following up on invalid PE checksum or other anomalies. In these cases, it is interesting to know not only what the checksum in the file is, but also what it should be.

Gotcha

Using the FOR /R command, I thought I would casually see how many files on my system have zero or incorrect checksums...

@ECHO OFF

SET LOG=%~dpn0.log
SET TO_LOG=^>^> "%LOG%" 2^>^&1

SET MFACS=%USERPROFILE%\Documents\MapFileAndCheckSum\mfacs32.exe

:Recurse
    FOR /R %SystemDrive%\ %%f IN (*.exe *.dll *.ocx *.cpl *.scr *.ax) DO (
        ECHO %%f %TO_LOG%
        "%MFACS%" "%%f" %TO_LOG%
    )
    GOTO :EOF

As it turns out, they number in the thousands. Non-zero invalid checksums were significantly less numerous than zero checksums, but still surprisingly frequent, including many signed binaries.

So, it's not a very good sign of badness all by itself. Even so, I found that PE checksums might still be able to tell you one or two interesting things about a binary and the (type of) person who built it:

• 0x00000000 - That this is not a "release" version of the binary, or that the developer uses the command line or some other non-Visual Studio build system, and has omitted the /RELEASE linker flag, causing link.exe to omit the computation and insertion of the PE checksum. So perhaps your coder is a command line-oriented person?
• Mismatch - That the binary has been modified after it was compiled, or perhaps that a toolchain other than Microsoft's was used to create the binary.

There might be other conclusions that could be drawn as well. In any case, the source code for the utility, which is really just a command-line wrapper for the Imagehlp!MapFileAndCheckSum function, can be had here:

https://github.com/strictlymike/mfacs