End-Point Log Consolidation with Windows Event Forwarder
Derek Banks //
I want to expand on our previous blog post on consolidated endpoint event logging and use Windows Event Forwarding and live off the Microsoft land for shipping events to a central location. Why do this?
I wanted a Windows-based server with all of the event logs from the environment so that I could use PowerShell for analysis purposes. Because then I could potentially just send the forwarded events to an upstream ELK server and visualization and have multiple options to work with the data. This architecture then forms a part of the set-up needed for our CredDefense Toolkit.
Also, there are some environments where deploying yet another agent to Windows endpoints may not be desirable. WEF has been around for quite some time, but many people do not realize that log consolidation capability is built into Windows and does not use an agent on the endpoint.
There were a few really good guides that already exist (mentioned in the references links), but they did not get me completely over the hump to getting WEF completely functional. This is probably due to different releases Microsoft Server OSes. Windows Server 2012 was used on the server-side for all of the lab systems and there was a mix of Windows 10 Enterprise and Pro and Windows 7 Pro for workstations.
Windows Event Forwarder Setup
The first step is to stand up the collector server that will receive the logs from the rest of the windows systems in the environment. The size of the system will be determined by your environment, but we will not be sending every event, so a modest server can be used and then sized up if requirements change.
First, run the following commands on the collector server:
C:>winrm qc
This starts the WinRM service, and sets the service startup type to auto-start as well as configures a listener for the ports that send and receive WS-Management protocol messages.
Next use wecutil to configure the Windows Event Collector service and that it also starts when the system is rebooted.
C:>wecutil qc
This will also result in a Service Principal Name being registered for Kerberos authentication. If you are using an existing server and it has an HTTP SPN already registered WEF will not work unless you remove the existing one. If you’re using a new system, you probably will not have to worry about it. If during setup you are having issues and need to check SPN registration, you can do so with:
setspn -t <domain> -q */*
Create a Test Subscription on Collector server
Create a domain security group for the endpoints that you wish to monitor and place the target systems in the group. Alternatively, you could just use “Domain Computers” if you are in a testing environment. Otherwise, using all computers in your environment to initially set up may not be the best idea. Better to start smaller and work outwards than stumble out of the gate.
Once you work out what the target systems are, on the collector server open Event Viewer and select the Subscriptions. You will likely be prompted to start an auto-configure the Windows Collector service. Select “Yes”.
Right-click on Subscriptions and select “Create Subscription”. For the Subscription Name enter “Security Log Cleared”. The Destination log should be “Forwarded Events”. Select the radio button for “Source computer initiated” and select “Select Computer Groups…”. Add the target group that you will initially monitor.
Next, select “Select Events…” and the Event log drop-down choose the Security log.
For the purposes of this guide, we will create one GPO that will contain all the settings for forwarding event logs for endpoint security analysis. Additionally, all domain member computers will be forwarding to the same WEF server.
Open the Group Policy Management panel and select your domain right-click and select Create a GPO in this domain, and Link it here… Type in a name, such as Windows Event Forwarding and select OK.
Configure Event Log Forwarding Entry
Under Computer>Policies>Admin Templates>Windows Components>Event Forwarding Right click on the Configure target Subscription Manager entry and select Edit. Select the Enabled radio button and “Show” next to Subscription Managers in the Options pane.
Enter the following line in for the value substituting the Fully Qualified Domain name for the “eventserver.domain.local” portion of the URL below:
Server=http://eventserver.domain.local:5985/wsman/SubscriptionManager/WEC,Refresh=60
Note that this configuration is forwarding over HTTP rather than HTTPS. The forwarded event traffic can be encrypted and use HTTPS if desired.
Turn on Windows Remote Management (WS-Management) Service via GPO
The Windows Remote Management (WS-Management) service will need to be started on all the systems that will forward events. Note that they do not need to be listening on HTTP or HTTPS – the only system that needs that needs to be listening and have firewall rules configured is the WEF server.
To enable the Windows Remote Management to start on boot, in the Group Policy Management Editor select Computer Configuration>Preferences>Control Panel Settings>Services. Then right-click in the services pane and select New>Service.
In the startup field, select Automatic (Delayed Start) and select the service name as WinRM – also listed as Windows Remote Management (WS-Management). Leave the service action to “Start Service”. Click Apply and OK.
Allow Local Network Service to Access Local Event Logs via GPO
The local system that will be forwarding the logs to the central WEF server will need to have the Network Service account granted access to read event logs. There is a built-in Windows group that comes in handy for this called “Event Log Readers”.
Under Computer Configuration>Windows Settings>Security Settings>Restricted Groups, right-click and select Add Group… and type in Event Log Readers and select OK. Right-click on the Event Log Readers group that you just added and select properties and add NETWORK SERVICE. Click Apply and OK.
Sysmon GPO
Microsoft’s Sysmon is a tool released as part of the Sysinternals Suite. It extends the endpoint’s logging capability beyond the standard event logs. Windows now can natively log the full command line of a process that executes, but Sysmon provides additional data that can be very useful.
- Hash of executed process
- Network Connections
- File creation time changes
- WMI filters and consumers
On the local system, it stores these logs in Event Viewer in Application and Services Logs>Microsoft>Windows>Sysmon>Operational. By default, Sysmon logging will create a fair amount of log noise. This is why a configuration file should be used at install time to filter events at the endpoint that are known to be good or alert on specifically known bad. This way, you’ll won’t be shipping more than necessary to the central collector.
We recommend that you start with the excellent @SwiftOnSecurity configuration file that can be found at their Github page. From there, you can add to the file what you need to further reduce noise in your environment.
One way to deal with Sysmon deployment is to create a startup script via GPO that runs a batch file to check to see if Sysmon is installed and if not install it with the correct configuration file. If it is installed, make sure the service is started.
A sample install script can be found here. Make sure that the Sysmon executable, the configuration file, and the batch file are all in a common share. We chose the SYSVOL share – edit the script accordingly to your choice in your environment.
In the GPO Editor, choose Computer Configuration>Windows Settings>Scripts (Startup/Shutdown) and add in the install script.
PowerShell Script Block and Module Logging
Leveraging PowerShell for attacks has become very popular with both pentesters and actual threat actors. In the past, PowerShell logging capabilities were lacking, but that changed with PowerShell 5.0. This is the default version of Windows 10, so if you have migrated all Windows Endpoints from Windows 7, you’re good to go.
At the time of this post, most places have yet to move away completely from Windows 7 though. If this is true for your environment, you’ll need to install Windows Management Framework 5.0 and turn on logging via GPO. This task is left to the reader to figure out what best fits for their environment for software deployment.
After the install, use the GPO editor to turn on Module and Script Block logging. This is in Administrative Templates>Windows Components>Windows PowerShell.
Module logging will record pipeline execution details in Event ID 4103 and has details on scripts and formatted data from the output. Script Block Logging will record code as it is executed by the PowerShell Engine, therefore recording de-obfuscated code, to event ID 4104.
An additional caveat for Windows 7 systems is to download the Administrative Templates for Windows 10 and copy the PowerShellExecutionPlicy.admx and PowerShellExecutionPolicy.adml to the \\sysvol\Policies\Policy Definitions directory.
Additional Subscriptions
At this point, you should be ready to test out Event Forwarding. Before creating additional subscriptions, clear the event log of one of the subscribed endpoints. You can verify the status of the clients checking in by right-clicking the subscription and choosing Runtime Status. If everything is working appropriately, the cleared log event will soon show up in the Forwarded Events log on the WEF server.
If all is working correctly, it’s time for creating additional subscriptions that facilitate collecting what matters. The NSA Spotting the Adversary publication can be used as a guide, below is what we suggest as a minimum.
Account and Group Activity
This subscription will collect domain and local group and account activity. The Security Event Log events to add are: 4624,4625,4648,4728,4732,4634,4735,4740,4756. These events will be necessary to perform authentication analysis.
Kerberos
This subscription if for event ID 4769 from Domain Controllers. There will be a large amount of data recorded as ticket requests are frequent, however, paired with a HoneyToken account, it has the potential to detected Kerberoasting attacks.
Powershell Logging
To collect the module and script block events that were enabled earlier, create a subscription to gather the Microsoft-Windows-PowerShell/Operational log and get Event IDs 4103 and 4104.
Sysmon
All of the Sysmon log will be shipped to the WEF server. Select the Microsoft-Windows-Sysmon/Operational Event log and leave the targeted computers to “All Computers”.
Shipping Logs to ELK
Similar to our previous posts on endpoint log consolidation we will use nxlog to ship the logs from the WEF server to an ELK stack. This architecture allows for usage of PowerShell and C3 tools on the Windows-based log server and EVTX files as well as providing the visualization search capabilities with ELK.
Use the previous blog post to get ELK up and running if you need to. The configuration file for nxlog will be different. Install nxlog on the WEF server and modify the configuration file here to point to your ELK stack.
A basic logstash filter for ingesting the forwarded logs can be found here.
Conclusion
Windows Event Forwarder provides a native way to consolidate Windows Endpoint logs. PowerShell and C# tools can be used on the WEF server for analysis of the forwarded events. Additionally shipping the events to an ELK stack provides visualization and hunting capabilities. The overall design paired with sending specific log files such as authentication, PowerShell module, and script block logging and Sysmon logs creates a DIY SIEM set up that can be used to detect potential attackers in your network.
References:
https://joshuadlewis.blogspot.com/2014/10/advanced-threat-detection-with-sysmon_74.html
https://github.com/iadgov/Event-Forwarding-Guidance
Ready to learn more?
Level up your skills with affordable classes from Antisyphon!
Available live/virtual and on-demand
