Volatility Memory Analysis – A Working Environment

There have been quite a few articles floating about recently talking about the Volatility forensic tool, so I decided to take a closer look and see just how difficult it is to get up and running, and how quickly I could do useful things. From some of the more recent blogposts, it appears to be absurdly easy to do very powerful stuff. I think Volatility as a project started about 6 years ago. It probably began life as a tool dedicated for a single operating platform, but at this point can be used on MS Windows, Linux, Macintosh, and the Android OS.

Volatility Overview

Volatility operates on memory images, that is to say it’s a forensics tool which can tell you useful things by examining a snapshot of computer memory at a particular point in time. This is most useful to computer security analysts and incident responders, if for example there were a malware incident on some company’s server. You can download Volatility here. There may be an installer program available, but I know from prior experience that Volatility runs as a Python script (specifically it runs on the Python 2.7x branch), so I simply downloaded the Volatility source code. To follow along, you will need to install Python, and I’ve provided these instructions in a prior article. What follows is specific to MS Windows7, since that is the environment I use most. You can perform essentially the same steps on other operating platforms, but the details will be slightly different.

Required Tools

An information page provided by the Volatility developers gives details on performing a complete installation of Volatility, and this is quite useful since the tool depends on some third-party tools and libraries for certain functionality. The instructions on that wiki page begin with the installation of Python. They also mention setting your system path, but later I’ll discuss an easier way to create your Volatility test environment with a simple Windows shortcut (nearly all your Volatility work is done in a DOS command environment). The steps on the wiki page also discuss setting up MSys-MinGW (apparently the C++ compiler gcc might be needed), and if you know enough about MinGW then you can do that, but for basic testing of memory images you won’t need MinGW.

Next the wiki page talks about installing something called PyCrypto. This is a set of Python scripts which perform cryptographic functions, and they say it’s not required by Volatility, but you get stern warnings if it’s not installed. I saw various instructions on how to install PyCrypto, with most of them talking about downloading and compiling the source. Apparently PyCrypto compiles both by using MinGW and VisualC++. This appeared to be another exercise in compiling someone else’s source code for my operating platform. Fortunately someone built an installer for Windows, and if you run that installer you will find a log located in your Python install directory which describes what was done. The PyCrypto installer dumps a bunch of files into the windows Python sub-directories.

The wiki page next talks about installing the Distorm3 Disassembler, and the Yara Malware Plugins. Once again I was fortunate to find MS Windows installers for these, since I don’t wish to set up more build environments for small tools. After downloading the distorm3-3.win-amd64.exe file I was able to install it in both Python 2.76 and Python 3.40 on my local machine. The Yara Plugins provide separate installers for the Python 2.76 and Python 3.40 branches (I used the yara-python-1.7.win-amd64-py2.7.exe and yara-python-1.7.win-amd64-py3.3.exe files). Once again these installers placed log files in the Python main directory which listed what files were placed in what Python sub-directories. The last thing the wiki page talks about is intalling a SVN client, and you are free to do this. I didn’t bother with the SVN client since everything is already installed and I don’t need the latest bleeding edge of these programs every day.

Setting Up The Test Environment

I promised to show how to set up your Volatility test environment, and this is actually quite easy. First pick a directory path where you will place your Volatility projects, such as K:\HackProjects\Volatility\ , and for initial testing I created a subdirectory to hold memory images here K:\HackProjects\Volatility\MemoryImages\ . Then I created a DOS command shortcut located at K:\HackProjects\Volatility\VolatilityLaunch.lnk which resolves to the target

C:\Windows\System32\cmd.exe /k set Path=%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;C:\ProgramsDev2\Python_276;C:\ProgramsDev2\Python_276\Scripts;C:\ProgramsDev2\Volatility_231;K:\MSysMinGWProjects\MSysMinGW\MinGw64\bin

 

Again you can leave the MinGW path out for basic testing. This target simply sets the path environment variable so that relevant files are found when I enter DOS commands. Now just double-click that shortcut whenever you want to perform work with Volatility.

Obtaining Volatility Memory Images

Volatility works with memory images, so to start of course you have to have a memory image. How do you get one? For the testing I did, I initially downloaded an image shared by a security analyst. I also performed a basic search to find out how to obtain a memory image on my own, and as usual I found several dozen tools which will perform this operation on any of several operating platforms. I only plan to talk about one or two of these imaging techniques, since there are just way too many of them.

From my reading I learned that I could perform a memory dump of a running VirtualBox machine, and since I do use VirtualBox quite frequently I decided to try this first. The first approach I tried said to launch VirtualBox with a debug parameter, and then use a particular debug command to dump the physical memory. The command to launch VirtualBox is this

VirtualBox.exe –dbg –startvm “my virtual machine name”

 

This command does indeed launch VirtualBox with a debug console window embedded at the bottom of the main window, and there appear to be lots of interesting debug commands one can perform. The debug command to dump the memory to file is the following

.pgmphystofile filename

 

After playing with this for a short while and looking up other peoples’ experiences with it, I learned that this is a debug feature which is not yet implmented. On one forum I read that VirtualBox will stream the machine state to file when it is suspended, and this can be performed from the menu with

Close Virtual Machine | Save the machine state

 

as well as on the command line with

VBoxManage controlvm “my virtual machine name” savestate

 

I tried this and found that it worked. Some kind of machine state blob was saved in the sub-directory Snapshots\ where the machine’s .vbox file is located, and on Windows7 this is at C:\Users\UserName\VirtualBox VMs\My Virtual Machine Name\ . I proceeded forward with this memory image, and I only learned later that Volatility was not able to identify any information from this data blob. The reason this image was unusable I suspect is because the machine state is being compressed and/or encoded somehow, because when I was able to get a good usable image it turned out to be approximately 4 times larger.

I finally ran into a technique which provided a good image of the VirtualBox memory state and saved it to file. The process is to first launch your virtual machine as you normally would, open a few applications for testing purposes, then open a DOS console in the host and issue the following command

vboxmanage debugvm “my virtual machine name” dumpguestcore –filename MyVirtualMachine.elf

 

I had to be in the VirtualBox install directory to run that command, and the file MyVirtualMachine.elf was created in the same directory. I used the extension .elf since that is the format used by the VirtualBox machine core. While this file did contain an image of the running memory of the machine, it also contained lots of VirtualBox meta information. I ran into a site providing a Volatility plugin which claims to limit the analysis to only the machine memory image. The reader is encouraged to test this plugin and learn from it, although I won’t be pursuing it further in this article. This example demonstrates one other thing, that Volatility plugins are really nothing more than Python scripts, and with a little Python knowledge you can build your own Volatility plugins.

Volatility Commands

Armed with a couple memory images and a working test environment, I decided to look briefly into what Volatility offers. The ‘language’ of Volatility, the set of commands, is not overly large, and appears to be quite powerful. The Volatility wiki provides a nice summary of each command, with just enough information so you can be productive without overwhelming the reader. This command reference page breaks down into command categories, which I’ve listed out

Image Identification
imageinfo – find suggested profiles, based on KDBG search
kdbgscan – designed to positively identify the correct profile and the correct KDBG address
kpcrscan – scan for potential KPCR structures

Processes and DLLs
pslist – list the processes of a system
pstree – view the process listing in tree form
psscan – enumerate processes using pool tag scanning (POOL_HEADER)
psdispscan – similar to psscan, except it enumerates processes by scanning for DISPATCHER_HEADER instead of pool tags
dlllist – display a process’s loaded DLLs
dlldump – extract a DLL from a process’s memory space and dump it to disk for analysis
handles – display the open handles in a process
getsids – view the SIDs (Security Identifiers) associated with a process
cmdscan – search the memory of csrss.exe on XP/2003/Vista/2008 and conhost.exe on Windows 7 for commands that attackers entered through a console shell (cmd.exe)
consoles – similar to cmdscan, find commands that attackers typed into cmd.exe or executed via backdoors
privs – show you which process privileges are present, enabled, and/or enabled by default
envars – display a process’s environment variables
verinfo – display the version information embedded in PE files
enumfunc – enumerate imported and exported functions from processes, dlls, and kernel drivers

Process Memory
memmap – show you exactly which pages are memory resident, given a specific process DTB
memdump – extract all memory resident pages in a process into an individual file
procmemdump – dump a process’s executable
procexedump – dump a process’s executable (not including the slack space)
vadinfo – display extended information about a process’s VAD nodes
vadwalk – inspect a process’s VAD nodes in table form
vadtree – display the VAD nodes in a visual tree form
vaddump – extract the range of pages described by a VAD node
evtlogs – extract and parse binary event logs from memory
iehistory – recovers fragments of IE history index.dat cache files

Kernel Memory and Objects
modules – view the list of kernel drivers loaded on the system
modscan – find LDR_DATA_TABLE_ENTRY structures by scanning physical memory for pool tags
moddump – extract a kernel driver to a file
ssdt – list the functions in the Native and GUI SSDTs
driverscan – find DRIVER_OBJECTs in physical memory using pool tag scanning
filescan – find FILE_OBJECTs in physical memory using pool tag scanning
mutantscan – scan physical memory for KMUTANT objects with pool tag scanning
symlinkscan – scan for symbolic link objects and output their information
thrdscan – find ETHREAD objects in physical memory with pool tag scanning
dumpfiles – extract all files cached in the system memory
unloadedmodules – display information on recently unloaded drivers

Networking
connections – view TCP connections that were active at the time of the memory acquisition
connscan – find _TCPT_OBJECT structures using pool tag scanning
sockets – detect listening sockets for any protocol (TCP, UDP, RAW, etc)
sockscan – find _ADDRESS_OBJECT structures using pool tag scanning
netscan – scan for network artifacts in 32- and 64-bit Windows Vista, Windows 2008 Server and Windows 7 memory dumps

Registry
hivescan – find the physical addresses of CMHIVEs (registry hives) in memory
hivelist – locate the virtual addresses of registry hives in memory, and the full paths to the corresponding hive on disk
printkey – display the subkeys, values, data, and data types contained within a specified registry key
hivedump – recursively list all subkeys in a hive
hashdump – extract and decrypt cached domain credentials stored in the registry
lsadump – dump LSA secrets from the registry
userassist – get the UserAssist keys from a sample
shellbags – parse and print Shellbag (pdf) information obtained from the registry
shimcache – parse the Application Compatibility Shim Cache registry key
getservicesids – calculate the SIDs for services on a machine and output them in Python dictionary format for future use

Crash Dumps, Hibernation, and Conversion
crashinfo – print crashdump header information
hibinfo – reveal additional information stored in the hibernation file
imagecopy – convert any existing type of address space (such as a crashdump, hibernation file, virtualbox core dump, vmware snapshot, or live firewire session) to a raw memory image
raw2dmp – convert a raw memory dump (for example from a win32dd acquisition or a VMware .vmem file) into a Microsoft crash dump
vboxinfo – pull details from a virtualbox core dump
vmwareinfo – analyze header information from vmware saved state (vmss) or vmware snapshot (vmsn) files
hpakinfo – show info from an hpak formatted memory dump created by FDPro.exe
hpakextract – extract and decompress the physical memory image from an hpak file whose contents are compressed

File System
mbrparser – scan for and parse potential Master Boot Records (MBRs)
mftparser – scan for potential Master File Table (MFT) entries in memory (using “FILE” and “BAAD” signatures) and print out information for certain attributes

Miscellaneous
strings – for a given image and a file with lines of the form :, output the corresponding process and virtual addresses where that string can be found
volshell – interactively explore a memory image
bioskbd – read keystrokes from the BIOS area of memory
patcher – patches memory image based on contents of an input XML file
pagecheck – use a kernel DTB (from the System/Idle process) and determine which pages should be memory resident
timeliner – create a timeline from various artifacts in memory from the following sources: processes, DLLs, modules, sockets, registry keys and embedded registry data

 

Looking around the Volatility wiki, it appears that many of these commands are actually plugins that hit the big time and became incorporated into the tool itself. Somebody put together a nice pdf describing how to compile and install these Volatility plugins. If you scan through that document you’ll find that about 80% of it describes how to set up MinGW and associated tools like 7-Zip, while only a small bit of it offers any advice on compiling plugins (and that little bit is confusing and misleading). My experience is that these plugins often reference things only found in Linux, so if you launch into a MSys-MinGW shell (and you have Python installed in that environment) then the setup is fairly easy, but it usually will not work simply by using a DOS command shell with a path to the MinGW compiler. I’ll put up a blogpost about this later after I investigate the plugin setup topic further.

Issuing A Volatility Command

Launching the shortcut which opens my Volatility test environment, I can now issue a Volatility command such as the following

C:\ProgramsDev2\Volatility_231\vol.py imageinfo -f K:\HackProjects\Volatility\MemoryImages\WIN-TTUMF6EI3O3-20140203-123134.raw

 

Volatility1
 

Since Python v2.76 is on my path, there is no need to call it directly (Windows already knows which executable to launch when a .py file is invoked). The Volatility command being used is placed somewhere on the command-line by itself, and the -f switch is followed by a memory image file. Depending on the speed of your computer, some of these commands may take a number of minutes to complete. I won’t go too deeply into specific Volatility commands or features, since my intent was simply outline and document the setup process, and give an overview of what Volatility is, what it does, and what it’s comprised of. However, I probably will write future articles describing specific Volatility commands in greater detail.

One final thing I’d like to mention is that when you download and unpack the Volatility distribution (again I didn’t use an installer, but simply unzipped it into its final location), there is a file there named README.txt which is crammed with valuable information, everything from descriptions for command-line switches to a complete listing of Volatility profiles (operating system image types). Have fun and don’t hurt yourself with these security tools.

Additional Resources

Volatility Labs Blog
Volatility Code Repository

 

1 comment for “Volatility Memory Analysis – A Working Environment

  1. April 13, 2014 at 1:33 am

    You need targeted visitors for your website so why not get some for free? There is a VERY POWERFUL and POPULAR company out there who now lets you try their traffic service for 7 days free of charge. I am so glad they opened their traffic system back up to the public! Sign up before it is too late: http://axr.be/17r1

Leave a Reply

Your email address will not be published. Required fields are marked *