Michael,
Thanks again - that really helps, and makes sense!
For clarity, given this is a Win7SP1x86 machine where I have disabled
PAE and disabled pagefile, is it correct to conclude that ALL of the
memory that this process is using is in the capture? (I gave it 512MB
RAM and captured all the RAM.)
Adam
On 30/08/13 22:27, Michael Cohen wrote:
> Adam,
> Virtual memory is not contiguous. memmap will show you the regions
> which are mapped to physical memory and the offset in the virtual
> address space where they are mapped into. The simple answer is that
> the process's virtual address range from 0x00000000-0x00010000 does
> not exist - all this means is that if the process was to reference
> addresses in that range, the processor will issue a page fault, and be
> killed.
>
> Note also that for 64 bit processes, the address space is much larger
> than 4GB (2^^48 actually).
>
> Michael.
>
> On 30 August 2013 22:50, Adam Bridge <adam.bridge(a)yahoo.com> wrote:
>> Hi all,
>>
>> Hoping for some more newbie assistance!
>>
>> I have a sample from Win7SP1x86.
>> When I took the capture I had notepad.exe running.
>>
>> Using pslist(1) I identified the pid and used this with memmap(2).
>> (1) python vol.py -f win7.raw --profile=Win7SP1x86 pslist
>> (2) python vol.py -f win7.raw --profile=Win7SP1x86 memmap -p 1260
>>
>> So really two questions:
>>
>> 1> Why does the first entry show a virtual offset of 0x00010000? Why
>> isn't it 0x00000000? Where are the first 0x00010000 bytes of this
>> process's virtual memory?
>>
>> 2> (and I know this is gonna be a face palm moment) Why aren't the
>> virtual memory offsets contiguous? If this is a dump of the process's
>> virtual memory shouldn't it be one big lump of 4GB? What's the obvious
>> thing I'm missing? (Is it simply that notepad.exe isn't using all 4GB so
>> the empty pages have been ignored?)
>>
>> Thank you!
>>
>> Adam
>>
>> --
>> Have you sent me your PGP Public Key yet?
>>
>> _______________________________________________
>> Vol-users mailing list
>> Vol-users(a)volatilityfoundation.org
>> http://lists.volatilityfoundation.org/mailman/listinfo/vol-users
--
Have you sent me your PGP Public Key yet?
Hi all,
Hoping for some more newbie assistance!
I have a sample from Win7SP1x86.
When I took the capture I had notepad.exe running.
Using pslist(1) I identified the pid and used this with memmap(2).
(1) python vol.py -f win7.raw --profile=Win7SP1x86 pslist
(2) python vol.py -f win7.raw --profile=Win7SP1x86 memmap -p 1260
So really two questions:
1> Why does the first entry show a virtual offset of 0x00010000? Why
isn't it 0x00000000? Where are the first 0x00010000 bytes of this
process's virtual memory?
2> (and I know this is gonna be a face palm moment) Why aren't the
virtual memory offsets contiguous? If this is a dump of the process's
virtual memory shouldn't it be one big lump of 4GB? What's the obvious
thing I'm missing? (Is it simply that notepad.exe isn't using all 4GB so
the empty pages have been ignored?)
Thank you!
Adam
--
Have you sent me your PGP Public Key yet?
Hi all,
I'm definitely still learning with memory forensics, but I can't get my
head around this one.
I created a Virtualbox VM of Win7SP1x86 with 512MB RAM.
I disabled the pagefile - confirmed with reboot that pagefile.sys
disappeared.
I disabled pae - confirmed with reboot followed by: wcim os get
PAEEnabled, returned FALSE.
I then used:
vboxmanage debugvm "Win7" dumpguestcore --filename test.elf
to grab the ELF64 dump.
This file is: 569.5MB
I then used:
python vol.py -f test.elf --profile=Win7SP1x86 imagecopy -O test.raw
test.raw is: 4.0GB
Given that pae is off and pagefile.sys is off, where has the extra data
come from?!
I get that in 32-bit, we can represent up to 0xFFFFFFFF (2^32) = 4GB,
but where has the extra data come from?
Is it all going to be 0-padded or have I done something wrong somewhere?!
Any clues, tips, links to read, and flames welcome.
Adam
--
If you like, we could go PGP..?
Hi All,
Currently, I am using Volatility to analyze a lime dump of an Android device and I have the same error message as the post of the no suitable address space mapping found (http://lists.volatilityfoundation.org/pipermail/vol-users/2013-July/000942.…)
I have followed the steps as indicated in the Volatility Android memory forensic instructions (https://code.google.com/p/volatility/wiki/AndroidMemoryForensics) and listed them below the dotted line in this mail.
However, the error No suitable address space mapping found is showing.
Anybody have any idea what is going / I am doing wrong ? (please see the steps I have performed below)
Winston
*****************************************
Steps I followed:
Memory research of Device : HTC One V
kernel device primou-ics-crc-3.0.16-133e482
Android : 4.0.3
Host system for Volatility: Ubuntu 13.04
Python 2.7.4 (default, Apr 19 2013, 18:32:33)
[GCC 4.7.3] on linux2
Steps as followed from https://code.google.com/p/volatility/wiki/AndroidMemoryForensics except for the emulator steps:
1. Downloaded lime, cross compiled lime and build a *.ko file and created a lime.dump (format=lime) file
2. Downloaded Volatility, created a zip profile
a. System.map retrieved from the device at /proc/kallsyms
b. Module.dwarf
$ head module.dwarf
.debug_info
<0><0x0+0xb><DW_TAG_compile_unit> DW_AT_producer<"GNU C 4.7"> DW_AT_language<DW_LANG_C89> DW_AT_name<"/android/volatility-2.2/tools/linux/module.c"> DW_AT_comp_dir<"/home/winston/htc/primou-ics-crc-3.0.16-133e482"> DW_AT_stmt_list<0x00000000>
<1><0x1d><DW_TAG_typedef> DW_AT_name<"__s8"> DW_AT_decl_file<0x00000001 include/asm-generic/int-ll64.h> DW_AT_decl_line<0x00000013> DW_AT_type<<0x00000028>>
<1><0x28><DW_TAG_base_type> DW_AT_byte_size<0x00000001> DW_AT_encoding<DW_ATE_signed_char> DW_AT_name<"signed char">
<1><0x2f><DW_TAG_typedef> DW_AT_name<"__u8"> DW_AT_decl_file<0x00000001 include/asm-generic/int-ll64.h> DW_AT_decl_line<0x00000014> DW_AT_type<<0x0000003a>>
<1><0x3a><DW_TAG_base_type> DW_AT_byte_size<0x00000001> DW_AT_encoding<DW_ATE_unsigned_char> DW_AT_name<"unsigned char">
<1><0x41><DW_TAG_typedef> DW_AT_name<"__s16"> DW_AT_decl_file<0x00000001 include/asm-generic/int-ll64.h> DW_AT_decl_line<0x00000016> DW_AT_type<<0x0000004c>>
<1><0x4c><DW_TAG_base_type> DW_AT_byte_size<0x00000002> DW_AT_encoding<DW_ATE_signed> DW_AT_name<"short int">
3. Using Volatility 2.2 and I have tried volatility 2.3-development and the latest volatility from svn co https://volatility.googlecode.com/svn/trunk (latest check out at 9th of august 2013)
a. $ python vol.py info
LinuxprofileHTCOneV2x86 - A Profile for Linux profileHTCOneV2 x86
b. Note, I implemented a work around since my system.map / proc/kallsyms sometimes contained four columns instead of 3.
Part of my system.map file:
c0682d70 A _etext
bf005000 t dhd_sleep_pm_callback [bcmdhd]
Error:
File "/android/volatility-2.2/volatility/plugins/overlays/linux/linux.py", line 86, in parse_system_map
(str_addr, symbol_type, symbol) = line.strip().split()
ValueError: too many values to unpack
Work around :
Added in /android/volatility-2.2/volatility/plugins/overlays/linux/linux.py, line 87:
(str_addr, symbol_type, symbol) = line.strip().split()[0:3] //added work around
#(str_addr, symbol_type, symbol) = line.strip().split() // original
c. $ python vol.py --profile=LinuxprofileHTCOneV2x86 -f /android/resultfiles/HTVOneV/lime7-31-13_1317.lime linux_pslist
Volatile Systems Volatility Framework 2.3_alpha
WARNING : volatility.obj : Overlay structure cpuinfo_x86 not present in vtypes
Offset Name Pid Uid Start Time
---------- -------------------- --------------- --------------- ----------
No suitable address space mapping found
Tried to open image as:
MachOAddressSpace: mac: need base
LimeAddressSpace: lime: need base
WindowsHiberFileSpace32: No base Address Space
WindowsCrashDumpSpace64: No base Address Space
WindowsCrashDumpSpace32: No base Address Space
JKIA32PagedMemoryPae: No base Address Space
AMD64PagedMemory: No base Address Space
JKIA32PagedMemory: No base Address Space
IA32PagedMemoryPae: Module disabled
IA32PagedMemory: Module disabled
MachOAddressSpace: MachO Header signature invalid
MachOAddressSpace: MachO Header signature invalid
LimeAddressSpace: Invalid Lime header signature
WindowsHiberFileSpace32: PO_MEMORY_IMAGE is not available in profile
WindowsCrashDumpSpace64: Header signature invalid
WindowsCrashDumpSpace32: Header signature invalid
JKIA32PagedMemoryPae - EXCEPTION: unsupported operand type(s) for -: 'NoneType' and 'long'
AMD64PagedMemory: Incompatible profile LinuxprofileHTCOneV2x86 selected
JKIA32PagedMemory - EXCEPTION: unsupported operand type(s) for -: 'NoneType' and 'long'
IA32PagedMemoryPae: Module disabled
IA32PagedMemory: Module disabled
FileAddressSpace: Must be first Address Space
ArmAddressSpace - EXCEPTION: unsupported operand type(s) for -: 'NoneType' and 'long'
Hey all,
A while ago I did some experiments with finding the swap_info_structs
for linux swap files/devices.
Since I don't have time to finish any work on that (this was the easy
part) I've uploaded it to my github:
https://github.com/Dutchy-/volatility-plugins/blob/master/linux/swap.py
Do with it as you wish :D note that a proper swap implementation would
likely need modification in volatility itself, not just a plugin.
Cheers,
Edwin
PS. I've also added all the plugins from my submission to the plugin contest.
This post is not specifically about Volatility; however, I know that the
topic will be of interest to many list members. Reliable memory
analysis depends on reliable memory acquisition. No matter how good an
analysis tool is, it can only analyze what is in the memory "dump." If
the memory acquisition tool produces an incomplete or erroneous memory
dump the analysis will also fail or be misleading. It is thus with
pleasure that I note the progress that is being made toward developing a
critical framework for evaluating and testing memory acquisition tools.
The latest contribution is by Stefan Vömel and Johannes Stüttgen whose
paper, "An evaluation platform for forensic memory acquisition software"
was presented at the recent DFRWS conference and may be downloaded from
the following link:
http://www.dfrws.org/2013/proceedings/DFRWS2013-11.pdf.
The authors adopt an approach which they call "white-box testing"
whereby the authors modify the source code for various open source tools
(win32dd, mdd, winpmem) to insert hypercalls at various locations in the
acquisition process. The hypercalls inform the test platform of various
important system events and operations and inspect the state of the
subject system at the moment of the hypercall. The state as recorded by
the hypercalls is then used as a metric to evaluate the reliability
(i.e. "correctness") of the tool which is under test.
The approach has a number of significant limitations which the authors
acknowledge, the most significant of which is that they require the
source code which must be modified in order to perform the test. Most
commercial tool vendors do not provide the source code to their memory
acquisition tools. Even one of the open source tools tested by Vömel
and Stüttgen, win32dd, is an old version of the current closed-source
Moonsols tool which contains many bug fixes which are not in the open
source precursor. As far as I am aware the MDD tool is no longer
supported or under active development. Michael Cohen's winpmem is the
only currently supported tool that the authors were able to test.
Another significant limitation is that the test platform is tied to a
highly customized version of the Bochs x86 PC emulator. The test
platform is restricted to 32-bit versions of Windows with not more than
2 GiB of memory. The acquisition of physical memory from systems
equipped with more than 4 GiB of memory and from 64-bit versions of
Microsoft Windows are areas where memory acquisition tool vendors have
stumbled in the past. Possibly all contemporary memory acquisition
tools handle 64-bit systems and systems with more than 4 GiB of memory
correctly; however, we would like to be able to test this and not rely
solely on faith.
One limitation which the authors do not discuss is the impact of
restricting the test platform to a particular VM (i.e. Bochs). In our
experience VMs provide a much more highly predictable and stable
environment than real hardware and may not be a good indication of how a
memory acquisition tool will perform on real hardware. In addition, as
was mentioned previously on this list, different VM manufacturers have
chosen to emulate very different PC designs. How a tool performs on
VMWare may not be a good indicator of how the same tool will perform on
Microsoft Hyper-V or VirtualBox or KVM.
Also, the authors do not acknowledge the possibility that memory
acquisition tools may perform differently on different versions of the
Microsoft operating system. Each new version of the Microsoft operating
system has brought changes to the Windows memory manager, in some cases
significant changes. Currently, we are (out of necessity) using
acquisition methods that differ in varying degrees on Microsoft Windows
XP, Windows 2003 SP1 and later, Windows Vista, Windows 7 and Windows 8.
For Windows 8.1 Preview we have found it necessary to invent new
methods that are different from all of the methods which we previously
employed.
Finally, the authors do not articulate the theoretical framework for
forensic volatile memory acquisition which serves as the basis for their
notion of "correctness." Historically, computer forensic experts have
evaluated the acquisition of volatile memory as an "imaging" process.
Most computer forensic experts were familiar with the imaging of "dead"
hard drives. It was natural to assume memory acquisition was doing much
the same thing. The problem is that a running computer system is by
nature a stochastic process (more precisely, it is a "continuous time
stochastic process") which cannot be "imaged." It can only be sampled.
Sampling is a common technique in forensic science, other than in
computer forensics. Computer forensics is somewhat unique among
forensic "sciences" in its almost exclusive reliance upon "imaging" as a
standard of reliability. It is difficult not to note the severe tension
which exists between this theoretic framework and the reality of modern
computer design. Not only is "live" imaging of hard drives now
commonplace for practical reasons (which equally aren't true images) but
also, as we now know, a hard drive is an embedded computer system which
may alter the physical layout of data on the disks any time power is
applied to the drive.
The theoretical approach which we have advocated (see, e.g. Garner, Maut
Design Document (unpublished manuscript, 2011), is to view volatile
memory acquisition as a process of sampling the state of a
continuous-time stochastic process. We further propose to use the
structural elements created by the operating system and the hardware as
a metric to evaluate the reliability of the sampling process. We
believe that these structural elements may be used for this purpose
because they possess the Markhov property. Their future state is
predictable, depending entirely on their present state and is
conditionally independent of the past. A sample is said to be reliable
when it is "representative" of the entire population. In other words, a
sample is reliable with respect to a specific issue of material fact if
an inference drawn upon the basis of the sample will arrive at the same
conclusion as if the inference were drawn upon the basis of the entire
population.
The authors appear to assume the tradition "imaging" framework for
volatile computer memory acquisition, however, this assumption should be
stated since the entire rest of the paper depends on it.
In conclusion, this paper makes an important contribution to a topic
that is important for the future of computer forensics. However, the
authors need to better articulate their assumptions. Development of a
professional memory forensic tool testing platform will require the
development of a test enviroment which overcomes the current limitations.
Regards,
George M. Garner Jr.
President
GMG Systems, Inc.
Hi all,
I wrote a little tool to convert a KVM/libvirt dump to a raw memory
file (https://github.com/juergh/lqs2mem) Volatility seems to be able
to handle the resulting file just fine for small dumps but not so much
the larger they get. Specifically, things start to break when the
memory size of the VM approaches 4 GB. I double and triple checked my
code and can't find anything obviously wrong (like using a 32bit
variable for a 64bit address or pointer). I also don't think that
Volatility has a problem with larger dumps since it can handle a 8 GB
memory dump that I obtained using some other means. I'm just running
out of ideas and am looking for some help or suggestions on how to
debug this further.
In my testing with Win 2008 R2 SP1 x64 I found that (see full outputs below):
1) imageinfo and pslist return the correct output for VMs with less than 3588 MB
2) pslist only returns a single task (System) for VMs larger than 3587 MB
3) imageinfo shows only 1 processor (when there are actually two) for
VMs larger than 3712 MB (give or take)
Any help is greatly appreciated.
Thanks
...Juerg
VM memory size: 3584 MB:
Determining profile based on KDBG search...
Suggested Profile(s) : Win2008R2SP0x64, Win7SP1x64,
Win7SP0x64, Win2008R2SP1x64
AS Layer1 : AMD64PagedMemory (Kernel AS)
AS Layer2 : FileAddressSpace
(/var/lib/libvirt/qemu/save/win-3584.ram)
PAE type : PAE
DTB : 0x187000L
KDBG : 0xf800017fb0a0
Number of Processors : 2
Image Type (Service Pack) : 1
KPCR for CPU 0 : 0xfffff800017fcd00L
KPCR for CPU 1 : 0xfffff880009b8000L
KUSER_SHARED_DATA : 0xfffff78000000000L
Image date and time : 2013-07-16 12:24:50 UTC+0000
Image local date and time : 2013-07-16 12:24:50 +0000
Offset(V) Name PID PPID Thds Hnds
Sess Wow64 Start Exit
------------------ -------------------- ------ ------ ------ --------
------ ------ ------------------------------
------------------------------
0xfffffa8002a7cb30 System 4 0 70 396
------ 0 2013-07-16 12:24:33 UTC+0000
0xfffffa80030f09d0 smss.exe 220 4 4 31
------ 0 2013-07-16 12:24:33 UTC+0000
0xfffffa80034574d0 csrss.exe 300 292 9 339
0 0 2013-07-16 12:24:34 UTC+0000
0xfffffa8003465b30 wininit.exe 352 292 7 93
0 0 2013-07-16 12:24:34 UTC+0000
0xfffffa8003469b30 csrss.exe 368 344 8 76
1 0 2013-07-16 12:24:34 UTC+0000
0xfffffa800349c280 winlogon.exe 412 344 5 83
1 0 2013-07-16 12:24:34 UTC+0000
0xfffffa80034a7160 services.exe 448 352 17 215
0 0 2013-07-16 12:24:34 UTC+0000
0xfffffa80034b4b30 lsass.exe 464 352 9 458
0 0 2013-07-16 12:24:34 UTC+0000
0xfffffa80034b64f0 lsm.exe 472 352 12 194
0 0 2013-07-16 12:24:34 UTC+0000
0xfffffa800350cb30 svchost.exe 584 448 17 355
0 0 2013-07-16 12:24:34 UTC+0000
0xfffffa8003522060 svchost.exe 664 448 13 221
0 0 2013-07-16 12:24:34 UTC+0000
0xfffffa8003547060 svchost.exe 724 448 16 312
0 0 2013-07-16 12:24:34 UTC+0000
0xfffffa8003552b30 LogonUI.exe 744 412 8 157
1 0 2013-07-16 12:24:34 UTC+0000
0xfffffa8003572b30 svchost.exe 812 448 43 782
0 0 2013-07-16 12:24:34 UTC+0000
0xfffffa8003594b30 svchost.exe 856 448 14 234
0 0 2013-07-16 12:24:34 UTC+0000
0xfffffa800359b9b0 svchost.exe 900 448 8 128
0 0 2013-07-16 12:24:34 UTC+0000
0xfffffa80035b3060 svchost.exe 940 448 19 361
0 0 2013-07-16 12:24:34 UTC+0000
0xfffffa80035fcb30 svchost.exe 372 448 16 259
0 0 2013-07-16 12:24:35 UTC+0000
0xfffffa80035f6b30 spoolsv.exe 1048 448 8 89
0 0 2013-07-16 12:24:35 UTC+0000
0xfffffa8003679650 blnsvr.exe 1076 448 7 100
0 0 2013-07-16 12:24:35 UTC+0000
0xfffffa80035e5450 svchost.exe 1116 448 4 50
0 0 2013-07-16 12:24:35 UTC+0000
0xfffffa8003732b30 WmiPrvSE.exe 1364 584 15 294
0 0 2013-07-16 12:24:35 UTC+0000
0xfffffa8003767250 svchost.exe 1484 448 12 241
0 0 2013-07-16 12:24:35 UTC+0000
0xfffffa80037df620 WmiApSrv.exe 1684 448 7 112
0 0 2013-07-16 12:24:36 UTC+0000
0xfffffa80037a56c0 WmiPrvSE.exe 1716 584 7 105
0 0 2013-07-16 12:24:36 UTC+0000
0xfffffa8003763270 WmiPrvSE.exe 1764 584 7 175
0 0 2013-07-16 12:24:38 UTC+0000
VM memory size: 3588 MB
Determining profile based on KDBG search...
Suggested Profile(s) : Win2008R2SP0x64, Win7SP1x64,
Win7SP0x64, Win2008R2SP1x64
AS Layer1 : AMD64PagedMemory (Kernel AS)
AS Layer2 : FileAddressSpace
(/var/lib/libvirt/qemu/save/win-3588.ram)
PAE type : PAE
DTB : 0x187000L
KDBG : 0xf8000180e0a0
Number of Processors : 2
Image Type (Service Pack) : 1
KPCR for CPU 0 : 0xfffff8000180fd00L
KPCR for CPU 1 : 0xfffff880009b8000L
KUSER_SHARED_DATA : 0xfffff78000000000L
Image date and time : 2013-07-16 12:50:59 UTC+0000
Image local date and time : 2013-07-16 12:50:59 +0000
Offset(V) Name PID PPID Thds Hnds
Sess Wow64 Start Exit
------------------ -------------------- ------ ------ ------ --------
------ ------ ------------------------------
------------------------------
0xfffffa800308d9e0 System 4 0 68 275
------ 0 2013-07-16 12:50:55 UTC+0000
VM memory size: 3840 MB
Determining profile based on KDBG search...
Suggested Profile(s) : Win2008R2SP0x64, Win7SP1x64,
Win7SP0x64, Win2008R2SP1x64
AS Layer1 : AMD64PagedMemory (Kernel AS)
AS Layer2 : FileAddressSpace
(/var/lib/libvirt/qemu/save/win-3840.ram)
PAE type : PAE
DTB : 0x187000L
KDBG : 0xf800018400a0
Number of Processors : 1
Image Type (Service Pack) : 1
KPCR for CPU 0 : 0xfffff80001841d00L
KUSER_SHARED_DATA : 0xfffff78000000000L
Image date and time : 2013-07-16 12:28:55 UTC+0000
Image local date and time : 2013-07-16 12:28:55 +0000
Offset(V) Name PID PPID Thds Hnds
Sess Wow64 Start Exit
------------------ -------------------- ------ ------ ------ --------
------ ------ ------------------------------
------------------------------
0xfffffa80033849e0 System 4 0 72 --------
------ 0 2013-07-16 12:28:47 UTC+0000
vol-users,
During last year's OMFW, I gave a presentation on a new Volatility plugin
called dumpfiles[1]. This plugin automates the process of extracting both
memory mapped and cached files. While we have distributed early versions
of the plugin in the Volatility training classes, we are in the final
stages of testing for its inclusion in the upcoming 2.3 release. If you
have some cycles and interest in helping us test, please send me a note
off-list.
Thanks,
AW
PS: Special thanks to Ikelos, MHL, Gleeda, attc, and Carl Pulley for their
help with earlier versions!
[1] http://volatility-labs.blogspot.com/2012/10/movp-44-cache-rules-everything-…
We are writing to announce that our upcoming Amsterdam class is
completely sold out and that our next public training will be in
Reston, VA in November. We have a blog post on the Reston training
here:
http://volatility-labs.blogspot.com/2013/06/memory-forensics-training-resto…
This will be our last public training of 2013 and we are actively
planning 2014 trainings. These will be announced when they are ready.
Our last Reston class sold out a full month in advance so please
contact us ASAP if you would like to attend.
Volatility Community,
If you are planning to attend the Open Memory Forensics Workshop 2013, I
would suggest registering soon. We only have a limited number of seats
remaining:
https://www.volatilityfoundation.org/default/omfw
PS: The only way to reserve a seat is by emailing:
info [at] volatilesystems [dot] com
Thanks,
AAron Walters
The Volatility Foundation
