PA193 - Secure coding
principles and practices
Dynamic analysis, fuzzing
Petr Švenda svenda@fi.muni.cz
Overview
• Lecture:
– Dynamic analysis of programs for potential bugs
– Memory analysis
– Fuzzing (blackbox testing)
– Tools
• Labs
– Using fuzzers
2 | PA193 - Dynamic analysis, fuzzing
DYNAMIC ANALYSIS
3 | PA193 - Dynamic analysis, fuzzing
What can dynamic analysis provide
• Dynamic analysis compile and execute tested program
– real or virtualized processor
• Inputs are supplied and outputs are observed
– sufficient number of inputs needs to be supplied
– code coverage should be high
• Memory, function calls and executed operations can be
monitored and evaluated
– invalid access to memory (buffer overflow)
– memory leak or double free
– calls to potentially sensitive functions
• http://www.embedded.com/design/safety-and-security/4419779
4 | PA193 - Dynamic analysis, fuzzing
Techniques used by dynamic analysis
• Debugger (full control over memory read/write, even ops)
• Insert data into program input points (integration tests, fuzzing…)
– stdin, network, files…
• Insert manipulation proxy between program and library (dll stub, memory)
• Trace of program’s external behavior (linux strace)
• Change source code (instrumentation, logging…)
• Change of application binary
• Run in lightweight virtual machine (Valgrind)
• Run in full virtual machine
• Follow propagation of specified values (Taint analysis)
• Mocking (create additional input points into program)
• Restrict programs environment (low memory, limited file descriptors, limited
rights…)
5 | PA193 - Dynamic analysis, fuzzing
Dynamic analysis tools
• Commercial
– HP/Fortify, IBM Purify, Veracode, Coverity, Klocwork, Parasoft...
(together with static analysis)
• Free
– GCC gcov tool
– Valgrind – set of dynamic analysis features
– Fuzzers
• Most performance analyzers are dynamic analyzers
– MS Visual StudioAnalyzeStart performance analysis
– gcc -Wall -fprofile-arcs -ftest-coverage main.c
• List of tools for dynamic analysis
– https://en.wikipedia.org/wiki/Dynamic_program_analysis
6 | PA193 - Dynamic analysis, fuzzing
DEBUGGING SYMBOLS
7 | PA193 - Dynamic analysis, fuzzing
Release vs. Debug
• Optimizations applied (compiler-specific settings)
– gcc –Ox (http://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html)
• -O0 no optimization (Debug)
• -O1 –g / -Og debug-friendly optimization
• -O3 heavy optimization
– msvc /Ox /Oi (http://msdn.microsoft.com/en-us/library/k1ack8f1.aspx)
• MSVS2010: Project propertiesC/C++optimizations
• Availability of debug information (symbols)
– gcc –g
• symbols inside binary
– msvc /Z7, /Zi
• symbols in detached file ($projectname.pdb)
| PA193 - Dynamic analysis, fuzzing8
Stripping out debug symbols
• Debug symbols are of great help for an “attacker”
– key called NSAKey in ADVAPI.dll? (Crypto 1998)
– http://www.heise.de/tp/artikel/5/5263/1.html
• Always strip out debug symbols in released binary
– check compiler flag
– Linux: run file or objdump --syms command
(stripped/not stripped)
– Windows: DependencyWalker
9 | PA193 - Dynamic analysis, fuzzing
VALGRIND SUITE
10 | PA193 - Dynamic analysis, fuzzing
Valgrind http://www.valgrind.org/
• Suite of multiple tools (valgrind --tool=<toolname>)
• Memcheck - memory management dynamic analysis
– most commonly used tool (memory leaks)
– replaces standard C memory allocator with its own implementation and
check for memory leaks, corruption (additional guards blocks)...
– dangling pointers, unclosed file descriptors, uninitialized variables
– http://www.valgrind.org/docs/manual/mc-manual.html
• Massif – heap profiler
• Hellgrind - detection of concurrent issues (later presentation)
• Callgrind – generation of all graphs
• ...
11 | PA193 - Dynamic analysis, fuzzing
Valgrind – core options
• Compile with debug symbols
– gcc –std=c99 –Wall –g –o program program.c
– will allow for more context information in Valgrind report
• Run program with Valgrind attached
– valgrind <options> ./program
– program cmd line arguments (if any) can be passed
– valgrind -v --leak-check=full ./program arg1
• Trace also into sub-processed
– --trace-children=yes
– necessary for multi-process / threaded programs
• Display unclosed file descriptors
– --track-fds=yes
12 | PA193 - Dynamic analysis, fuzzing
Memcheck – memory leaks
• Detailed report of memory leaks checks
– --leak-check=full
• Memory leaks
– Definitely lost: memory is directly lost (no pointer exists)
– Indirectly lost: only pointers in lost memory points to it
– Possibly lost: address of memory exists somewhere, but
might be just randomly correct value (usually real leak)
13 | PA193 - Dynamic analysis, fuzzing
Memcheck – uninitialized values
• Detect usage of uninitialized variables
– -undef-value-errors=yes (default)
• Track from where initialized variable comes from
– --track-origins=yes
– introduces high performance overhead
14 | PA193 - Dynamic analysis, fuzzing
Memcheck – invalid reads/writes
• Writes outside allocated memory (buffer overflow)
• Only for memory located on heap!
– allocated via dynamic allocation (malloc, new)
• Will not detect problems on stack or static (global)
variables
– https://en.wikipedia.org/wiki/Valgrind#Limitations_of_Memcheck
• Writes into already de-allocated memory
– Valgrind tries to defer reallocation of freed memory as long
as possible to detect subsequent reads/writes here
15 | PA193 - Dynamic analysis, fuzzing
EXAMPLES OF ANALYSIS
16 | PA193 - Dynamic analysis, fuzzing
17 | PA193 - Dynamic analysis, fuzzing
#include <iostream>
int Static[5];
int memcheckFailDemo(int* arrayStack, unsigned int arrayStackLen,
int* arrayHeap, unsigned int arrayHeapLen) {
int Stack[5];
Static[100] = 0;
Stack[100] = 0;
for (int i = 0; i <= 5; i++) Stack [i] = 0;
int* array = new int[5];
array[100] = 0;
arrayStack[100] = 0;
arrayHeap[100] = 0;
for (unsigned int i = 0; i <= arrayStackLen; i++) {
arrayStack[i] = 0;
}
for (unsigned int i = 0; i <= arrayHeapLen; i++) {
arrayHeap[i] = 0;
}
return 0;
}
int main(void) {
int arrayStack[5];
int* arrayHeap = new int[5];
memcheckFailDemo(arrayStack, 5, arrayHeap, 5);
return 0;
18 | PA193 - Dynamic analysis, fuzzing
#include <iostream>
int Static[5];
int memcheckFailDemo(int* arrayStack, unsigned int arrayStackLen,
int* arrayHeap, unsigned int arrayHeapLen) {
int Stack[5];
Static[100] = 0; /* Error - Static[100] is out of bounds */
Stack[100] = 0; /* Error - Stack[100] is out of bounds */
for (int i = 0; i <= 5; i++) Stack [i] = 0; /* Error - for Stack[5] */
int* array = new int[5];
array[100] = 0; /* Error - array[100] is out of bounds */
arrayStack[100] = 0; /* Error - arrayStack[100] is out of bounds */
arrayHeap[100] = 0; /* Error - arrayHeap[100] is out of bounds */
for (unsigned int i = 0; i <= arrayStackLen; i++) { /* Error - off by one *
arrayStack[i] = 0;
}
for (unsigned int i = 0; i <= arrayHeapLen; i++) { /* Error - off by one */
arrayHeap[i] = 0;
}
/* Problem Memory leak – array */
return 0;
}
int main(void) {
int arrayStack[5];
int* arrayHeap = new int[5];
memcheckFailDemo(arrayStack, 5, arrayHeap,
return 0;
}
Problems detected – compile time
• g++ -ansi -Wall -Wextra -g -o test test.cpp
– clean compilation
• MSVC (Visual Studio 2012) /W4
– only one problem detected, Stack[100] = 0;
19 | PA193 - Dynamic analysis, fuzzing
test.cpp(56): error C4789: buffer 'Stack' of size 20 bytes will
be overrun; 4 bytes will be written starting at offset 400
20 | PA193 - Dynamic analysis, fuzzing
#include <iostream>
int Static[5];
int memcheckFailDemo(int* arrayStack, unsigned int arrayStackLen,
int* arrayHeap, unsigned int arrayHeapLen) {
int Stack[5];
Static[100] = 0; /* Error - Static[100] is out of bounds */
Stack[100] = 0; /* Error - Stack[100] is out of bounds */
for (int i = 0; i <= 5; i++) Stack [i] = 0; /* Error - for Stack[5] */
int* array = new int[5];
array[100] = 0; /* Error - array[100] is out of bounds */
arrayStack[100] = 0; /* Error - arrayStack[100] is out of bounds */
arrayHeap[100] = 0; /* Error - arrayHeap[100] is out of bounds */
for (unsigned int i = 0; i <= arrayStackLen; i++) { /* Error - off by one *
arrayStack[i] = 0;
}
for (unsigned int i = 0; i <= arrayHeapLen; i++) { /* Error - off by one */
arrayHeap[i] = 0;
}
/* Problem Memory leak – array */
return 0;
}
MSVC /W4
Visual Studio 2012 & GCC – runtime checks
• Corruption (usually) causes runtime exceptions
– Stack around variable ‘Stack’ was corrupted
– Stack around variable ‘arrayStack’ was corrupted
• MSVC: /RTC, /GS, /DYNAMICBASE (ASLR) and
/NXCOMPAT (DEP)
• GCC: -fstack-protector-all, --no_execstack (DEP),
kernel.randomize_va_space=1 (ASLR)
• May preventing successful exploit, but is only last defense
21 | PA193 - Dynamic analysis, fuzzing
Cppcheck --enable=all static.cpp
• (Some memory leaks also detected)
22 | PA193 - Dynamic analysis, fuzzing
[static.cpp:7]: (error) Array 'Static[5]' accessed at index 100, which is out of bounds.
[static.cpp:8]: (error) Array 'Stack[5]' accessed at index 100, which is out of bounds.
[static.cpp:10]: (error) Buffer is accessed out of bounds: Stack
[static.cpp:30] -> [static.cpp:15]: (error) Array 'arrayStack[5]' accessed at
index 100, which is out of bounds.
[static.cpp:13]: (error) Array 'array[5]' accessed at index 100, which is out of bounds.
[static.cpp:25]: (error) Memory leak: array
[static.cpp:31]: (error) Memory leak: arrayHeap
23 | PA193 - Dynamic analysis, fuzzing
#include <iostream>
int Static[5];
int memcheckFailDemo(int* arrayStack, unsigned int arrayStackLen,
int* arrayHeap, unsigned int arrayHeapLen) {
int Stack[5];
Static[100] = 0; /* Error - Static[100] is out of bounds */
Stack[100] = 0; /* Error - Stack[100] is out of bounds */
for (int i = 0; i <= 5; i++) Stack [i] = 0; /* Error - for Stack[5] */
int* array = new int[5];
array[100] = 0; /* Error - array[100] is out of bounds */
arrayStack[100] = 0; /* Error - arrayStack[100] is out of bounds */
arrayHeap[100] = 0; /* Error - arrayHeap[100] is out of bounds */
for (unsigned int i = 0; i <= arrayStackLen; i++) { /* Error - off by one *
arrayStack[i] = 0;
}
for (unsigned int i = 0; i <= arrayHeapLen; i++) { /* Error - off by one */
arrayHeap[i] = 0;
}
/* Problem Memory leak – array */
return 0;
}
Cppcheck --enable=all file.cpp
/* Not all memory leaks are caught! */
if (1 == 2) delete[] array; /* caught */
if (Stack[0] == 1) delete[] array; /* missed */
if (Stack[0] == 1) delete[] arrayHeap; /*-//-*/
Visual Studio 2012 & PREfast
• Additional two problems detected
– Static[100] = 0;
– for (int i = 0; i <= 5; i++) Stack [i] = 0;
• arrayStack and arrayHeap overruns still missed
24 | PA193 - Dynamic analysis, fuzzing
test.cpp(55): warning : C6200: Index '100' is out of valid index
range '0' to '4' for non-stack buffer 'int * Static'.
test.cpp(58): warning : C6201: Index '5' is out of valid index
range '0' to '4' for possibly stack allocated buffer 'Stack'.
test.cpp(55): warning : C6386: Buffer overrun while writing to 'Static':
the writable size is '20' bytes, but '404' bytes might be written.
test.cpp(62): warning : C6386: Buffer overrun while writing to 'array':
the writable size is '5*4' bytes, but '404' bytes might be written.
25 | PA193 - Dynamic analysis, fuzzing
#include <iostream>
int Static[5];
int memcheckFailDemo(int* arrayStack, unsigned int arrayStackLen,
int* arrayHeap, unsigned int arrayHeapLen) {
int Stack[5];
Static[100] = 0; /* Error - Static[100] is out of bounds */
Stack[100] = 0; /* Error - Stack[100] is out of bounds */
for (int i = 0; i <= 5; i++) Stack [i] = 0; /* Error - for Stack[5] */
int* array = new int[5];
array[100] = 0; /* Error - array[100] is out of bounds */
arrayStack[100] = 0; /* Error - arrayStack[100] is out of bounds */
arrayHeap[100] = 0; /* Error - arrayHeap[100] is out of bounds */
for (unsigned int i = 0; i <= arrayStackLen; i++) { /* Error - off by one *
arrayStack[i] = 0;
}
for (unsigned int i = 0; i <= arrayHeapLen; i++) { /* Error - off by one */
arrayHeap[i] = 0;
}
/* Problem Memory leak – array */
return 0;
}
Visual Studio 2012 & PREfast
Visual Studio 2012 & PREfast & SAL
26 | PA193 - Dynamic analysis, fuzzing
test.cpp(11): warning : C6200: Index '100' is out of valid index
range '0' to '4' for non-stack buffer 'int * Static'.
test.cpp(14): warning : C6201: Index '5' is out of valid index
range '0' to '4' for possibly stack allocated buffer 'Stack'.
test.cpp(11): warning : C6386: Buffer overrun while writing to 'Static':
the writable size is '20' bytes, but '404' bytes might be written.
test.cpp(17): warning : C6386: Buffer overrun while writing to 'array':
the writable size is '5*4' bytes, but '404' bytes might be written.
test.cpp(23): warning : C6386: Buffer overrun while writing to 'arrayStack':
the writable size is '_Old_2`arrayStackLen' bytes, but '8' bytes might be written.
test.cpp(26): warning : C6386: Buffer overrun while writing to 'arrayHeap':
the writable size is '_Old_2`arrayHeapLen' bytes, but '8' bytes might be written.
int memcheckFailDemo(
_Out_writes_bytes_all_(arrayStackLen) int* arrayStack,
unsigned int arrayStackLen,
_Out_writes_bytes_all_(arrayHeapLen) int* arrayHeap,
unsigned int arrayHeapLen);
27 | PA193 - Dynamic analysis, fuzzing
#include <iostream>
int Static[5];
int memcheckFailDemo(int* arrayStack, unsigned int arrayStackLen,
int* arrayHeap, unsigned int arrayHeapLen) {
int Stack[5];
Static[100] = 0; /* Error - Static[100] is out of bounds */
Stack[100] = 0; /* Error - Stack[100] is out of bounds */
for (int i = 0; i <= 5; i++) Stack [i] = 0; /* Error - for Stack[5] */
int* array = new int[5];
array[100] = 0; /* Error - array[100] is out of bounds */
arrayStack[100] = 0; /* Error - arrayStack[100] is out of bounds */
arrayHeap[100] = 0; /* Error - arrayHeap[100] is out of bounds */
for (unsigned int i = 0; i <= arrayStackLen; i++) { /* Error - off by one *
arrayStack[i] = 0;
}
for (unsigned int i = 0; i <= arrayHeapLen; i++) { /* Error - off by one */
arrayHeap[i] = 0;
}
/* Problem Memory leak – array */
return 0;
}
/* Error – still off by one, but not detected by SAL */
for (unsigned int i = 0; i < arrayStackLen + 1; i++) {
arrayStack[i] = 0;
}
Visual Studio 2012 & PREfast & SAL
28 | PA193 - Dynamic analysis, fuzzing
: valgrind --tool=memcheck ./test
== Invalid write of size 4
== at 0x4006AB: memcheckFailDemo(int*, unsigned int, int*, unsigned int) (test.cpp:14)
== by 0x40075D: main (test.cpp:33)
== Address 0x595f230 is not stack'd, malloc'd or (recently) free'd
==
== Invalid write of size 4
== at 0x4006CB: memcheckFailDemo(int*, unsigned int, int*, unsigned in t) (test.cpp:17)
== by 0x40075D: main (test.cpp:33)
== Address 0x595f1d0 is not stack'd, malloc'd or (recently) free'd
==
== Invalid write of size 4
== at 0x400710: memcheckFailDemo(int*, unsigned int, int*, unsigned int) (test.cpp:23)
== by 0x40075D: main (test.cpp:33)
== Address 0x595f054 is 0 bytes after a block of size 20 alloc'd
== at 0x4C28152: operator new[](unsigned long) (vg_replace_malloc.c:363)
== by 0x40073F: main (test.cpp:32)
== LEAK SUMMARY:
== definitely lost: 40 bytes in 2 blocks
== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 6 from 6)
Invalid write detected
(array[100] = 0;)
Memory leaks detected
(array, arrayHeap)
Valgrind --tool=memcheck
Invalid write detected
(arrayHeap[100] = 0;)
Invalid write detected
(arrayHeap[i] = 0;)
29 | PA193 - Dynamic analysis, fuzzing
#include <iostream>
int Static[5];
int memcheckFailDemo(int* arrayStack, unsigned int arrayStackLen,
int* arrayHeap, unsigned int arrayHeapLen) {
int Stack[5];
Static[100] = 0; /* Error - Static[100] is out of bounds */
Stack[100] = 0; /* Error - Stack[100] is out of bounds */
for (int i = 0; i <= 5; i++) Stack [i] = 0; /* Error - for Stack[5] */
int* array = new int[5];
array[100] = 0; /* Error - array[100] is out of bounds */
arrayStack[100] = 0; /* Error - arrayStack[100] is out of bounds */
arrayHeap[100] = 0; /* Error - arrayHeap[100] is out of bounds */
for (unsigned int i = 0; i <= arrayStackLen; i++) { /* Error - off by one *
arrayStack[i] = 0;
}
for (unsigned int i = 0; i <= arrayHeapLen; i++) { /* Error - off by one */
arrayHeap[i] = 0;
}
/* Problem Memory leak – array */
return 0;
}
Valgrind --tool=memcheck
Valgrind --tool=exp-sgcheck
30 | PA193 - Dynamic analysis, fuzzing
==15979== Invalid write of size 4
==15979== at 0x40067C: memcheckFailDemo(int*, unsigned int, int*,
unsigned int) (test.cpp:11)
==15979== by 0x40075D: main (test.cpp:33)
==15979== Address 0x7fefffe34 expected vs actual:
==15979== Expected: stack array "Stack" of size 20 in this frame
==15979== Actual: unknown
==15979== Actual: is 0 after Expected
==15979==
==15979== Invalid write of size 4
==15979== at 0x4006E5: memcheckFailDemo(int*, unsigned int, int*,
unsigned int) (test.cpp:20)
==15979== by 0x40075D: main (test.cpp:33)
==15979== Address 0x7fefffe74 expected vs actual:
==15979== Expected: stack array "arrayStack" of size 20 in frame 1 back from here
==15979== Actual: unknown
==15979== Actual: is 0 after Expected
==15979==
==15979==
==15979== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 28 from 28)
Invalid write detected
for (int i = 0; i <= 5; i++) Stack[i] = 0;
Invalid write detected
... arrayStack[i] = 0;
31 | PA193 - Dynamic analysis, fuzzing
#include <iostream>
int Static[5];
int memcheckFailDemo(int* arrayStack, unsigned int arrayStackLen,
int* arrayHeap, unsigned int arrayHeapLen) {
int Stack[5];
Static[100] = 0; /* Error - Static[100] is out of bounds */
Stack[100] = 0; /* Error - Stack[100] is out of bounds */
for (int i = 0; i <= 5; i++) Stack [i] = 0; /* Error - for Stack[5] */
int* array = new int[5];
array[100] = 0; /* Error - array[100] is out of bounds */
arrayStack[100] = 0; /* Error - arrayStack[100] is out of bounds */
arrayHeap[100] = 0; /* Error - arrayHeap[100] is out of bounds */
for (unsigned int i = 0; i <= arrayStackLen; i++) { /* Error - off by one *
arrayStack[i] = 0;
}
for (unsigned int i = 0; i <= arrayHeapLen; i++) { /* Error - off by one */
arrayHeap[i] = 0;
}
/* Problem Memory leak – array */
return 0;
}
Valgrind --tool=exp-sgcheck
(MSVS 2012) _CrtDumpMemoryLeaks();
32 | PA193 - Dynamic analysis, fuzzing
Detected memory leaks!
Dumping objects ->
{155} normal block at 0x00600AD0, 20 bytes long.
Data: < > CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD
{154} normal block at 0x00600A80, 20 bytes long.
Data: < > 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Object dump complete.
Tools - summary
• Compilers (MSVC, GCC) will miss many problems
• Compiler flags (/RTC and /GS; -fstack-protector-all) flags
– detect (some) stack based corruptions at runtime
– additional preventive flags /DYNAMICBASE (ASLR) and /NXCOMPAT (DEP)
• Valgrind memcheck
– will not find stack based problems, only heap corruptions (dynamic allocation)
• Valgrind exp-sgcheck
– will detect stack based problem, but miss first (possibly incorrect) access
• Cppcheck
– detect multiple problems (even memory leaks), but mostly limited to single function
• PREfast will find some stack based problems, limited to single function
• PREfast with SAL annotations will find additional stack and some heap
problems, but not all
33 | PA193 - Dynamic analysis, fuzzing
FUZZING (BLACKBOX)
34 | PA193 - Dynamic analysis, fuzzing
35 | PA193 - Dynamic analysis, fuzzing
What is wrong?
36 | PA193 - Dynamic analysis, fuzzing
Tag ‘ff fe’ + length of COM section
length of comment = length – 2;
strlen(“hello fuzzy world”) == ?
length of COM section == 00 00
length of comment = 0 – 2;
-2 == 0xFFFFFFFFFFFFFFFE == ~4GB
byte* pComment = new byte[MAX_SHORT];
memcpy(pComment, buffer, length);
MS04-028: Microsoft's JPEG GDI+ vulnerability (2004)
I love GDI+ vulnerability because…
• Lack of proper input checking
• Type signed/unsigned mismatch
• Type overflow
• Buffer overflow
• Heap overflow
• Source code was not available (blackbox testing)
• Huge impact (core MS library)
• Easily exploitable
38 | PA193 - Dynamic analysis, fuzzing
INTRO TO FUZZING
39 | PA193 - Dynamic analysis, fuzzing
Very simple fuzzer
cat /dev/random | ./target_app
40 | PA193 - Dynamic analysis, fuzzing
What do you miss here?
42 | PA193 - Dynamic analysis, fuzzing
http://iconarchive.com,
http://awicons.com,
http://www.pelfusion.com
1. Investigate app in/out 2. Prepare data model 3. Validate data model
4. Generate fuzzed inputs
5. Send fuzzed input to app
6. Monitor target app
7. Analyze logs
Fuzzing: key characteristics
1. More or less random modification of inputs
2. Monitoring of target application
3. Huge amount of inputs for target are send
4. Automated and repeatable
44 | PA193 - Dynamic analysis, fuzzing
Fuzzing - advantages/disadvantages
• Fuzzing advantages
– Very simple design
– Allow to find bugs missed by human eye
– Sometimes the only way to test (closed system)
– Repeatable (crash inputs stored)
• Fuzzing disadvantages
– Usually simpler bugs found (low hanging fruit)
– Increased difficulty to evaluate impact or dangerosity
– Closed system is often evaluated, black box testing
46 | PA193 - Dynamic analysis, fuzzing
What kind of bugs is usually found?
• Memory corruption bugs (buffer overflows...)
• Parser bugs (crash of parser on malformed input)
• Invalid error handling (other then expected error)
• Threading errors (requires sufficient setup)
• Correctness bugs (reference vs. new impl.)
48 | PA193 - Dynamic analysis, fuzzing
What kind of bugs are usually missed?
• Bugs after input validation (if not modeled properly)
• High-level / architecture bugs (e.g. weak crypto)
• Usability bugs
• …
49 | PA193 - Dynamic analysis, fuzzing
What kind of applications can be fuzzed?
• Any application/module with an input
– (sometimes even without inputs, e.g., fault induction)
• Custom (“DIY”) fuzzer
– Usually full knowledge about target app
– Kind of randomized integration test (but still repeatable!)
• File fuzzer – input via files
• Network fuzzer – input received via network
• General fuzzing framework
– Preprepared tools and functions for common tasks (file, packet…)
– Custom plugins, pre-prepared and custom data models
50 | PA193 - Dynamic analysis, fuzzing
Microsoft’s SDL MiniFuzz File Fuzzer
52 | PA193 - Dynamic analysis, fuzzing
53 | PA193 - Dynamic analysis, fuzzing
<?xml version="1.0"?>
<failures>
<failure type="Exception Event:Tid=8504, 0x80000003, unhandled, address=0x7740e34d" datetime="11:21:12 12.
<registers RAX="00000000" RBX="00000000" RCX="7FFF5FC5180A" RDX="00000000" RSI="00000000" RDI="000
<process name="C:\Program Files (x86)\IrfanView\i_view32.exe" />
<file name="-std=c99 -Wall C:\minifuzz\temp\beer-0rsw9!h2jf.jpg" />
</failure>
</failures>
MiniFuzz: gcc fuzzing
54 | PA193 - Dynamic analysis, fuzzing
#include<stdio.h>
int main() {
printf("Hello Fuzzy World");
return 0;
}
Binary fuzzing of source code???
How to improve test coverage?
What if file is not an input?
INVESTIGATE APPLICATION
55 | PA193 - Dynamic analysis, fuzzing
What kind of inputs and strategy?
• Type of inputs?
– File, network packets, structure, data model, state(-less)
• What environment setup is necessary?
– Fuzzing on live system?
– Multiple entities inside VMs? Networking?
• Isolated vs. cooperating components?
– We don’t like to mock everything
• What tools are readily available?
56 | PA193 - Dynamic analysis, fuzzing
MODELLING
59 | PA193 - Dynamic analysis, fuzzing
Input preparation
• Time intensive part of fuzzing (if model !exists yet)
1. Fully random data
2. Random modification of valid input
3. Modification of valid input with fuzz vectors
4. Modification of valid input with mutator
5. Fuzzing via intermediate proxy
60 | PA193 - Dynamic analysis, fuzzing
Radamsa fuzzer
• “…easy-to-set-up general purpose shotgun test to
expose the easiest cracks…”
– https://code.google.com/p/ouspg/wiki/Radamsa
• Just provide input files, all other settings automatic
– cat file | radamsa > file.fuzzed
61 | PA193 - Dynamic analysis, fuzzing
>echo "1 + (2 + (3 + 4))" | radamsa --seed 12 -n 4
1 + (2 + (2 + (3 + 4?)
1 + (2 + (3 +?4))
18446744073709551615 + 4)))
1 + (2 + (3 + 170141183460469231731687303715884105727))
How to generate fuzzed input?
• Generational fuzzing (Recursive fuzzing)
– Produces data based only on data model description
– E.g., iterates over range of values of given alphabet
• Mutational fuzzing (Replacive fuzzing)
– Produces data based on templates and supplied model
– Known border values or malicious malformed input
– Fuzz test vectors
– String-based mutators, number-based mutators…
62 | PA193 - Dynamic analysis, fuzzing
Fuzzing via intermediate proxy
• Fuzzer modifies valid flow according to data model
• Usually used for fuzzing of state-full protocols
– Modelling states and interactions would be difficult
– Target application(s) takes care of states and valid input
65 | PA193 - Dynamic analysis, fuzzing
OWASP’s ZAP – fuzz strategy settings
66 | PA193 - Dynamic analysis, fuzzing
APDUPlay - Smart card fuzzing
• Host to smart card communication done via PC/SC
• Custom winscard.dll stub written
• Manipulate incoming/outgoing APDUs
– modify packet content
– replay of previous packets
– …
| PA193 - Dynamic analysis, fuzzing
[RULE1]
MATCH1=in=1;t=0;cla=00;ins=a4;p1=04;
ACTION=in=0;data0=90 00;le=02;
00 a4 04 00 08 01 02 03 04 05 06 07 08
winscard.dll (stub)
90 00
67
http://www.fi.muni.cz/~xsvenda/apduinspect.html
VALIDATION
68 | PA193 - Dynamic analysis, fuzzing
Peach Validator 3.0
70 | PA193 - Dynamic analysis, fuzzing
Model doesn’t match valid input
American fuzzy lop
• New, but actively developed tool
• High speed fuzzer http://lcamtuf.coredump.cx/afl/
• Sophisticated generation of test cases (coverage)
• Automatic generation of input templates
– E.g., valid JPEG image from “hello” string after few days
– http://lcamtuf.blogspot.cz/2014/11/pulling-jpegs-out-of-thin-air.html
• Lots of real bugs found
71 | PA193 - Dynamic analysis, fuzzing
Test coverage
• Random inputs have low coverage (usually)
– Number of blocks visited in target binary
• Smart fuzzing tries to improve coverage
– Way how to generate new inputs from existing
• E.g., Peach’s minset tool
– Gather a lot of inputs (files)
– Run minset tool, traces with coverage stats are collected
– Minimal set of files to achieve coverage is computed
– Selected files are used as templates for fuzzing
72 | PA193 - Dynamic analysis, fuzzing
START, GENERATE, MONITOR
73 | PA193 - Dynamic analysis, fuzzing
How to detect “hit”?
• Application crash, uncaught exception…
– Clear faults, easy to detect
• Error returned
– Some errors are valid response
– Some errors are valid response only in selected states
• Input accepted even when it shouldn't be
– E.g., packet with incorrect checksum or modified field
• Some operation performed in incorrect state
– E.g., door open without proper authentication
• Application behavior is impaired
– E.g., response time significantly increases
• …
74 | PA193 - Dynamic analysis, fuzzing
Peach monitors
75 | PA193 - Dynamic analysis, fuzzing
ANALYZE
81 | PA193 - Dynamic analysis, fuzzing
What to do with hit results?
• Time intensive part of fuzzing
• Not all hits are relevant (at least at the beginning)
– Crashes by values not controllable by an attacker
– !exploitable https://msecdbg.codeplex.com/
• Hits reproduction
– Hit can be result of cumulative series of operations
• Many hits are duplicates
– Inputs are different, but hit caused in the same part of code
• (Automatic) Bucketing of hits
– E.g., Peach performs bucking based on signature of callstack
82 | PA193 - Dynamic analysis, fuzzing
Summary
• Fuzzers are cheap way to detect simpler bugs
– If you don’t use it, others will
• Try to find tool that fits your particular scenario
– Check activity of development, support
• Fuzzing frameworks can ease variety of setups
– But bit steaper learning curve
• If fuzzing will not find any bugs, check your model
• Try it!
85 | PA193 - Dynamic analysis, fuzzing
TAINT ANALYSIS
86 | PA193 - Dynamic analysis, fuzzing
Taint analysis
• Form of flow analysis
• Follow propagation of sensitive values inside
program
– e.g., user input that can be manipulated by an attacker
– find all parts of program where value can “reach”
• “Information flows from object x to object y,
denoted x→y , whenever information stored in x is
transferred to, object y.” D. Denning
• Native support in some languages (Ruby, Perl)
– But not C++/Java 
87 | PA193 - Dynamic analysis, fuzzing
Taint sources
• Files (*.pdf, *.doc, *.js, *.mp3...)
• User input (keyboard, mouse, touchscreen)
• Network traffic
• USB devices
• ...
• Every time there is information flow from value from
untrusted source to other object X, object X is tainted
– labeled as “tainted”
88 | PA193 - Dynamic analysis, fuzzing
Execution of sensitive operation
• Before sensitive operation (e.g., system()) is
executed with value, taint label is checked
– if value is tainted, alert is issued
• Untrusted data reaching privilege location is
detected
– can detect even unknown attacks
– (but sometimes we need to use user input)
89 | PA193 - Dynamic analysis, fuzzing
Taint analysis - tools
• Taintgrind
– http://www.cl.cam.ac.uk/~wmk26/taintgrind/
– additional module to Valgrind
– dynamic taint analyzer for C/C++
– output memory traces (information flows) already produced by
Valgrind
• Tanalysis
– http://code.google.com/p/tanalysis/
– static taint analyzer for C
– plugin for Frama-C http://frama-c.com/
• Read more about taint analysis
– http://users.ece.cmu.edu/~ejschwar/papers/oakland10.pdf
90 | PA193 - Dynamic analysis, fuzzing
Microsoft PREfast + Taint analysis
• Warning C6029 is issued when tainted value is
passed to parameter marked as [Post(Tainted=No)]
– without any checking (any condition statement)
• http://msdn.microsoft.com/en-
us/library/ms182047%28v=vs.100%29.aspx
91 | PA193 - Dynamic analysis, fuzzing
// C
#include <CodeAnalysis\SourceAnnotations.h>
void f([SA_Pre(Tainted=SA_Yes)] int c);
// C++
#include <CodeAnalysis\SourceAnnotations.h>
using namespace vc_attributes;
void f([Pre(Tainted=Yes)] int c);
Coverity taint analysis
• TAINTED_SCALAR
– http://blog.coverity.com/2014/04/18/coverity-heartbleed-
part-2/#.U1l4k2dOURo
– http://security.coverity.com/blog/2014/Apr/on-detecting-
heartbleed-with-static-analysis.html
92 | PA193 - Dynamic analysis, fuzzing
Conclusions
• Dynamic analyzers can profile application
– and find bugs not found by static analysis
• Fuzzing is “cheap” blackbox approach via
malformed inputs
93 | PA193 - Dynamic analysis, fuzzing
Questions
94 | PA193 - Dynamic analysis, fuzzing
References
• Some books available, but…
• Michael Eddington, Demystifying fuzzers
– Comparison of open-source tools, cost of adoption
– BlackHat 2009, https://www.blackhat.com/presentations/bh-usa-
09/EDDINGTON/BHUSA09-Eddington-DemystFuzzers-PAPER.pdf
– https://www.blackhat.com/presentations/bh-usa-
09/EDDINGTON/BHUSA09-Eddington-DemystFuzzers-SLIDES.pdf
– RSA Conference 2010 talk
https://www.youtube.com/watch?v=Bm3Mfndrl1Y
• OWASP fuzzing guidelines
– https://www.owasp.org/index.php/Fuzzing
– https://www.owasp.org/index.php/OWASP_Testing_Guide_Appendix_C:_F
uzz_Vectors
• Tutorials and research papers on fuzzing http://fuzzing.info/papers/
95 | PA193 - Dynamic analysis, fuzzing
Peach tutorials
• Basic usage against vulnserver
– http://rockfishsec.blogspot.ch/2014/01/fuzzing-vulnserver-
with-peach-3.html
• Advanced tutorial (ZIP format fuzzing) – very good
– http://www.flinkd.org/2011/07/fuzzing-with-peach-part-1/
• Tutorial for RAR fuzzing
– http://www.flinkd.org/2011/11/fuzzing-with-peach-part-2-
fixups-2/
96 | PA193 - Dynamic analysis, fuzzing
References
• MS post on Test coverage by fuzzing
– http://blogs.technet.com/b/srd/archive/2010/02/24/using-
code-coverage-to-improve-fuzzing-results.aspx
• Application and file fuzzing
– http://resources.infosecinstitute.com/application-and-file-
fuzzing/
• How I Learned to Stop Fuzzing and Find More Bugs
– https://www.defcon.org/images/defcon-15/dc15-
presentations/dc-15-west.pdf
97 | PA193 - Dynamic analysis, fuzzing
98 | PA193 - Dynamic analysis, fuzzing
99 | PA193 - Dynamic analysis, fuzzing
DYNAMIC ANALYSIS PROFILING
(WHITEBOX)
100 | PA193 - Dynamic analysis, fuzzing
| PA193 - Dynamic analysis, fuzzing
Automatic measurement - profiling
• Automatic tool to measure time and memory used
• “Time” spend in specific function
• How often a function is called
• Call tree
– what function called actual one
– based on real code execution (condition jumps)
• Many other statistics, depend on the tools
• Helps to focus and scope security analysis
101
| PA193 - Dynamic analysis, fuzzing
MS Visual Studio Profiler
• AnalyzeLaunch Performance Wizard
• Profiling method: CPU Sampling
– check periodically what is executed on CPU
– accurate, low overhead
• Profiling method: Instrumentation
– automatically inserts special accounting code
– will return exact function call counter
– (may affect performance timings a bit)
• additional code present
• May require admin privileges (will ask)
102
| PA193 - Dynamic analysis, fuzzing
MS VS Profiler – results (Summary)
• Where to start the optimization work?
103
| PA193 - Dynamic analysis, fuzzing
MS VS Profiler – results (Functions)
• Result given in number of sampling hits
– meaningful result is % of total time spend in function
• Inclusive sampling
– samples hit in function or its children
– aggregate over call stack for given function
• Exclusive sampling
– samples hit in exclusively in given function
– usually what you want
• fraction of time spend in function code (not in
subfunctions)
104
| PA193 - Dynamic analysis, fuzzing
MS VS Profiler – results (Functions)
Doubleclick to move into
Function Details view
105
GCC gcov tool
• http://gcc.gnu.org/onlinedocs/gcc/Gcov.html#Gcov
1. Compile program by GCC with additional flags
– gcc -Wall -fprofile-arcs -ftest-coverage main.c
– gcc -Wall --coverage main.c
– additional monitoring code is added to binary
2. Execute program
– files with “.bb" ".bbg" and ".da" extension are created
3. Analyze resulting files with gcov
– gcov main.c
– annotated source code is created
• Lcov - graphical front-end for gcov
– http://ltp.sourceforge.net/coverage/lcov.php
106 | PA193 - Dynamic analysis, fuzzing
107 | PA193 - Dynamic analysis, fuzzing
Taken from http://ltp.sourceforge.net/coverage/lcov/output/example/methods/iterate.c.gcov.html