P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\titulka.jpg PA193 - Secure coding principles and practices Dynamic analysis, fuzzing •Petr Švenda • svenda@fi.muni.cz @rngsec P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Overview •Lecture: –Dynamic analysis of programs for potential bugs –Memory analysis –Fuzzing (blackbox testing) –Tools •Labs –No labs this week because of national holidays –Slides/tools available – you can learn anyway (fuzzers) 2 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg DYNAMIC ANALYSIS • 3 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg What can dynamic analysis provide •Dynamic analysis compile and execute tested program –real or virtualized processor •Inputs are supplied and outputs are observed –sufficient number of inputs needs to be supplied –code coverage should be high •Memory, function calls and executed operations can be monitored and evaluated –invalid access to memory (buffer overflow) –memory leak or double free –calls to potentially sensitive functions •http://www.embedded.com/design/safety-and-security/4419779 – – 4 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Techniques used by dynamic analysis •Debugger (full control over memory read/write, even ops) •Insert data into program input points (integration tests, fuzzing…) –stdin, network, files… •Insert manipulation proxy between program and library (dll stub, memory) •Trace of program’s external behavior (linux strace) •Change source code (instrumentation, logging…) •Change of application binary •Run in lightweight virtual machine (Valgrind) •Run in full virtual machine •Follow propagation of specified values (Taint analysis) •Mocking (create additional input points into program) •Restrict programs environment (low memory, limited file descriptors, limited rights…) • 5 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Dynamic analysis tools •Commercial –HP/Fortify, IBM Purify, Veracode, Coverity, Klocwork, Parasoft... (together with static analysis) •Free –GCC gcov tool –Valgrind – set of dynamic analysis features –Fuzzers •Most performance analyzers are dynamic analyzers –MS Visual Studio®Analyze®Start performance analysis –gcc -Wall -fprofile-arcs -ftest-coverage main.c •List of tools for dynamic analysis –https://en.wikipedia.org/wiki/Dynamic_program_analysis – • 6 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg DEBUGGING SYMBOLS • 7 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Release vs. Debug •Optimizations applied (compiler-specific settings) –gcc –Ox (http://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html) •-O0 no optimization (Debug) •-O1 –g / -Og debug-friendly optimization •-O3 heavy optimization –msvc /Ox /Oi (http://msdn.microsoft.com/en-us/library/k1ack8f1.aspx) •MSVS2010: Project properties®C/C++®optimizations •Availability of debug information (symbols) –gcc –g •symbols inside binary –msvc /Z7, /Zi •symbols in detached file ($projectname.pdb) • • •| PA193 - Dynamic analysis, fuzzing 8 P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Stripping out debug symbols •Debug symbols are of great help for an “attacker” –key called NSAKey in ADVAPI.dll? (Crypto 1998) –http://www.heise.de/tp/artikel/5/5263/1.html •Always strip out debug symbols in released binary –check compiler flag –Linux: run file or objdump --syms command (stripped/not stripped) –Windows: DependencyWalker 9 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg VALGRIND SUITE • 10 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Valgrind http://www.valgrind.org/ •Suite of multiple tools (valgrind --tool=) •Memcheck - memory management dynamic analysis –most commonly used tool (memory leaks) –replaces standard C memory allocator with its own implementation and check for memory leaks, corruption (additional guards blocks)... –dangling pointers, unclosed file descriptors, uninitialized variables –http://www.valgrind.org/docs/manual/mc-manual.html •Massif – heap profiler •Hellgrind - detection of concurrent issues •Callgrind – generation of all graphs •... 11 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Valgrind – core options •Compile with debug symbols –gcc –std=c99 –Wall –g –o program program.c –will allow for more context information in Valgrind report •Run program with Valgrind attached –valgrind ./program –program cmd line arguments (if any) can be passed –valgrind -v --leak-check=full ./program arg1 •Trace also into sub-processed –--trace-children=yes –necessary for multi-process / threaded programs •Display unclosed file descriptors –--track-fds=yes 12 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Memcheck – memory leaks •Detailed report of memory leaks checks –--leak-check=full •Memory leaks –Definitely lost: memory is directly lost (no pointer exists) –Indirectly lost: only pointers in lost memory points to it –Possibly lost: address of memory exists somewhere, but might be just randomly correct value (usually real leak) • • 13 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Memcheck – uninitialized values •Detect usage of uninitialized variables –-undef-value-errors=yes (default) •Track from where initialized variable comes from –--track-origins=yes –introduces high performance overhead 14 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Memcheck – invalid reads/writes •Writes outside allocated memory (buffer overflow) •Only for memory located on heap! –allocated via dynamic allocation (malloc, new) •Will not detect problems on stack or static (global) variables –https://en.wikipedia.org/wiki/Valgrind#Limitations_of_Memcheck •Writes into already de-allocated memory –Valgrind tries to defer reallocation of freed memory as long as possible to detect subsequent reads/writes here – 15 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg EXAMPLES OF ANALYSIS • 16 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg • 17 | PA193 - Dynamic analysis, fuzzing #include int Static[5]; int memcheckFailDemo(int* arrayStack, unsigned int arrayStackLen, int* arrayHeap, unsigned int arrayHeapLen) { int Stack[5]; Static[100] = 0; Stack[100] = 0; for (int i = 0; i <= 5; i++) Stack [i] = 0; int* array = new int[5]; array[100] = 0; arrayStack[100] = 0; arrayHeap[100] = 0; for (unsigned int i = 0; i <= arrayStackLen; i++) { arrayStack[i] = 0; } for (unsigned int i = 0; i <= arrayHeapLen; i++) { arrayHeap[i] = 0; } return 0; } int main(void) { int arrayStack[5]; int* arrayHeap = new int[5]; memcheckFailDemo(arrayStack, 5, arrayHeap, 5); return 0; } P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg • 18 | PA193 - Dynamic analysis, fuzzing #include int Static[5]; int memcheckFailDemo(int* arrayStack, unsigned int arrayStackLen, int* arrayHeap, unsigned int arrayHeapLen) { int Stack[5]; Static[100] = 0; /* Error - Static[100] is out of bounds */ Stack[100] = 0; /* Error - Stack[100] is out of bounds */ for (int i = 0; i <= 5; i++) Stack [i] = 0; /* Error - for Stack[5] */ int* array = new int[5]; array[100] = 0; /* Error - array[100] is out of bounds */ arrayStack[100] = 0; /* Error - arrayStack[100] is out of bounds */ arrayHeap[100] = 0; /* Error - arrayHeap[100] is out of bounds */ for (unsigned int i = 0; i <= arrayStackLen; i++) { /* Error - off by one */ arrayStack[i] = 0; } for (unsigned int i = 0; i <= arrayHeapLen; i++) { /* Error - off by one */ arrayHeap[i] = 0; } /* Problem Memory leak – array */ return 0; } int main(void) { int arrayStack[5]; int* arrayHeap = new int[5]; memcheckFailDemo(arrayStack, 5, arrayHeap, 5); return 0; } P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Problems detected – compile time •g++ -ansi -Wall -Wextra -g -o test test.cpp –clean compilation • •MSVC (Visual Studio 2012) /W4 –only one problem detected, Stack[100] = 0; – – •MSVC (Visual Studio 2015) /W4 –No problem reported – – – • – 19 | PA193 - Dynamic analysis, fuzzing test.cpp(56): error C4789: buffer 'Stack' of size 20 bytes will be overrun; 4 bytes will be written starting at offset 400 P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg • 20 | PA193 - Dynamic analysis, fuzzing #include int Static[5]; int memcheckFailDemo(int* arrayStack, unsigned int arrayStackLen, int* arrayHeap, unsigned int arrayHeapLen) { int Stack[5]; Static[100] = 0; /* Error - Static[100] is out of bounds */ Stack[100] = 0; /* Error - Stack[100] is out of bounds */ for (int i = 0; i <= 5; i++) Stack [i] = 0; /* Error - for Stack[5] */ int* array = new int[5]; array[100] = 0; /* Error - array[100] is out of bounds */ arrayStack[100] = 0; /* Error - arrayStack[100] is out of bounds */ arrayHeap[100] = 0; /* Error - arrayHeap[100] is out of bounds */ for (unsigned int i = 0; i <= arrayStackLen; i++) { /* Error - off by one */ arrayStack[i] = 0; } for (unsigned int i = 0; i <= arrayHeapLen; i++) { /* Error - off by one */ arrayHeap[i] = 0; } /* Problem Memory leak – array */ return 0; } •MSVC /W4 P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Visual Studio 2012 & GCC – runtime checks •Corruption (usually) causes runtime exceptions –Stack around variable ‘Stack’ was corrupted –Stack around variable ‘arrayStack’ was corrupted •MSVC: /RTC, /GS, /DYNAMICBASE (ASLR) and /NXCOMPAT (DEP) •GCC: -fstack-protector-all, --no_execstack (DEP), kernel.randomize_va_space=1 (ASLR) • •May preventing successful exploit, but is only last defense – 21 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Cppcheck --enable=all static.cpp • • • • • • •(Some memory leaks also detected) 22 | PA193 - Dynamic analysis, fuzzing D:\CppcheckPortable_128.png [static.cpp:7]: (error) Array 'Static[5]' accessed at index 100, which is out of bounds. [static.cpp:8]: (error) Array 'Stack[5]' accessed at index 100, which is out of bounds. [static.cpp:10]: (error) Buffer is accessed out of bounds: Stack [static.cpp:30] -> [static.cpp:15]: (error) Array 'arrayStack[5]' accessed at index 100, which is out of bounds. [static.cpp:13]: (error) Array 'array[5]' accessed at index 100, which is out of bounds. [static.cpp:25]: (error) Memory leak: array [static.cpp:31]: (error) Memory leak: arrayHeap P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg • 23 | PA193 - Dynamic analysis, fuzzing #include int Static[5]; int memcheckFailDemo(int* arrayStack, unsigned int arrayStackLen, int* arrayHeap, unsigned int arrayHeapLen) { int Stack[5]; Static[100] = 0; /* Error - Static[100] is out of bounds */ Stack[100] = 0; /* Error - Stack[100] is out of bounds */ for (int i = 0; i <= 5; i++) Stack [i] = 0; /* Error - for Stack[5] */ int* array = new int[5]; array[100] = 0; /* Error - array[100] is out of bounds */ arrayStack[100] = 0; /* Error - arrayStack[100] is out of bounds */ arrayHeap[100] = 0; /* Error - arrayHeap[100] is out of bounds */ for (unsigned int i = 0; i <= arrayStackLen; i++) { /* Error - off by one */ arrayStack[i] = 0; } for (unsigned int i = 0; i <= arrayHeapLen; i++) { /* Error - off by one */ arrayHeap[i] = 0; } /* Problem Memory leak – array */ return 0; } •Cppcheck --enable=all file.cpp /* Not all memory leaks are caught! */ if (1 == 2) delete[] array; /* caught */ if (Stack[0] == 1) delete[] array; /* missed */ if (Stack[0] == 1) delete[] arrayHeap; /*-//-*/ P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Visual Studio 2012 & PREfast •Additional two problems detected –Static[100] = 0; –for (int i = 0; i <= 5; i++) Stack [i] = 0; – – – – – – •arrayStack and arrayHeap overruns still missed 24 | PA193 - Dynamic analysis, fuzzing test.cpp(55): warning : C6200: Index '100' is out of valid index range '0' to '4' for non-stack buffer 'int * Static'. test.cpp(58): warning : C6201: Index '5' is out of valid index range '0' to '4' for possibly stack allocated buffer 'Stack'. test.cpp(55): warning : C6386: Buffer overrun while writing to 'Static': the writable size is '20' bytes, but '404' bytes might be written. test.cpp(62): warning : C6386: Buffer overrun while writing to 'array': the writable size is '5*4' bytes, but '404' bytes might be written. P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg • 25 | PA193 - Dynamic analysis, fuzzing #include int Static[5]; int memcheckFailDemo(int* arrayStack, unsigned int arrayStackLen, int* arrayHeap, unsigned int arrayHeapLen) { int Stack[5]; Static[100] = 0; /* Error - Static[100] is out of bounds */ Stack[100] = 0; /* Error - Stack[100] is out of bounds */ for (int i = 0; i <= 5; i++) Stack [i] = 0; /* Error - for Stack[5] */ int* array = new int[5]; array[100] = 0; /* Error - array[100] is out of bounds */ arrayStack[100] = 0; /* Error - arrayStack[100] is out of bounds */ arrayHeap[100] = 0; /* Error - arrayHeap[100] is out of bounds */ for (unsigned int i = 0; i <= arrayStackLen; i++) { /* Error - off by one */ arrayStack[i] = 0; } for (unsigned int i = 0; i <= arrayHeapLen; i++) { /* Error - off by one */ arrayHeap[i] = 0; } /* Problem Memory leak – array */ return 0; } •Visual Studio 2012 & PREfast P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Visual Studio 2012 & PREfast & SAL • 26 | PA193 - Dynamic analysis, fuzzing test.cpp(11): warning : C6200: Index '100' is out of valid index range '0' to '4' for non-stack buffer 'int * Static'. test.cpp(14): warning : C6201: Index '5' is out of valid index range '0' to '4' for possibly stack allocated buffer 'Stack'. test.cpp(11): warning : C6386: Buffer overrun while writing to 'Static': the writable size is '20' bytes, but '404' bytes might be written. test.cpp(17): warning : C6386: Buffer overrun while writing to 'array': the writable size is '5*4' bytes, but '404' bytes might be written. test.cpp(23): warning : C6386: Buffer overrun while writing to 'arrayStack': the writable size is '_Old_2`arrayStackLen' bytes, but '8' bytes might be written. test.cpp(26): warning : C6386: Buffer overrun while writing to 'arrayHeap': the writable size is '_Old_2`arrayHeapLen' bytes, but '8' bytes might be written. int memcheckFailDemo( _Out_writes_bytes_all_(arrayStackLen) int* arrayStack, unsigned int arrayStackLen, _Out_writes_bytes_all_(arrayHeapLen) int* arrayHeap, unsigned int arrayHeapLen); P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg • 27 | PA193 - Dynamic analysis, fuzzing #include int Static[5]; int memcheckFailDemo(int* arrayStack, unsigned int arrayStackLen, int* arrayHeap, unsigned int arrayHeapLen) { int Stack[5]; Static[100] = 0; /* Error - Static[100] is out of bounds */ Stack[100] = 0; /* Error - Stack[100] is out of bounds */ for (int i = 0; i <= 5; i++) Stack [i] = 0; /* Error - for Stack[5] */ int* array = new int[5]; array[100] = 0; /* Error - array[100] is out of bounds */ arrayStack[100] = 0; /* Error - arrayStack[100] is out of bounds */ arrayHeap[100] = 0; /* Error - arrayHeap[100] is out of bounds */ for (unsigned int i = 0; i <= arrayStackLen; i++) { /* Error - off by one */ arrayStack[i] = 0; } for (unsigned int i = 0; i <= arrayHeapLen; i++) { /* Error - off by one */ arrayHeap[i] = 0; } /* Problem Memory leak – array */ return 0; } /* Error – still off by one, but not detected by SAL */ for (unsigned int i = 0; i < arrayStackLen + 1; i++) { arrayStack[i] = 0; } •Visual Studio 2012 & PREfast & SAL P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg • 28 | PA193 - Dynamic analysis, fuzzing •Invalid write detected •(array[100] = 0;) •Memory leaks detected •(array, arrayHeap) Valgrind --tool=memcheck •Invalid write detected •(arrayHeap[100] = 0;) •Invalid write detected •(arrayHeap[i] = 0;) P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg • 29 | PA193 - Dynamic analysis, fuzzing #include int Static[5]; int memcheckFailDemo(int* arrayStack, unsigned int arrayStackLen, int* arrayHeap, unsigned int arrayHeapLen) { int Stack[5]; Static[100] = 0; /* Error - Static[100] is out of bounds */ Stack[100] = 0; /* Error - Stack[100] is out of bounds */ for (int i = 0; i <= 5; i++) Stack [i] = 0; /* Error - for Stack[5] */ int* array = new int[5]; array[100] = 0; /* Error - array[100] is out of bounds */ arrayStack[100] = 0; /* Error - arrayStack[100] is out of bounds */ arrayHeap[100] = 0; /* Error - arrayHeap[100] is out of bounds */ for (unsigned int i = 0; i <= arrayStackLen; i++) { /* Error - off by one */ arrayStack[i] = 0; } for (unsigned int i = 0; i <= arrayHeapLen; i++) { /* Error - off by one */ arrayHeap[i] = 0; } /* Problem Memory leak – array */ return 0; } •Valgrind --tool=memcheck P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Valgrind --tool=exp-sgcheck • 30 | PA193 - Dynamic analysis, fuzzing ==15979== Invalid write of size 4 ==15979== at 0x40067C: memcheckFailDemo(int*, unsigned int, int*, unsigned int) (test.cpp:11) ==15979== by 0x40075D: main (test.cpp:33) ==15979== Address 0x7fefffe34 expected vs actual: ==15979== Expected: stack array "Stack" of size 20 in this frame ==15979== Actual: unknown ==15979== Actual: is 0 after Expected ==15979== ==15979== Invalid write of size 4 ==15979== at 0x4006E5: memcheckFailDemo(int*, unsigned int, int*, unsigned int) (test.cpp:20) ==15979== by 0x40075D: main (test.cpp:33) ==15979== Address 0x7fefffe74 expected vs actual: ==15979== Expected: stack array "arrayStack" of size 20 in frame 1 back from here ==15979== Actual: unknown ==15979== Actual: is 0 after Expected ==15979== ==15979== ==15979== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 28 from 28) •Invalid write detected •for (int i = 0; i <= 5; i++) Stack[i] = 0; •Invalid write detected •... arrayStack[i] = 0; P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg • 31 | PA193 - Dynamic analysis, fuzzing #include int Static[5]; int memcheckFailDemo(int* arrayStack, unsigned int arrayStackLen, int* arrayHeap, unsigned int arrayHeapLen) { int Stack[5]; Static[100] = 0; /* Error - Static[100] is out of bounds */ Stack[100] = 0; /* Error - Stack[100] is out of bounds */ for (int i = 0; i <= 5; i++) Stack [i] = 0; /* Error - for Stack[5] */ int* array = new int[5]; array[100] = 0; /* Error - array[100] is out of bounds */ arrayStack[100] = 0; /* Error - arrayStack[100] is out of bounds */ arrayHeap[100] = 0; /* Error - arrayHeap[100] is out of bounds */ for (unsigned int i = 0; i <= arrayStackLen; i++) { /* Error - off by one */ arrayStack[i] = 0; } for (unsigned int i = 0; i <= arrayHeapLen; i++) { /* Error - off by one */ arrayHeap[i] = 0; } /* Problem Memory leak – array */ return 0; } •Valgrind --tool=exp-sgcheck P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg (MSVS 2012) _CrtDumpMemoryLeaks(); • 32 | PA193 - Dynamic analysis, fuzzing Detected memory leaks! Dumping objects -> {155} normal block at 0x00600AD0, 20 bytes long. Data: < > CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD {154} normal block at 0x00600A80, 20 bytes long. Data: < > 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 Object dump complete. P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Tools - summary •Compilers (MSVC, GCC) will miss many problems •Compiler flags (/RTC and /GS; -fstack-protector-all) flags –detect (some) stack based corruptions at runtime –additional preventive flags /DYNAMICBASE (ASLR) and /NXCOMPAT (DEP) •Valgrind memcheck –will not find stack based problems, only heap corruptions (dynamic allocation) •Valgrind exp-sgcheck –will detect stack based problem, but miss first (possibly incorrect) access •Cppcheck –detect multiple problems (even memory leaks), but mostly limited to single function •PREfast will find some stack based problems, limited to single function •PREfast with SAL annotations will find additional stack and some heap problems, but not all 33 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg FUZZING (BLACKBOX) • 34 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg • 35 | PA193 - Dynamic analysis, fuzzing C:\Picts\fuzzing\beer_fuzzed.jpg P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg What is wrong? • 36 | PA193 - Dynamic analysis, fuzzing Tag ‘ff fe’ + length of COM section length of comment = length – 2; strlen(“hello fuzzy world”) == ? length of COM section == 00 00 length of comment = 0 – 2; -2 == 0xFFFFFFFFFFFFFFFE == ~4GB > byte* pComment = new byte[MAX_SHORT]; memcpy(pComment, buffer, length); MS04-028: Microsoft's JPEG GDI+ vulnerability (2004) P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Microsoft's JPEG GDI+ vulnerability (2004) •Problem how GDI+ handles comments in JPEG picture •Comment header segment starts with marker 0xFFFE followed by 16bits length of comment •Length of comment is including the length word itself (2 bytes) •Subtract 2 bytes (length of length) to obtain comment length •If length of comment field is maliciously 0 or 1 then –0 – 2 == -2 length of comment as signed integer –converted to unsigned integer 0xFFFFFFFFFFFFFFFE –Comment with length about 4GB instead 65kB max •Buffer overflow when comment is copied into heap buffer – –http://technet.microsoft.com/en-us/security/bulletin/ms04-028 •http://www.slideshare.net/ashishmalik10/microsoft-gdi-jpeg-integer-underflow-vulnerability •http://www.securityfocus.com/bid/11173/exploit • • • 37 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg I love GDI+ vulnerability because… •Lack of proper input checking •Type signed/unsigned mismatch •Type overflow •Buffer overflow •Heap overflow •Source code was not available (blackbox testing) •Huge impact (core MS library) •Easily exploitable • 38 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg INTRO TO FUZZING • 39 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Very simple fuzzer • • cat /dev/random | ./target_app • 40 | PA193 - Dynamic analysis, fuzzing D:\Documents\Obrázky\question.png What do you miss here? P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg What is missing? •Where fuzzing fits in development process? (developer side, CI, SDL) •What type of bugs fuzzing tends to find? •Which apps can be fuzzed? •How to detect that app mishandled fuzzed input (“hit”)? (crash, signal, exception, error…) •How to react on detected “hit”? (save seed and crashing inputs, bucketing of inputs) •How to create more meaningful inputs then random bytes? (valid inputs, proxy) •How to fuzz non-binary inputs? (string patterns, regexpr, mouse movements…) •How to fuzz applications without input as files? (http requests, dll injection, ZAP example) •How to fuzz efficiently? (known problematic values (fuzz vectors)) •How to fuzz files/inputs with defined structure? (grammar, example Peach) •How to make fuzzer protocol-aware? (Peach example) •How to fuzz state-full protocols? (proxy like fuzzing) •How to analyse and react on detected hits? •Which tools one can use? •How to detect less visible “hits”? (side-channels) •What else can we fuzz? (test coverage testing, DDOS resiliency, hardware inputs) • 41 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg D:\Documents\Obrazky\fuzzing\gif_icon.png 42 | PA193 - Dynamic analysis, fuzzing D:\Documents\Obrazky\fuzzing\Apps-system-software-update-icon.png http://iconarchive.com, http://awicons.com, http://www.pelfusion.com D:\Documents\Obrazky\fuzzing\search-icon.png D:\Documents\Obrazky\fuzzing\search-icon.png D:\Documents\Obrazky\fuzzing\gif_icon_fuzzed1.png D:\Documents\Obrazky\fuzzing\gif_icon_fuzz3.png D:\Documents\Obrazky\fuzzing\gif_icon_fuzz2.png D:\Documents\Obrazky\fuzzing\Apps-utilities-system-monitor-icon.png D:\Documents\Obrazky\fuzzing\Apps-system-software-update-icon.png D:\Documents\Obrazky\fuzzing\gif_binary.png D:\Documents\Obrazky\fuzzing\gif_data_model.png D:\Documents\Obrazky\fuzzing\log-icon.png D:\Documents\Obrazky\fuzzing\machine_icon.png D:\Documents\Obrazky\fuzzing\gif_icon.png D:\Documents\Obrazky\fuzzing\bullet_hole.png D:\Documents\Obrazky\fuzzing\search-icon.png 1. Investigate app in/out 2. Prepare data model (optional) 3. Validate data model 4. Generate fuzzed inputs 5. Send fuzzed input to app 6. Monitor target app 7. Analyze logs P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Fuzzing process 1.Investigate input and output of target application 2.Prepare model for fuzzed input generation 3.Validate your model against reality 4.Start sending fuzzed inputs to application 5.Monitor application for faults, errors, crashes… 6.Analyze results 7.Mitigate problems found 43 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Fuzzing: key characteristics 1.More or less random modification of inputs 2.Monitoring of target application 3.Huge amount of inputs for target are send 4.Automated and repeatable 5. 44 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Why to fuzz? •Black-box negative testing software technique •Good cost per bug ratio •“Research” fuzzing –Dedicated effort, e.g., pen-testing, QA… –Mostly human analysis of results (with help of tools) –Whole process not necessary fully automated •Fuzzing as part of Secure Development Lifecycle –Automated run “every day” (CI) –Limited time to complete (parallelization?) –Automated analysis (no human involved) 45 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Fuzzing - advantages/disadvantages •Fuzzing advantages –Very simple design –Allow to find bugs missed by human eye –Sometimes the only way to test (closed system) –Repeatable (crash inputs stored) •Fuzzing disadvantages –Usually simpler bugs found (low hanging fruit) –Increased difficulty to evaluate impact or dangerosity –Closed system is often evaluated, black box testing 46 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg What type of bugs fuzzing tends to find? •Fuzzers tend to find simpler bugs • •Success of fuzzing depends on input structure –E.g., checksums prevents large parts of random fuzzing –Bugs in format parsing vs. bugs in data interpretation • •Success increases with time spend on modelling –More protocol-aware fuzzer is, problems are found faster (and less weird ones) – • 47 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg What kind of bugs is usually found? •Memory corruption bugs (buffer overflows...) •Parser bugs (crash of parser on malformed input) •Invalid error handling (other then expected error) •Threading errors (requires sufficient setup) •Correctness bugs (reference vs. new impl.) • 48 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg What kind of bugs are usually missed? •Bugs after input validation (if not modeled properly) •High-level / architecture bugs (e.g. weak crypto) •Usability bugs •… • 49 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg What kind of applications can be fuzzed? •Any application/module with an input –(sometimes even without inputs, e.g., fault induction) •Custom (“DIY”) fuzzer –Usually full knowledge about target app –Kind of randomized integration test (but still repeatable!) •File fuzzer – input via files •Network fuzzer – input received via network •General fuzzing framework –Preprepared tools and functions for common tasks (file, packet…) –Custom plugins, pre-prepared and custom data models • – 50 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg D:\Documents\Obrázky\services_icon_full_bw5.jpg Microsoft’s SDL MiniFuzz File Fuzzer •Application input files fuzzer –http://www.microsoft.com/en-us/download/details.aspx?id=21769 •Templates for valid input files (multiple) •Modify valid input file (randomly, % aggressiveness) •Run application with partially modified inputs •Log resulting crash (if happen) –exception, CPU registers... •Can be incorporated into Visual Studio, part of SDL •Video overview –http://msdn.microsoft.com/en-us/security/gg675011.aspx – 51 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Microsoft’s SDL MiniFuzz File Fuzzer • 52 | PA193 - Dynamic analysis, fuzzing D:\Documents\Obrazky\fuzzing\minifuzz_settings_irfan.png D:\Documents\Obrázky\services_icon_full_bw5.jpg P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg • 53 | PA193 - Dynamic analysis, fuzzing D:\Documents\Obrazky\fuzzing\minifuzz_irfan.png P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg MiniFuzz: gcc fuzzing • 54 | PA193 - Dynamic analysis, fuzzing D:\Documents\Obrazky\fuzzing\minifuzz.png #include int main() { printf("Hello Fuzzy World"); return 0; } > Binary fuzzing of source code??? How to improve test coverage? What if file is not an input? D:\Documents\Obrázky\question.png D:\Documents\Obrázky\services_icon_full_bw5.jpg P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg INVESTIGATE APPLICATION • 55 | PA193 - Dynamic analysis, fuzzing D:\Documents\Obrazky\fuzzing\process_all.png P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg What kind of inputs and strategy? •Type of inputs? –File, network packets, structure, data model, state(-less) •What environment setup is necessary? –Fuzzing on live system? –Multiple entities inside VMs? Networking? •Isolated vs. cooperating components? –We don’t like to mock everything •What tools are readily available? – 56 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Microsoft’s SDL Regex Fuz •Test of regular expressions evaluations –May cause denial-of-service attack •Use when your program use regex evaluation –Extract all your used patterns, test it by SDL Regex Fuz •Video overview –http://msdn.microsoft.com/en-us/security/gg675012.aspx –http://blogs.msdn.com/b/sdl/archive/2010/10/12/new-tool-sdl-regex-fuzzer.aspx •Example: ^(\d+)+$ 57 | PA193 - Dynamic analysis, fuzzing D:\Documents\Obrázky\services_icon_full_bw5.jpg P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg • 58 | PA193 - Dynamic analysis, fuzzing D:\regex.png P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg MODELLING • 59 | PA193 - Dynamic analysis, fuzzing D:\Documents\Obrazky\fuzzing\process_all.png P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Input preparation •Time intensive part of fuzzing (if model !exists yet) 1.Fully random data 2.Random modification of valid input 3.Modification of valid input with fuzz vectors 4.Modification of valid input with mutator 5.Fuzzing via intermediate proxy 60 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Radamsa fuzzer •“…easy-to-set-up general purpose shotgun test to expose the easiest cracks…” –https://code.google.com/p/ouspg/wiki/Radamsa •Just provide input files, all other settings automatic –cat file | radamsa > file.fuzzed • 61 | PA193 - Dynamic analysis, fuzzing D:\Documents\Obrázky\services_icon_full_bw5.jpg >echo "1 + (2 + (3 + 4))" | radamsa --seed 12 -n 4 1 + (2 + (2 + (3 + 4?) 1 + (2 + (3 +?4)) 18446744073709551615 + 4))) 1 + (2 + (3 + 170141183460469231731687303715884105727)) P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg How to generate fuzzed input? •Generational fuzzing (Recursive fuzzing) –Produces data based only on data model description –E.g., iterates over range of values of given alphabet •Mutational fuzzing (Replacive fuzzing) –Produces data based on templates and supplied model –Known border values or malicious malformed input –Fuzz test vectors –String-based mutators, number-based mutators… • 62 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Peach fuzzer: DataModel (gif) • 63 | PA193 - Dynamic analysis, fuzzing … http://pastebin.com/9Y2yENqG D:\Documents\Obrázky\services_icon_full_bw5.jpg D:\Documents\Obrazky\fuzzing\beer_gif_hex.png P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Peach Pit Library •Predefined support for various protocols • • • • • • •Peach Fuzzer™ Overview – datasheet, 2014 •(available only in Professional and higher versions) • 64 | PA193 - Dynamic analysis, fuzzing D:\peach_pits.png D:\Documents\Obrázky\services_icon_full_bw5.jpg P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Fuzzing via intermediate proxy •Fuzzer modifies valid flow according to data model •Usually used for fuzzing of state-full protocols –Modelling states and interactions would be difficult –Target application(s) takes care of states and valid input • 65 | PA193 - Dynamic analysis, fuzzing D:\Documents\Obrazky\fuzzing\Home-Server-Logo-large.png D:\Documents\Obrazky\fuzzing\firefox-logo.jpg D:\Documents\Obrazky\fuzzing\zap_logo.png D:\Documents\Obrazky\fuzzing\games_play_game_dices_yatzy_bet.png P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg OWASP’s ZAP – fuzz strategy settings • 66 | PA193 - Dynamic analysis, fuzzing D:\Documents\Obrazky\fuzzing\zap.png D:\Documents\Obrázky\services_icon_full_bw5.jpg P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg TLS fuzzer •https://github.com/tomato42/tlsfuzzer/ 67 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg APDUPlay - Smart card fuzzing •Host to smart card communication done via PC/SC •Custom winscard.dll stub written •Manipulate incoming/outgoing APDUs –modify packet content –replay of previous packets –… • •| PA193 - Dynamic analysis, fuzzing •[RULE1] •MATCH1=in=1;t=0;cla=00;ins=a4;p1=04; •ACTION=in=0;data0=90 00;le=02; •00 a4 04 00 08 01 02 03 04 05 06 07 08 •winscard.dll (stub) •90 00 • • 68 http://www.fi.muni.cz/~xsvenda/apduinspect.html D:\Documents\Obrázky\services_icon_full_bw5.jpg P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg VALIDATION • 69 | PA193 - Dynamic analysis, fuzzing D:\Documents\Obrazky\fuzzing\process_all.png P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Validation of model •Are fuzzed inputs according to your need? –Smarter fuzzing understands a data format –Wrong data format usually fails early on initial parsing •Check between fuzzing data model and real input –E.g., Peach Validator tool •Are template files providing good test coverage? –E.g., Peach minset tool 70 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Peach Validator 3.0 • 71 | PA193 - Dynamic analysis, fuzzing C:\Picts\fuzzing\peachvalidator_beer.png C:\Picts\fuzzing\peachvalidator_beer_valid.png D:\Documents\Obrázky\services_icon_full_bw5.jpg Model doesn’t match valid input P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg American fuzzy lop •Relatively new, but actively developed tool •High speed fuzzer http://lcamtuf.coredump.cx/afl/ •Sophisticated generation of test cases (coverage) •Automatic generation of input templates –E.g., valid JPEG image from “hello” string after few days –http://lcamtuf.blogspot.cz/2014/11/pulling-jpegs-out-of-thin-air.html •Lots of real bugs found • • 72 | PA193 - Dynamic analysis, fuzzing D:\Documents\Obrazky\fuzzing\first_jpegs.jpg D:\Documents\Obrázky\services_icon_full_bw5.jpg P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Test coverage •Random inputs have low coverage (usually) –Number of blocks visited in target binary •Smart fuzzing tries to improve coverage –Way how to generate new inputs from existing •E.g., Peach’s minset tool –Gather a lot of inputs (files) –Run minset tool, traces with coverage stats are collected –Minimal set of files to achieve coverage is computed –Selected files are used as templates for fuzzing •E.g. AFL fuzzer uses compile time instrumentation + genetic programming to create test cases 73 | PA193 - Dynamic analysis, fuzzing D:\Documents\Obrázky\services_icon_full_bw5.jpg P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg START, GENERATE, MONITOR • 74 | PA193 - Dynamic analysis, fuzzing D:\Documents\Obrazky\fuzzing\process_all.png P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg How to detect “hit”? •Application crash, uncaught exception… –Clear faults, easy to detect •Error returned –Some errors are valid response –Some errors are valid response only in selected states •Input accepted even when it shouldn't be –E.g., packet with incorrect checksum or modified field •Some operation performed in incorrect state –E.g., door open without proper authentication •Application behavior is impaired –E.g., response time significantly increases •… 75 | PA193 - Dynamic analysis, fuzzing D:\Documents\Obrazky\fuzzing\bullet_hole.png P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Peach monitors 76 | PA193 - Dynamic analysis, fuzzing D:\Documents\Obrázky\services_icon_full_bw5.jpg P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg http_requests GANet fuzzer •Are there any inputs processed longer then usual? •Deny of service as target of fuzzing –Can we craft input causing significant load? –Fuzzing, but not completely random (GA, GP) •Input: http / SSL request (fuzzed – content, time) •Setup: application inside VMs •Monitor: memory/CPU/network load •Analyze: outstanding load detected • •Bukac et al., Challenges of fiction in network security – perspective of virtualized environments, SPW’15, Cambridge, 2015 77 | PA193 - Dynamic analysis, fuzzing D:\Documents\Obrázky\services_icon_full_bw5.jpg P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg GANet fuzzing setup • 78 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Peach Pit – specifying fuzz actions •Peach Pit is XML file –http://old.peachfuzzer.com/v3/TutorialFileFuzzing/CreateDataModel.html •Data Models http://old.peachfuzzer.com/v3/DataModel.html –Format description of input data •State Models http://old.peachfuzzer.com/v3/StateModel.html –How fuzzer should behave in different states •Agents http://old.peachfuzzer.com/v3/AgentsMonitors.html –Peach processes running program you fuzz, debugger for monitoring… –WinDbg, Linux Crash Monitor, Ping monitor, SSH monitor… •Publisher http://old.peachfuzzer.com/v3/Publisher.html –How to communicate wit target application (TCP, files…) •Tests http://old.peachfuzzer.com/v3/TestConfig.html –Setting all parameters of test (Publishers, Loggers, Agents and StateModel) – 79 | PA193 - Dynamic analysis, fuzzing D:\Documents\Obrázky\services_icon_full_bw5.jpg P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg | PA193 - Dynamic analysis, fuzzing Model of input data ‘HTER anything \r\n’ 1.Read any string 2.Send fuzzed input 3.Read any string Agent responsible for starting target application with debugger connected Test scenario with specified settings How to communicate with target application How to store results 80 P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg • 81 | PA193 - Dynamic analysis, fuzzing D:\Documents\Obrazky\fuzzing\peach_hter_pit.png Example from http://rockfishsec.blogspot.ch/2014/01/fuzzing-vulnserver-with-peach-3.html D:\Documents\Obrazky\fuzzing\vulnserver_crashed.png P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg ANALYZE • 82 | PA193 - Dynamic analysis, fuzzing D:\Documents\Obrazky\fuzzing\process_all.png P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg What to do with hit results? •Time intensive part of fuzzing •Not all hits are relevant (at least at the beginning) –Crashes by values not controllable by an attacker –!exploitable https://msecdbg.codeplex.com/ •Hits reproduction –Hit can be result of cumulative series of operations •Many hits are duplicates –Inputs are different, but hit caused in the same part of code •(Automatic) Bucketing of hits –E.g., Peach performs bucking based on signature of callstack 83 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Motto: “Test your test coverage” •Sebastian Kurfürst’s php fuzzer •Idea: –Ordinary UT coverage tool just count visited LoC –Run tests and compute test coverage –Foreach (line:source_code) •Comment out / modify line of code •Verify valid syntax – if yes, then run unit tests again •Detect if tests pass even when line was removed / modified •If yes, then your UT is not sufficient •https://github.com/sandstorm/Fuzzer – 84 | PA193 - Dynamic analysis, fuzzing D:\Documents\Obrázky\services_icon_full_bw5.jpg P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Teaser: Fuzzing hands-on •Some file fuzzing •General fuzzing framework - Peach –Setup, components, plugins, configuration… –Custom input modeling and validation •Some network fuzzing with OWASP ZAP? –Setup, options… –Proxy-based network fuzzing 85 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Summary •Fuzzers are cheap way to detect simpler bugs –If you don’t use it, others will •Try to find tool that fits your particular scenario –Check activity of development, support •Fuzzing frameworks can ease variety of setups –But bit steeper learning curve •If fuzzing will not find any bugs, check your model •Try it! • • 86 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg TAINT ANALYSIS • 87 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Taint analysis •Form of flow analysis •Follow propagation of sensitive values inside program –e.g., user input that can be manipulated by an attacker –find all parts of program where value can “reach” •“Information flows from object x to object y, denoted x→y , whenever information stored in x is transferred to, object y.” D. Denning •Native support in some languages (Ruby, Perl) –But not C++/Java L • 88 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Taint sources •Files (*.pdf, *.doc, *.js, *.mp3...) •User input (keyboard, mouse, touchscreen) •Network traffic •USB devices •... • •Every time there is information flow from value from untrusted source to other object X, object X is tainted –labeled as “tainted” 89 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Execution of sensitive operation •Before sensitive operation (e.g., system()) is executed with value, taint label is checked –if value is tainted, alert is issued •Untrusted data reaching privilege location is detected –can detect even unknown attacks –(but sometimes we need to use user input) 90 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Taint analysis - tools •Taintgrind –http://www.cl.cam.ac.uk/~wmk26/taintgrind/ –additional module to Valgrind –dynamic taint analyzer for C/C++ –output memory traces (information flows) already produced by Valgrind •Tanalysis –http://code.google.com/p/tanalysis/ –static taint analyzer for C –plugin for Frama-C http://frama-c.com/ •Read more about taint analysis –http://users.ece.cmu.edu/~ejschwar/papers/oakland10.pdf – • 91 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Microsoft PREfast + Taint analysis •Warning C6029 is issued when tainted value is passed to parameter marked as [Post(Tainted=No)] –without any checking (any condition statement) •http://msdn.microsoft.com/en-us/library/ms182047%28v=vs.100%29.aspx • 92 | PA193 - Dynamic analysis, fuzzing // C #include void f([SA_Pre(Tainted=SA_Yes)] int c); // C++ #include using namespace vc_attributes; void f([Pre(Tainted=Yes)] int c); P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Coverity taint analysis •TAINTED_SCALAR –http://blog.coverity.com/2014/04/18/coverity-heartbleed-part-2/#.U1l4k2dOURo –http://security.coverity.com/blog/2014/Apr/on-detecting-heartbleed-with-static-analysis.html • 93 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Conclusions •Dynamic analyzers can profile application –and find bugs not found by static analysis •Fuzzing is “cheap” blackbox approach via malformed inputs • 94 | PA193 - Dynamic analysis, fuzzing question •Questions P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg References •Some books available, but… •Michael Eddington, Demystifying fuzzers –Comparison of open-source tools, cost of adoption –BlackHat 2009, https://www.blackhat.com/presentations/bh-usa-09/EDDINGTON/BHUSA09-Eddington-DemystFuzzers-PAPER.pd f –https://www.blackhat.com/presentations/bh-usa-09/EDDINGTON/BHUSA09-Eddington-DemystFuzzers-SLIDES. pdf –RSA Conference 2010 talk https://www.youtube.com/watch?v=Bm3Mfndrl1Y •OWASP fuzzing guidelines –https://www.owasp.org/index.php/Fuzzing –https://www.owasp.org/index.php/OWASP_Testing_Guide_Appendix_C:_Fuzz_Vectors •Tutorials and research papers on fuzzing http://fuzzing.info/papers/ – • 95 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Peach tutorials •Basic usage against vulnserver –http://rockfishsec.blogspot.ch/2014/01/fuzzing-vulnserver-with-peach-3.html •Advanced tutorial (ZIP format fuzzing) – very good –http://www.flinkd.org/2011/07/fuzzing-with-peach-part-1/ •Tutorial for RAR fuzzing –http://www.flinkd.org/2011/11/fuzzing-with-peach-part-2-fixups-2/ – • 96 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg References •MS post on Test coverage by fuzzing –http://blogs.technet.com/b/srd/archive/2010/02/24/using-code-coverage-to-improve-fuzzing-results.a spx •Application and file fuzzing –http://resources.infosecinstitute.com/application-and-file-fuzzing/ •How I Learned to Stop Fuzzing and Find More Bugs –https://www.defcon.org/images/defcon-15/dc15-presentations/dc-15-west.pdf – • 97 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg • 98 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg DYNAMIC ANALYSIS - PROFILING (WHITEBOX) • 99 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg •| PA193 - Dynamic analysis, fuzzing Automatic measurement - profiling •Automatic tool to measure time and memory used •“Time” spend in specific function •How often a function is called •Call tree –what function called actual one –based on real code execution (condition jumps) •Many other statistics, depend on the tools •Helps to focus and scope security analysis • • 100 P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg •| PA193 - Dynamic analysis, fuzzing MS Visual Studio Profiler •Analyze®Launch Performance Wizard •Profiling method: CPU Sampling –check periodically what is executed on CPU –accurate, low overhead •Profiling method: Instrumentation –automatically inserts special accounting code –will return exact function call counter –(may affect performance timings a bit) •additional code present •May require admin privileges (will ask) 101 P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg •| PA193 - Dynamic analysis, fuzzing MS VS Profiler – results (Summary) •Where to start the optimization work? – perf_summary 102 P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg •| PA193 - Dynamic analysis, fuzzing MS VS Profiler – results (Functions) •Result given in number of sampling hits –meaningful result is % of total time spend in function •Inclusive sampling –samples hit in function or its children –aggregate over call stack for given function •Exclusive sampling –samples hit in exclusively in given function –usually what you want •fraction of time spend in function code (not in subfunctions) – 103 P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg •| PA193 - Dynamic analysis, fuzzing MS VS Profiler – results (Functions) • perf_Functions •Doubleclick to move into Function Details view 104 P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg GCC gcov tool •http://gcc.gnu.org/onlinedocs/gcc/Gcov.html#Gcov 1.Compile program by GCC with additional flags –gcc -Wall -fprofile-arcs -ftest-coverage main.c –gcc -Wall --coverage main.c –additional monitoring code is added to binary 2.Execute program –files with “.bb" ".bbg" and ".da" extension are created 3.Analyze resulting files with gcov –gcov main.c –annotated source code is created •Lcov - graphical front-end for gcov –http://ltp.sourceforge.net/coverage/lcov.php – – • 105 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg • 106 | PA193 - Dynamic analysis, fuzzing D:\lcov.png Taken from http://ltp.sourceforge.net/coverage/lcov/output/example/methods/iterate.c.gcov.html P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg REVERSE ENGINEERING (BLACKBOX) • 107 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg •| PA193 - Dynamic analysis, fuzzing Reverse engineering •Art of discovering principles through analysis of structure, functions and operation •Legality –Own binary without documentation –Interoperability –Anti-virus research –Fair use, education –Forensics •Problem with recent copyright laws –even attempt to circumvent is illegal –not only selling circumvented content 108 P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg •| PA193 - Dynamic analysis, fuzzing Disassembler vs. debugger •Static vs. dynamic code analysis •Debugger vs. Debugger with advanced modification tools (Visual Studio vs. OllyDbg) •Assembler vs. bytecode –Instruction set –Register-based vs. stack-based execution • • 109 P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg •| PA193 - Dynamic analysis, fuzzing Lena tutorials •Nice introduction tutorials for reversing/cracking •Win32 binary –Lena tutorials 1 and 2 •Name of the registers –(EAX 32bit, AX 16bit, AH/AL 8bit) •Registers (FPU): –Z – zero flag, C – carry flag, S – sign flag –EIP ... next address to execute (instruction pointer) –EBX ... usually loop counter 110 P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg •| PA193 - Dynamic analysis, fuzzing Startup resources •The Reverse Code Engineering Community: http://www.reverse-engineering.net/ •Tutorials for You: http://www.tuts4you.com •RE on Wikipedia: http://en.wikipedia.org/wiki/Reverse_engineering 111 P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Disassembling binary code •Interactive Disassembler is legendary full-fledged disassembler with ability to disassemble many different platforms. –Free version available for non-commercial uses –Free version disassemble only Windows binaries –http://www.hex-rays.com/idapro/idadownfreeware.htm •Very nice visualization and debugger feature (similar as OllyDbg) –Try it! 112 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Decompiling binary code •Decompiler is able to produce source code from binary code. Decompiler needs to do disassembling first and then try to create code that will in turn produce binary code you have at the beginning. •Resulting code will NOT contain information removed during compilation –(comments, function names, formatting...) –Read http://www.debugmode.com/dcompile/ for more info •Still can be of great help •Problem to find well working free disassembler –http://en.wikibooks.org/wiki/X86_Disassembly/Disassemblers_and_Decompilers – • 113 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Resources •The Reverse Code Engineering Community: http://www.reverse-engineering.net/ •Tutorials for You: http://www.tuts4you.com •Disassembling tutorial http://www.codeproject.com/KB/cpp/reversedisasm.aspx • 114 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg OLD FUZZING • 115 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Example with many possible inputs •Multiple inputs to application •Not possible to evaluate manually –or done very frequently: UT, TDD, continuous integration •Sometimes not possible to bruteforce at all –to many combinations –usual situation! •Easy to overlook potential problem 116 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Fuzzer - principle 1.Send high number of different inputs into application –user interface (UI), command line options, import/export capabilities 2.Inputs generated randomly or according to predefined pattern –Protocol/file-format/data-type dependant –E.g., input string with different lengths (1MB user name) –E.g., valid input with certain percentage of random modifications (jpg file with random changes) 3.Fuzzer monitors application for crash or emitted error –Memory corruption, invalid state… –When detected, problem is manually inspected •https://www.owasp.org/index.php/Fuzzing – • 117 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Fuzzing – advantages/disadvantages •Fuzzers tend to find simple bugs –more protocol-aware fuzzer is, less weird problems will find •Fuzzing advantages –very simple design –allow to find bugs missed by human eye –sometimes only way to test (completely closed system) •Fuzzing disadvantages –increased difficulty to evaluate impact/dangerosity •closed system is evaluated, black box testing 118 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Types of fuzzing •Application fuzzing –generates inputs for application (stdin, memory...) •Protocol fuzzing –manipulation of protocol level •File format fuzzing –generates malformed file samples –if program crashes, debug log is created –attack against parser layer –attack against codec/application layer –example: MS04-028 Microsoft's JPEG GDI+ vulnerability •http://technet.microsoft.com/en-us/security/bulletin/ms04-028 119 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Fuzzing – approaches •Capture valid inputs and modify some bytes –randomly –according to given regular expression –Random changes with post-processing (e.g., correct CRC) •Binary vs. text oriented fuzzing 120 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Available tools •Microsoft’s SDL MiniFuzz File Fuzzer •Microsoft’s SDL Regex Fuzzer •Ilja van Sprundel’s mangle.c –https://ext4.wiki.kernel.org/index.php/Filesystem_Testing_Tools/mangle.c –filename and header size –change between 0 and 10% of header with random bytes –example data •zzuf - multi-purpose fuzzer –application input fuzzer –intercepting file and network operations and changing random bits in the program’s input –http://sam.zoy.org/zzuf/ – • 121 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Microsoft’s SDL MiniFuzz File Fuzzer •Application input files fuzzer –http://www.microsoft.com/en-us/download/details.aspx?id=21769 –UsingMiniFuzz.htm •Templates for valid input files (multiple) •Modify valid input file (randomly, % aggressiveness) •Run application with partially modified inputs •Log resulting crash (if happen) –exception, CPU registers... •Can be incorporated directly into Visual Studio •Video overview –http://msdn.microsoft.com/en-us/security/gg675011.aspx – 122 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg • 123 | PA193 - Dynamic analysis, fuzzing D:\minifuzz.png P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg Microsoft’s SDL Regex Fuz •Test of regular expressions evaluations •May cause denial-of-service attack •Use when your program use regex evaluation •Video overview –http://msdn.microsoft.com/en-us/security/gg675012.aspx •http://blogs.msdn.com/b/sdl/archive/2010/10/12/new-tool-sdl-regex-fuzzer.aspx •Example: ^(\d+)+$ 124 | PA193 - Dynamic analysis, fuzzing P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg • 125 | PA193 - Dynamic analysis, fuzzing D:\regex.png P:\CRCS\2012_0178_Redesign_loga_a_JVS\PPT_prezentace\sablona\pracovni\normalni.jpg SPIKE •Tool for fuzzing analysis of network protocols –http://www.immunitysec.com/resources-freesoftware.shtml •Overview of SPIKE capabilities –https://www.blackhat.com/presentations/bh-usa-02/bh-us-02-aitel-spike.ppt •Fuzzing tutorial with SPIKE on Vulnserver –http://resources.infosecinstitute.com/intro-to-fuzzing/ –Windows & Linux version •Another SPIKE tutorial –http://pentest.cryptocity.net/fuzzing/ – 126 | PA193 - Dynamic analysis, fuzzing