Introduction

MiniFuzz is an easy to use file fuzzer that can help uncover security and reliability bugs in software. Many real-world security bugs have been found through file fuzzing. Examples of fuzzing bugs found in various file format parsers in Microsoft software include:

· ANI (MS07-017)

· XLS (MS06-012)

· BMP (MS06-005, MS05-002)

· TNEF (MS06-003)

· EOT (MS06-002)

· WMF (MS06-001, MS05-053)

· EMF (MS06-053)

· PNG (MS05-009)

· GIF (MS05-052, MS04-025)

· JPG (MS04-028)

· ICC (MS05-036)

· ICO (MS05-002)

· CUR (MS05-002)

· ANI (MS05-002)

· DOC (MS05-035)

· ZIP (MS04-034)

· ASN.1 (MS04-007)

It’s probably not a stretch to say, if you have never fuzz-tested your application, you will probably find bugs.

Because fuzzing is very effective at finding bugs, it is a required activity in the Verification Phase of the Microsoft Security Development Lifecycle (SDL). With the release of MiniFuzz, we have made a simple file fuzzer available to you to assist your efforts to find and address more bugs in your code before you ship the code to customers.

MiniFuzz can run as a standalone application or as an integrated tool within Visual Studio. In addition, MiniFuzz can output to your Team Foundation Server and integrate with the Microsoft SDL Process Template for VSTS or any other Visual Studio Team Systems process template.

If you are new to fuzzing you should read the following section, “Overview of Fuzzing and MiniFuzz.”

Target AudiEnce for MiniFuzz

MiniFuzz is a very simple fuzzer designed to ease adoption of fuzz testing by non-security people who are unfamiliar with file fuzzing tools or have never used them in their current software development processes.

Overview of Fuzzing and MiniFuzz

Fuzzing is a testing technique that can help find denial of service and security vulnerabilities in software. The principle of fuzzing is very simple: create invalid data, force an application to consume that malformed data, and then observe the application as it executes. If the application crashes, then a bug may have been found in the target application. By identifying this crash, you are able to quickly target potential problems in the underlying code and determine if changes are needed to fix the crash (and any related potential security issues) from affecting your users.

MiniFuzz is a small file fuzzer created by the SDL team at Microsoft to demonstrate basic file fuzzing concepts and help software developers outside of Microsoft find denial of services and security vulnerabilities in their software products before they release the products to customers.

There are two classes of fuzzers: dumb fuzzers and smart fuzzers. The current version of MiniFuzz is a dumb fuzzer, which means it manipulates files with no regard for the file format. We will continue to evaluate if a future version of MiniFuzz should support smart fuzzing, which allows the fuzzing logic to manipulate data based on file format information.

File Fuzzing, as an SDL requirement, requires a minimum of 100,000 malformed files per file parser. So if your application parses three discrete file formats, .FOO files and .BAR files, then you need to create and correctly parse (ie; not crash) 100,000 FOO files and 100,000 BAR files.

The process of fuzzing is relatively simple, and follows this process:

1. Create or gather a large sample of correctly formed files, this collection is referred to as the set of template files.

2. The fuzzer selects a file at random from the set template files.

3. The fuzzer saves the file to a temporary location.

4. The fuzzer corrupts the temporary file.

5. The fuzzer loads the target application (the application under test) and passes the name of the file on the command line.

6. The fuzzer runs the target application using the Windows debugger functionality so it can catch errors as the application runs.

7. If the application does not crash after the predetermined and configurable amount of the time (default is 2.0 seconds), the application is shut down and the temporary file is deleted.

8. If the application does crash, the offending file is saved along with crash information. This data is either stored in the file system, or in the Team Foundation Server bug tracking database.

9. Go back to step 2

Installing MiniFuzz

Simply run the setup.exe program.

If you are installing MiniFuzz as a Visual Studio tool you must also run setup.exe and then follow these steps to add it to the Tools menu:

· Run Visual Studio

· Click Tools | External Tools

· Click Add

· Enter the following elements:

o Set the Title to MiniFuzz

o Set the Command to the location of minifuzz.exe

o Set the Arguments to $(TargetPath) %1

o Set the Initial directory to MiniFuzz installation folder.

· Click OK

The %1 characters indicate command-line arguments that should be passed to the fuzz target, MiniFuzz will automatically populate this with the correct filename.

Launching MiniFuzz

MiniFuzz is a simple fuzzer that takes a series of template files and uses those as the basis of the fuzzing process. Minifuzz can run as a standalone application or it can be run inside of Visual Studio. Whether it runs standalone or as a Visual Studio tool, it can also integrate with Team Foundation Server to automatically log fuzzing crashes as bugs.

Launching MiniFuzz as a Standalone Application

To run MiniFuzz as a standalone application, simply navigate to the c:\program files\microsoft\minifuzz folder and run minifuzz.exe.

Launching MiniFuzz from within Visual Studio

Once the tool is added to Visual Studio’s menu system, you can run the fuzzer and automatically populate the process to fuzz by selecting Tools | MiniFuzz.

Integrating with Visual Studio Team Foundation Server

When you run MiniFuzz as a standalone application or from within Visual Studio, MiniFuzz will attempt to detect if the user is presently connected to Team Foundation Server, and if so, MiniFuzz will also connect to the same TFS server.

Whether the user is connected to TFS or not, the user can connect or disconnect at any time.

Using MiniFuzz to test your Application

There are some steps you must take to prepare your application for fuzzing. This section describes those steps and how to start using MiniFuzz to test your application.

Step 1: Collect a good sample of correctly-formed template files

There is no hard rule that determines how many template files you should have, but you should aim for over a hundred if possible. The files should also represent as many variations as possible. For example, if you build code to render images from digital cameras, you should get real photos:

· from every camera model your application supports, both old and new models

· in file format you support (usually JPEG, TIFF and raw formats)

· of different supported sizes (often small, medium and large)

· of different supported quality settings (often low, medium, high and very high)

· in all supported compression formats

· etc.

And use that collection as your set of template files; the more varied your collection more chance you have of finding bugs in your parser code when using MiniFuzz.

Step 2: Configure MiniFuzz

You should only need to do this once, but you need to set some folders to read and store fuzzing information. The configuration file is named minifuzz.cfg, and can be found in the folder that includes all the binaries used by MiniFuzz, which by default is C:\Program Files\Microsoft\MiniFuzz

By default, the file is:

<?xml version="1.0" encoding="utf-8"?>

</folders>

</extensions>

</config>

The following table explains the elements.

Element	Comment
<templates location>	Folder that holds the file templates, this can be changed in the MiniFuzz user interface.
<temp location>	Folder for all temporary fuzzed files.
<log location>	Location of the log file if the tool is not using Team Foundation Server.
<crashedfiles location>	Location to store files that led to a crash.
<postprocess location>	Currently unused
<extensions>	Currently unused

The configuration string can use one predefined sentinel value, %desktop% which points to the current user’s desktop.

NOTE: If a required folder does not exist, MiniFuzz will attempt to create it.

Step 3: Start MiniFuzz and set important fuzzing parameters

Start MiniFuzz, as outlined in “Launching MiniFuzz” on page 2 and you are presented with the following tool:

Now, you need to update some of the fuzzing parameters which are explained here.

Field	Description
Process to fuzz	This is the full path and the name of the executable under test.
Command line args	Place holder and optional switches to pass to the process under test. %1 is a placeholder for the name of the corrupted file, and because the name is random, you won’t know the filename ahead of time but the fuzzer does. If your tool requires a command line switch to set the name of the file, then you should include that also. For example, if your application requires you set the filename using the /F:<name> option, then you can use this: /F:%1
Allow process to run for	This is the maximum time in seconds to allow the process to run before killing the process. Whatever time you set, it should be long enough to allow the process to render or parse the file. If the application does not die in the allotted time, MiniFuzz will terminate the process. Important: Do not be overly aggressive with this value by making the value very small because the application under test might not have time to start up, configure itself, load the file and then render the file before MiniFuzz tears the process down. Start out at 2.0 seconds, and then after a couple of tests, throttle back to 1.5, then if needed 1.0 or, the lowest possible value, 0.5 seconds. Also, for very large files, over 1Mb, you should increase this setting substantially because it will take time for the process under test to load the file off the disk.
Shutdown method	MiniFuzz can close an application down by using one of three methods: Thread Injection injects a new thread into the target process and then calls ExitProcess on that threat. This works for all Windows applications and is a graceful shutdown. This is the default setting. WM_CLOSE is the most graceful shutdown method as it instructs the application to shut down cleanly. This only works for graphical applications, not for console applications. The TerminateProcess API tears running applications down. It’s brutal, and it can lead to some resources left un-freed. Any application type can be shut down this way, but it should be used only as a last resort. Do not be alarmed if you see dozens of your target processes running at once, this is quite normal because shutting processes down can take time for some kinds of processes. If you see hundreds of instances of you application, then the fuzzer is not shutting the process down correctly, or your application is preventing shutdown.
Shutdown delay	Some applications may take time to shutdown, especially when using the WM_CLOSE shutdown method- The default shutdown times should be adequate for most programs.
Template Files	This is where you have stored your set of template files; you can override this setting if you need to.
Temporary files	This is the location of all the corrupted files. MiniFuzz deletes these files in the background, so it’s not unusual to see about ten files in this folder at any given time. MiniFuzz will also attempt to delete any files remaining in this folder when MiniFuzz exits. This location is configurable through the XML configuration file described in “Step 2: Configure MiniFuzz” on page 3.
Log files	This is the location of the log file that tracks all crashes. More on this file in section foo
Crash files	In the application under test crashes, the offending file is copied to this folder and removed from the temporary folder. This location is configurable through the XML configuration file described in “Step 2: Configure MiniFuzz” on page 3.
Aggressiveness	This setting determines how much of the incoming file to fuzz. There is no value that is better than any other number because sometimes a very subtle single bit change might trigger an application to fail. Large changes may render a file so badly malformed that your parsing code rejects the file very early in the parsing process. In short, you need to work out what the most appropriate level or aggressiveness is for your parsing code.
Always on Top	This means MiniFuzz always runs as the top-most window.
Start Fuzzing	As the name suggests, this starts the fuzzing process once you have set all the appropriate settings.
Stop Fuzzing	Stops the fuzzing process, but does not reset the progress data.
View Logfile Folder	Opens the folder that contains all the log files.
TFS Settings	Opens a dialog box that allows you to view and configure Visual Studio Team Foundation Server settings.
About	Brings up the About dialog box.
Progress section	This section lists ongoing tally information such as the number of files read and the number of crashes.

VERY IMPORTANT

Applications being fuzz-tested behave differently when forced to shutdown. You might have to experiment with the “Allow process to run for”, “Shutdown method” and “Shutdown delay” options to find the optimal settings for the target application. This is especially true if you see the application remain in memory after a test pass.

Step 4. Start the Fuzzing Process

Once you have configured MiniFuzz, simply click the Start Fuzzing button, make sure the application loads and shuts down correctly and if all seems well, leave the application to run for a few hours. If you have never fuzzed your code before, MiniFuzz will likely find some bugs pretty quickly.

If the fuzzer triggers a bug in the application under test, the time of the crash as well as the offending filename is displayed in the Progress section.

Step 5. Triaging the results

If after a few hours you have one or more crashes, it’s important that you analyze the crash information preserved by MiniFuzz. This information includes:

· The file that caused the crash.

· Register information.

The former is stored in the crash folder and the latter is stored in the log file. At this point the workflow depends on whether you are a person who can understand crash information held in machine registers or not. If you understand crash type and register data then take a look at each crash in the log file to determine the nature of the crash and to determine how many discrete crashes you have, You might see 134 crashes, but that may only be 7 discrete bugs; the Instruction Pointer (EIP or RIP, depending on the CPU type) points to the crash location and this should be your first indication of how many discrete bugs you have encountered so far.

If you are a debugging expert then you should keep one of each file that caused each discrete crash.

Whether you are a debugging expert or not, you should run a debug (also called checked) build of your application under a debugger (such as Visual Studio or WinDbg) with each file that caused a crash. Once the application crashes, you should get a nice pointer to the vulnerable code.

Step 6. File bug(s)

If you are using Visual Studio Team Foundation Server, MiniFuzz will log a bug for you for each crash. The current version will do this for every crash it detects, which might lead to many bug duplicates. Do not delete these bugs; rather, investigate each and resolve them as duplicates of the first bug because this will give you an indication of how often you have hit the bug.

Step 7. Fix the Bug(s)

It is beyond the scope of this document to describe how you should fix your code. If you don’t know how to fix the code, then you should refer to the many excellent resources at the SDL web site, http://www.microsoft.com/sdl.

Step 8. Repeat from Step 3

After you have fixed the code bugs, you must repeat the process until you can reach 100,000 malformed files with no crashes.

Integrating with Microsoft Team Foundation Server

MiniFuzz integrates with the bug database functionality of TFS 2008 and 2010; filing a bug for each crash it encounters. Using TFS requires that you have an account to a TFS server. If you don’t have an account or you do not know the name of the TFS and project you wish to connect to, please see your administrator.

You will also need software that connects with your TFS server, such as Visual Studio Team Explorer, which is a free download from Microsoft.com.

After you have clicked the “TFS Settings …” button on the main dialog, you will see the following dialog allowing you to select the TFS server, the project collections on that server, and then the project.

Once you have selected the data, click Connect. You are now connected to a TFS team project and any bugs found by MiniFuzz will be filed in the selected project.

Appendix A: Integration With !Exploitable

!exploitable, written by the same security team at Microsoft that created MiniFuzz, is an extension for the native Windows debuggers, such as WinDbg, that provides automated crash analysis and security risk assessment.

MiniFuzz does not use !exploitable directly, but you should use it when you find a new discrete failure. After MiniFuzz finds a failure, load the application along with the name of the file causing the failure into WinDbg:

windbg badapp.exe \users\mike\desktop\minifuzz\crashes\foobar8776.bad

Then make sure that !exploitable is loaded:

!load winext\msec.dll

Run the process and have it parse the file:

Finally, run !exploitable to take a first pass analysis of the failure:

!exploitable

APPENDIX B: Resources

· The Security Development Lifecycle, Microsoft Press, Howard and Lipner, Chapter 12

· The Security Development Lifecycle version 4.1 documentation – http://www.microsoft.com/sdl

· Writing Fuzzable Code http://blogs.msdn.com/b/sdl/archive/2010/07/07/writing-fuzzable-code.aspx

· !exploitable Crash Analyzer - MSEC Debugger Extensions http://msecdbg.codeplex.com/

· The History of the !exploitable Crash Analyzer http://blogs.technet.com/srd/archive/2009/04/08/the-history-of-the-exploitable-crash-analyzer.aspx