Your Reporting Matters: How to Improve Pen Test Reporting

Brian B. King //

This is a companion post to BBKing’s “Hack for Show, Report for Dough” report, given at BSides Cleveland in June 2019.

The fun part of pentesting is the hacking. But the part that makes it a viable career is the reporting. You can develop the most amazing exploit for the most surprising vulnerability, but if you can’t document it clearly for the people who need to fix it, then you’re just having fun. Which is fine! Fun is awesome! But if you want to get hired again, your reports need to be at least as clear and useful as your hacks are awesome.

Guiding Principles

Before getting into the details, it’s helpful to have a mental framework to hang them on. Everything about your report should help the reader understand what you, in your expert opinion, found interesting about the target of your test.

“Interesting” is a word that can’t usually stand by itself. It’s too vague. Here, interesting means the vulnerabilities, of course, but it also includes positive things. If you have a go-to technique or attack that usually works but failed this time – that’s interesting. Anything you noticed that was unusually good or consistent is interesting and should be included too.

Two Audiences

There will be at least two kinds of people reading your report, and you need to communicate clearly with both of them.

You’ll have technical people who will need to understand how you did what you did so they can replicate your results and devise fixes that actually work. Imagine writing to yourself at the skill level you had one year ago. If you’ve been at this a long time, picture an intelligent and motivated friend who does system administration or software development. Someone who understands the context but maybe not the details. Someone you want to help.

That’s your technical reader. Write the body of the report so that person can re-do the things you did and see the things you saw.

You’ll also have business people who will need to understand why your findings are important, and what business processes are involved, either as causes or solutions. Imagine a person who cares deeply about what you have to say – they hired you because you’re an expert after all – but who focuses more on what gets done than on the details of how. This person wants to keep the business running and would love it if that can be done securely. They think in terms of business processes, not computer systems. They may not have a favorite text editor. Just like your technical reader, your business readers are intelligent and motivated – they just work with different things, one or two layers of abstraction away from the actual systems.

That’s your business reader. Write the executive summary so this person appreciates the overall situation, the few most significant findings, and what business processes, policies or cultural norms can be brought to bear on them.

Facts, Then Context, Then Opinions (i.e. “an argument”)

The facts that you find are at the core of the report, but they themselves are not the report. Your expert judgment about those facts is also fundamental, but your judgment is not the same kind of thing as a fact. Your judgment is certainly valuable – that’s part of why you were asked to do this – but it’s a different kind of thing. Make that difference obvious in the way you write the report.

Describe the facts of the situation. Provide evidence to substantiate the facts. Put the facts in context by showing how they affect the environment and its security. Then discuss what can be done about them (not what must be done about them). Provide references to reputable independent sources that cover the same topic. If the issue is controversial, include references that take an alternate view. It’s OK to explain why you disagree with that view, but be sure to lay out the whole story so your reader can make an informed decision about what’s right for them.

Inform the reader with facts and references. Persuade the reader with argument. Make your best recommendation, and then leave the final decision to them.

Illustrate, Don’t Decorate

Screenshots can save a lot of typing. A good screenshot can illustrate the core “fact” and also provide much of the needed “context” around it.

A good screenshot is…

• accurate in that it shows a fact that is relevant to the issue at hand.
• precise in that it shows as little else as practical.
• sized so that any relevant text in it is readable and about the same size as the text near it in the report.

A bad screenshot…

• makes the reader figure out which part of it is important.
• is scaled so the text is unreadably small or far larger than the surrounding text.
• makes the reader start to ignore your screenshots so that even the good ones stop helping.

Here’s an example. Can you tell what the problem is, here, in this uncropped screenshot with nothing to direct your attention?

As a general rule, if your webapp screenshot includes the browser chrome, think about whether you can make it more focused. Like this:

Now that’s more clear. The URL is legible. The boxes and arrows force the reader to notice the interesting parts. There’s a caption to put it into words. If you know about webapps, you can’t look at this and not recognize the problem.

You can disagree about the problem, of course. Maybe there’s a Good Reason ™ for not forcing HTTPS. But this screenshot helps you know for sure what you’re disagreeing about, and that’s the whole point.

MSWord Tricks

Now for a few quick things that can save time and help with consistency when writing reports in Microsoft Word.

Instant Screenshots

You can take screenshots directly in Word. Put your cursor where you want the screenshot and then do:

The MS Word window will minimize itself and the entire desktop will gray out slightly. Click and drag a rectangle over what you want in the screenshot. When you release the mouse button, the screenshot will be in your document.

If you want to edit it later, save it as a file first in order to get the full resolution. Right-click and “save as…” then open that file in your image editor.

Word started adding “automatic alt text” to screenshots sometime in 2019. As you might guess, it’s not very good at this, but it can be entertaining. If you’re done being entertained by it, you can disable it via File > Options > Ease of Access > Automate Alt Text. Uncheck that box.

Abusing Autocorrect

The same thing that changes “teh” to “the” can be used to change any text into anything else. The replacement can have formatting embedded in it.

Here’s some possible text (including formatting) I just made up for an example.

Select whatever you want as the replacement text (i.e. the thing you don’t want to have to type again) and copy it to the clipboard. In this example, select all three lines above and tap Ctrl-C.

Then, open the AutoCorrect Options

Notice that the contents of your clipboard are pre-filled in the “replace with” column. Type the abbreviation you want to stand in for this. Make sure it’s not something you’d type as a standalone word so that it doesn’t trigger unexpectedly. Suggestion: start these with the letter ‘i’ (for “insert”) to avoid collisions.

Once this is set, any time you type “issl” it will be replaced with what’s in the second column. To trigger the replacement, your abbreviated text must appear as a word: with whitespace both before and after it.

Legitimate Macros

If you do a thing to blocks of text repeatedly, look at recording a macro for whatever that thing is. You don’t have to write the macro, you can set Word to record, then do the steps, then save the recording.

Then you can re-run what was recorded, or you can edit the macro however you need to, using the code it generated as a guide.

Customize the Quick Access Toolbar

The Quick Access Toolbar is the left side of the title bar of any Word window. By default, it has Save, Undo, and Redo. You can remove those if you don’t want them, and you can add anything you like here, including macros.

To customize this, go to 

File > Options > Quick Access Toolbar

Your Report Matters More than Your Hacking

A test is not a thing you can deliver. Nobody pays for a test. A test is a series of actions that you do in the moment. The thing you deliver – the actual item the customer is paying for – is the report. Most tests fade into memory and blur quickly. The report will live forever. The report will drive decisions. Help your reader make good decisions. Make sure that everything in your report is helpful and clear. Accurate and precise. It matters.

Video Link: