Editing Invalid Characters in text files

So last night I was working with @bwall last night on his tool distributed-hash-solving, I ran into an issue in my MD5 pot file. It was showing NUL and other characters when viewing it in Notepad++.

After a little google-fu, I figured out a fix for it and then realized it was more then just NUL characters, it was almost every ASCII character that could be input as a two/three letter character.

We start with opening N++ and getting a sample of the character we need to remove.


As you can see, it decided to input between the hash and the correct output of the has. A text sample of it would be:


Now the fix for this is hitting CTRL-F and choosing the Replace tab and choosing the corresponding ASCII hex character. In this case, it’s \x00 :


Now you want to select Replace ALL. It took less than a minute and replaced over 150 instances of the NUL character. The output came to this:


As you can see, properly fixed!

Now with that one character being fixed, I also ran into almost every other possible HEX character being thrown into that file. The quick list of these to check for is:


\x00 – \x08  (9 actually counting as a character in some passwords you don’t want to remove this)
\x0B – \x0F  (A would do the same thing as \n, or your enter key, so again you don’t want to remove this)
\x10 – \x19
\x1A – \x1F

To ensure you ARE finding the correct characters, I recommend you hit the find next button before replacing to ensure there IS an actual character that needs to be replaced AND that you aren’t going to goof up your .pot/.txt/* file like I did the first time I did this.

If you have any questions feel free to contact me on freenode #hacktalk #intern0t #isdpodcast or #offtopicsec and on twitter @drb0n3z.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s