Checksumming No Se Funciona

Alright, so the last part of the puzzle that needs to fall in place is the checksum. Conveniently, the algorithm for checksumming IP and TCP packets is the same (albeit you feed them different sets of information).

The algorithm works somewhat like so (source IBM):

ushort checksum16(uchar* data, int len)
{
    uint sum = 0;
    if ((len & 1) == 0)
        len = len >> 1;
    else
        len = (len >> 1) + 1;
    while (len > 0) {
        sum += *((ushort*)data);
        data += sizeof(ushort);
        len--;
    }
    sum = (sum >> 16) + (sum & 0xffff);
    sum += (sum >> 16);
    return htons(~sum);
}

That is, it sums all of the 16-bit words, does some 1's compliment fanciness, and then returns the 16 least significant bits (inverted). Of course, the operation can be done with any arbitrary multiple of 16 bits. Since most machines nowadays use 32- or 64-bit arithmetic, there's no reason not to do it with 32 bits (or 64). However, the final result is always 16 bits wide.

In Python, the code looks like (or so I think) the following:

def checksum( bytes ):
    tmpbytes = bytes
    total = 0
    # Must be a number of bytes equal to a multiple of two.
    if len( tmpbytes ) % 2 == 1:
        tmpbytes += "\0"
    # For each pair, add the value to the total.
    for i in range( len( tmpbytes ) / 2 ):
        total += ( unpack( "!H", tmpbytes[( 2 * i ):( 2 * i ) + 2] )[0] )
    # Keep shifting
    while total >> 16:
        total = ( total >> 16 ) + ( total & 0xffff )
    # One's complement
    total = ~total
    # Make into 2 bytes
    total = total & 0xffff
    # Return value
    return total

However, given an IP layer: 0x4500004096da40004006eca1c0a80168d04524e6, the checksum is 0xECA1. When calculating the checksum, the checksum field is set to zero (0x0000). So, the checksum is calculated like so:

>>> from binascii import *
>>> from struct import *
>>> data = unhexlify('4500004096da40004006<b>0000</b>c0a80168d04524e6')
>>> data
'E\x00\x00@\x96\xda@\x00@\x06\x00\x00\xc0\xa8\x01h\xd0E$\xe6'
>>> cksum = checksum(data)
>>> cksum
60577
>>> hex(cksum)
'0xeca1'

Obviously, the end result is incorrect. Turns out that the values that I was using for testing were incorrect. I've since fixed them in the blog post and the offending code.

Read more...