Implementing TCP Stack Stuff

So I'm implementing more and more of the TCP stack. The idea is that if I actually implement a TCP stack (which was expected, in small chunks) then I [1] understand how TCP should operate much better which allows me to [2] write tests much better and [3] identify corner-cases much better.

One of the issues that I ran into tonight reaches back to my idea to run a separate process that's always "listening" for new packets, and does all of the "Segment Arrives" processing (see pp65 in the RFC).

In order to manipulate the TCP stack of some remote host, it is expected that it will be necessary to get one side of the connection into a specific state (e.g., get the remote state to be SYN-SENT, or the local state to be SYN-RECEIVED) and then perform specific operations that test very specific points of the specification.

In order to do this efficiently, it becomes of the utmost importance to be able to get into these states very easily. Connection buildup and tear-down must be simple, as they are not important except for the individual tests that test these portions (and even then, we might want to skip right to receiving the SYN-ACK packet, before sending the ACK packet to enter the Established state).

In order to perform these state transitions gracefully and properly, it becomes necessary to implement the entire TCP stack, and build in hooks that halt at certain points (on a state transition, specific packet sent or received, malformed packet, etc.).

Now, it turns out that I still hadn't done enough planning to foresee the issue with packet ordering. The model that I thought would work would be to call a "recv()" method that would receive the next packet, and then the test would do something interesting with it. That becomes a problem when the test is expecting a specific field -- say, with a specific ACK number -- but receives that packet after receiving some other packet. The test might not properly realize that the first packet (which didn't match the expected pattern) was pertinent, and would be needed later in the test.

This gives two options:

1. The recv() method should only return the next packet (i.e. the segment's sequence number == rcv.nxt)
   
2. The recv() method simply fetches the next packet off of a queue, to which packets are added in-order by a separate thread/process that manages the state.
   

Both of these approaches have their limitations. Consider a test in which we want to send a malformed ACK packet in response to a SYN-ACK packet. The code for that might look something like this:

tsm = TcpStateMachine()
i = tsm.packetsRecvd.size()
syn = tsm.newPacket({'syn':1})
tsm.sendRawTcp(syn)
startTime = time.time()
# Keep looping. Note that this loop would be rolled into a seperate method.
while True:
  # 30 second timeout. Test fails.
  if time.time() < startTime + 30:
    return False
  # If no new packets we received, sleep
  if tsm.packetsRecvd.size() <= i:
    time.sleep(0.1)

  # Loop through available packets.
  for i in range(i, tsm.packetsRecvd.size()):
    p = tsm.packetsRecvd[i]
    # See if it's a SYN-ACK and the ack number matches.
    if p.ack and p.syn and p.ack_number == syn.sequence + 1:
      break
  # Do stuff with packet at offset 'i'.

The code would send a SYN packet, with the other fields auto-filled, then wait until some packet is received, then do processing based on that.

Actually, after writing that, performing some method of callback on a state transition could lead to un-necessary complications. What might be more useful is a way to turn off auto-processing (not receipt of, just processing of) packets upon some trigger (i.e. after transitioning to state 'FIN-WAIT-1', don't do anything with the packets), and a way to see if that trigger has been hit (although checking to see if the state is FIN-WAIT-1 should be sufficient).

Hmm.

Working all of this out in writing might have cleared things up a bit.

Thanks!

Read more...