Docs Added

Lots of docs added to some of the core components. I'm on my way to the Apple store to have them fix this piece-o-junk, so hopefully I'll have it back soon!

Read more...

Committed all changes

All changes have been committed to P4 (TcpRegression) and Git (PyPcap). Eveyrthing works on my dev box, but we all know that means nothing. I'll sort out any odds and ends tomorrow. I need to get up for work in 3 hours.

- Zach

Read more...

And so it ends

SoC is coming to an end tonight.

Overall, I think the project went very well. I'm going to try to bust out a few deliverable tests tonight since those were the original goal, but I think the volume of work that I got done will speak for itself.

• I forked PyPcap, fixed bugs, extended the functionality, wrote better tests.
  
• I forked PCS-0.5, fixed bugs, extended the functionality (although I didn't write tests for PCS).
  
• I've essentially implemented a reference userland implementation of TCP on top of PCS and PyPcap, which should allow for even more applications than a simple regression testing suite, complete with tests of the code itself, as well as a few deliverable tests that should validate functionality of various TCP stacks.
  

In addition to continuing work with the TcpRegression suite, I've sent an email to the Metasploit project to see if they could use another leisure-pace developer. That should be an interesting project, and maybe I'll be able to apply some of the knowledge about TCP that I gained over the summer. Overall, it's been very productive - for my personal benefit (fiscal and intellectual), and hopefully the FOSS community and FreeBSD as well.

I've got a few minor changes to PyPcap that need to go up -- mostly it's just the inclusion of a function that will print out a string of bytes in the same format that they appear in Wireshark's Packet Bytes view.

There's a load of TcpRegression functionality and tests that will be in the next commit. Over the next week or so, I'll work on the documentation and cleaning up the code, and getting Google their code sample. I'd like a nice, solid "0.1" release. It's also crossed my mind to separate the regression tests themselves from the main framework, and re-badging it "TCPython". The name looks like it hasn't been taken yet, but we'll see how that goes.

Special thanks to Titus Brown and George Neville-Neil for helping me throughout the summer, you both helped me out of a few ruts along the way that could've made the whole project a lot less enjoyable.

Read more...

Published changes...

Published changes to the PyPcap library. Get the new hotness here:

http://zachriggle.github.com/pypcap/
or
git clone git://github.com/zachriggle/pypcap

This build tested on Mac OS X 10.5.7 and FreeBSD 7.1

CHANGELOG

pypcap-1.2:
- included changes from George Neville-Neil's PCS (http://pcs.sf.net)
- added more tests
- added logging functionality
- implemented patches provided by several users from the original PyPcap site on Google Code. Thanks to...
getxsick
dirk-loss.de

- Changed some interfaces, but made sure to allow backwards compatibility.
Examples:

__init__ now has separate args for interface, filename, and dump file.
dump_close from PCS is now closeDumps
the filter can be set by accessing the '.filter' property
each pcap object can now be re-opened after closing with

- openLive
- openOffline
- openDump

- exposed findalldevs() [getxsick]
- exposed lookupnet() [getxsick]
- exposed 'cnt' arg of loop() [getxsick]
- fixed setnonblock, loop() [dirk-loss.de]

Read more...

Busy Day

Busy day. Added some more PyPcap functionality, fixed some bugs, and implemented tests. All of the functionality is at least touched by the tests...
- Live capture
- Dumping to file
- Reading from file
- Packet filter

Additionally, migrated the 'tcpfilter' class to use bpf instead of manually inspecting the fields of each packet after it had been constructed by PCS. This should be MUCH faster.

Also learned that in Python...

class X():
....y = someObject()

... means that all X objects will be instantiated with the *same instance* of someObject, that is instantiated when the class is declared, instead of each object being allocated when the class instance is created. Weird. I thought it was run when the class was instantiated, and just allowed for the constructor to be a little bit less cluttered. Guess I was wrong (or that there's a Python bug).

Read more...

C'mon Nose, stop being a stupid WHAAAAAAAAAT

In python...

>>> t = TestPcap('')
>>> t.testEnumerateInterfaces()
>>> t.testOpenDefaultInterface()
>>> t.testOpenSpecificInterface()

But...

zach@Zachs-Computer:~/Documents/workspace/zjriggl_tcpregression/src/pcs/pcap/tests$nosetests testPcap.py
E.E
======================================================================
ERROR: testEnumerateInterfaces (testPcap.TestPcap)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/zach/Documents/workspace/zjriggl_tcpregression/src/pcs/pcap/tests/testPcap.py", line 33, in testEnumerateInterfaces
listOfIfs = pcap.findalldevs()
AttributeError: 'module' object has no attribute 'findalldevs'

======================================================================
ERROR: testOpenSpecificInterface (testPcap.TestPcap)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/zach/Documents/workspace/zjriggl_tcpregression/src/pcs/pcap/tests/testPcap.py", line 38, in testOpenSpecificInterface
iface = pcap.findalldevs()[0]
AttributeError: 'module' object has no attribute 'findalldevs'

----------------------------------------------------------------------
Ran 3 tests in 0.004s

FAILED (errors=2)

Here's the output of a "print dir(pcap)" from within the test, run through nosetests.

-------------------- >> begin captured stdout << ---------------------
['DLT_ARCNET', 'DLT_AX25', 'DLT_CHAOS', 'DLT_EN10MB', 'DLT_EN3MB', 'DLT_FDDI', 'DLT_IEEE802', 'DLT_LINUX_SLL', 'DLT_LOOP', 'DLT_NULL', 'DLT_PFLOG', 'DLT_PFSYNC', 'DLT_PPP', 'DLT_PRONET', 'DLT_RAW', 'DLT_SLIP', '__author__', '__builtins__', '__copyright__', '__doc__', '__file__', '__license__', '__maintainer__', '__name__', '__revison__', '__url__', '__version__', 'bpf', 'calendar', 'dltoff', 'ex_name', 'lookupdev', 'pcap', 'sys', 'time']
--------------------- >> end captured stdout << ----------------------

Obviously, it's not there. Pcap is imported as:

try:
import pcs.pcap as pcap
except:
import pcs

Here's the output of "print pcs.__file__" from Python:
/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/pcs/pcap.so

Aaaand from Nose-tests:
/Library/Python/2.5/site-packages/pcs/pcap.so

Whaaaaaaaaaaat?

Read more...

Time

Lost track of time tonight, stayed up way later than I wanted to. Overhauled a LOT of code in the PyPcap library (both C any Python code). It's now a lot cleaner, and a lot of the functionality that was only available via creating a new pcap object is now available at runtime (i.e. open/reopen an interface or offline capture).

Also changed a bit of the Windows code that (for whatever reason) attempted to implement pcap_lookupdev, even though WinPcap has that function. Other minor changes were included in the C code (a little bit of refactoring).

The Python code now includes logging for most of its functionality, which is disabled by default. It does require either [1] modifying the PYX file or [2] modifying pcap.DEBUG_LEVEL before instantiating the pcap object that logging is desired for. Lots of refactoring. Hopefully I didn't break anything.

Must. Get. To. Sleeeeeeeeeeep.

Read more...