Holy Crap How Time Flies

Lots of time has gone by since I've had a chance to put some hours into the TCP Regression framework. I haven't forgotten it, just been busy. A trip to New York City, followed by Halloween, and two absolutely gorgeous November sundays kept me away from the computer for the past few weeks.

The coming weekends, since I'm not going home for Thanksgiving, as well as Christmas-time (since most of the other interns leave Dec. 17, and I am staying until Dec. 31) will probably see a spark of productivity.

Read more...

Build Issues when Installing

In the process of investigating the build issues on Mac OS X 10.6 Snow Leopard, I decided to build on FreeBSD.

The combo:

python setup.py config

python setup.py build

sudo python setup.py test

Works just fine. However, once the package is installed... things go haywire (full output below). The issue appears to be in the final linking stage.

make

cc -shared -pthread build/temp.freebsd-7.1-RELEASE-i386-2.5/pcap.o build/temp.freebsd-7.1-RELEASE-i386-2.5/pcap_ex.o -L/usr/lib -lpcap -o build/lib.freebsd-7.1-RELEASE-i386-2.5/pcap.so

make install

cc -shared -pthread build/temp.freebsd-7.1-RELEASE-i386-2.5/pcap.o build/temp.freebsd-7.1-RELEASE-i386-2.5/pcap_ex.o -o build/lib.freebsd-7.1-RELEASE-i386-2.5/pcap.so

For whatever reason, libpcap isn't being linked in. The tests attempt to import pcap, then make sure that it is valid, and if that doesn't work, import pcs.pcap (as seen at the end). When attempting to import the installed version of pcap, the "import pcap" statement fails. Grrrr...

[zjriggl@freebsd ~/zachriggle-pypcap-1f0484c]$ ls

CHANGES LICENSE Makefile README pcap.c pcap.pyx pcap_ex.c pcap_ex.h setup.py tests testsniff.py

[zjriggl@freebsd ~/zachriggle-pypcap-1f0484c]$ make

pyrexc pcap.pyx

python setup.py config

running config

found {'libraries': ['pcap'], 'library_dirs': ['/usr/lib'], 'include_dirs': ['/usr/include']}

python setup.py build

running build

running build_ext

pyrexc pcap.pyx --> pcap.c

building 'pcap' extension

creating build

creating build/temp.freebsd-7.1-RELEASE-i386-2.5

cc -fno-strict-aliasing -DNDEBUG -O2 -fno-strict-aliasing -pipe -D__wchar_t=wchar_t -DTHREAD_STACK_SIZE=0x20000 -fPIC -I/usr/include -I/usr/local/include/python2.5 -c pcap.c -o build/temp.freebsd-7.1-RELEASE-i386-2.5/pcap.o

pcap.c: In function '__pyx_f_4pcap_4pcap_dispatch':

pcap.c:2298: warning: passing argument 3 of 'pcap_dispatch' from incompatible pointer type

pcap.c: In function '__pyx_f_4pcap_4pcap_nextPacket':

pcap.c:2381: warning: passing argument 3 of 'pcap_ex_next' from incompatible pointer type

pcap.c: In function '__pyx_f_4pcap_4pcap_dump':

pcap.c:2902: warning: passing argument 1 of 'pcap_dump' from incompatible pointer type

cc -fno-strict-aliasing -DNDEBUG -O2 -fno-strict-aliasing -pipe -D__wchar_t=wchar_t -DTHREAD_STACK_SIZE=0x20000 -fPIC -I/usr/include -I/usr/local/include/python2.5 -c pcap_ex.c -o build/temp.freebsd-7.1-RELEASE-i386-2.5/pcap_ex.o

pcap_ex.c: In function 'pcap_ex_next':

pcap_ex.c:272: warning: passing argument 3 of 'pcap_next_ex' from incompatible pointer type

creating build/lib.freebsd-7.1-RELEASE-i386-2.5

cc -shared -pthread build/temp.freebsd-7.1-RELEASE-i386-2.5/pcap.o build/temp.freebsd-7.1-RELEASE-i386-2.5/pcap_ex.o -L/usr/lib -lpcap -o build/lib.freebsd-7.1-RELEASE-i386-2.5/pcap.so

[zjriggl@freebsd ~/zachriggle-pypcap-1f0484c]$ sudo make test

python setup.py test

running test

testDispatch (tests.testPcap.TestPcap) ... ok

testEnumerateInterfaces (tests.testPcap.TestPcap) ... ok

testErrors (tests.testPcap.TestPcap) ... ok

testInjectedPacketIsReceived (tests.testPcap.TestPcap) ... ok

testIter (tests.testPcap.TestPcap) ... ok

testIterable (tests.testPcap.TestPcap) ... ok

testOpenDefaultInterface (tests.testPcap.TestPcap) ... ok

testOpenLive (tests.testPcap.TestPcap) ... ok

testPacketFilter (tests.testPcap.TestPcap) ... ok

testProperties (tests.testPcap.TestPcap) ... ok

testReadpkts (tests.testPcap.TestPcap) ... ok

testWriteDump_ReadOffline (tests.testPcap.TestPcap) ... ok

----------------------------------------------------------------------

Ran 12 tests in 1.541s

[zjriggl@freebsd ~/zachriggle-pypcap-1f0484c]$ sudo make install

python setup.py install

running install

running build

running build_ext

pyrexc pcap.pyx --> pcap.c

building 'pcap' extension

cc -fno-strict-aliasing -DNDEBUG -O2 -fno-strict-aliasing -pipe -D__wchar_t=wchar_t -DTHREAD_STACK_SIZE=0x20000 -fPIC -I/usr/local/include/python2.5 -c pcap.c -o build/temp.freebsd-7.1-RELEASE-i386-2.5/pcap.o

pcap.c: In function '__pyx_f_4pcap_4pcap_dispatch':

pcap.c:2298: warning: passing argument 3 of 'pcap_dispatch' from incompatible pointer type

pcap.c: In function '__pyx_f_4pcap_4pcap_nextPacket':

pcap.c:2381: warning: passing argument 3 of 'pcap_ex_next' from incompatible pointer type

pcap.c: In function '__pyx_f_4pcap_4pcap_dump':

pcap.c:2902: warning: passing argument 1 of 'pcap_dump' from incompatible pointer type

cc -fno-strict-aliasing -DNDEBUG -O2 -fno-strict-aliasing -pipe -D__wchar_t=wchar_t -DTHREAD_STACK_SIZE=0x20000 -fPIC -I/usr/local/include/python2.5 -c pcap_ex.c -o build/temp.freebsd-7.1-RELEASE-i386-2.5/pcap_ex.o

pcap_ex.c: In function 'pcap_ex_next':

pcap_ex.c:272: warning: passing argument 3 of 'pcap_next_ex' from incompatible pointer type

cc -shared -pthread build/temp.freebsd-7.1-RELEASE-i386-2.5/pcap.o build/temp.freebsd-7.1-RELEASE-i386-2.5/pcap_ex.o -o build/lib.freebsd-7.1-RELEASE-i386-2.5/pcap.so

running install_lib

copying build/lib.freebsd-7.1-RELEASE-i386-2.5/pcap.so -> /usr/local/lib/python2.5/site-packages

running install_egg_info

Removing /usr/local/lib/python2.5/site-packages/pcap-1.2-py2.5.egg-info

Writing /usr/local/lib/python2.5/site-packages/pcap-1.2-py2.5.egg-info

[zjriggl@freebsd ~/zachriggle-pypcap-1f0484c]$ sudo make test

python setup.py test

running test

Traceback (most recent call last):

File "setup.py", line 120, in

provides = ['pcap'] )

File "/usr/local/lib/python2.5/distutils/core.py", line 151, in setup

dist.run_commands()

File "/usr/local/lib/python2.5/distutils/dist.py", line 974, in run_commands

self.run_command(cmd)

File "/usr/local/lib/python2.5/distutils/dist.py", line 994, in run_command

cmd_obj.run()

File "setup.py", line 91, in run

from tests.testPcap import PcapTestSuite

File "/usr/home/zjriggl/zachriggle-pypcap-1f0484c/tests/testPcap.py", line 17, in

import pcs.pcap as pcap

ImportError: No module named pcs.pcap

*** Error code 1

Stop in /usr/home/zjriggl/zachriggle-pypcap-1f0484c.

Read more...

PCS-0.5 Diff with PCS version used in TcpRegression

I've created a Diff file that shows all of the changes that I made to PCS over the course of the summer. Some of these might be merged back into the PCS mainline, others might not.

The highlight of the changes are:

• BitString class. The code for encoding individual bits, even if there is a lack of alignment on 8-bit boundaries, is a bit iffy (even the code says "This makes my head hurt"). The BitString class allows manipulation of individual bits just like a Python list object, and can be translated into an integer value, string value, binary string (via bin()), hex string (via hex()), and allows arbitrary manipulation.
• The names of the pcs.packets.tcp.tcp class fields were assigned variables. Instead of object['fieldName'] which is prone to typos going undetected, I made a bunch of variables so that they can be referred to as object[f_fieldname]. If mistyped, the interpreter picks up on the nonexistent variable.

GNN: I've explicitly emailed you about this, as this was something that you requested.

Everyone else can get a copy here:

http://p4db.freebsd.org/fileDownLoad.cgi?FSPC=//depot/projects/soc2009/zjriggl%5ftcpregression/diff/pcs%2d0.5.diff&REV=1

From looking at the diff, it does not appear that my changes rely on anything in the forked PCap library (and for that matter, don't involve PCap at all).

Zach

Read more...

Yep, 10.6 Did It (Methinks)

The build.py script was only set to look in the first include directory for pcap.h. On 10.6, /usr/include/pcap.h includes /usr/local/pcap/pcap.h, which contains all of the definitions the build system was looking for.

Fixed the build script, will add it to Git soon. Read more...

10.6 Breaks PyPcap?

I think Snow Leopard breaks PyPcap -- for whatever reason, I'm getting compile errors that didn't previously exist. I'll have to boot up one of the VMs and see if everything still works OK, or if there was some change that I didn't properly test that I also completely forgot that I made in the first place. Read more...

Docs Added

Lots of docs added to some of the core components. I'm on my way to the Apple store to have them fix this piece-o-junk, so hopefully I'll have it back soon!

Read more...

Committed all changes

All changes have been committed to P4 (TcpRegression) and Git (PyPcap). Eveyrthing works on my dev box, but we all know that means nothing. I'll sort out any odds and ends tomorrow. I need to get up for work in 3 hours.

- Zach

Read more...