Testing, testing

I have been testing the Realeyes IDS at a local college for about 8 months now. However, it took almost 10 months of planning before the testing began. I contacted a half dozen sites and only the one felt capable of letting me set up the system in their environment. Looking at this from the point of view of the sites contacted, I consider myself quite lucky to have had this response. My advice to anyone looking for a similar situation is:

• Be clear: Write a letter or email that explains what you would like to do concisely and give some background on yourself to build credibility.


• Be professional: In your first face to face meeting, have a presentation that covers the main points of your project, explain how your project might provide the site something in return, and dress conservatively (I wore my best suit). We have even signed an agreement which spelled out what was to be provided by both parties, including hardware, software, and time. Now that we have been working together for a while, the relationship has become less formal, but initially I believe the formality gave them confidence that they would not be creating problems for themselves.


• Don't get frustrated: The site I am working with handled several major tasks while I was waiting for them to provide me with a single host and a single connection to the monitoring port of a switch. To me, it wasn't asking for much, but now that I have been there, I can see that for them it was quite a bit of time and planning.


• Be gracious: I have found opportunities to thank the people I am working with, including management and sysadmins, at least twice a month. I have also pointed out how well run their operation is (and it is, so I'm not just brown-nosing).
  

What they provided for me was a 733MHz CPU with 2Gig of RAM, a 100Mbps network interface, and a 16Gig hard disk. The most important issue for me was the memory, so 2Gig of RAM is fantastic. As far as everything else goes, I would rather test on moderate equipment and make my code more efficient to get adequate performance than have the platform hide problems.

Of course, the least of my worries was about problems being hidden. In the first couple of weeks, the IDS failed within less than 1 hour. First it was buffer space issues, next it was bugs, then it was buffer space issues again. But after a month, I had it running long enough to actually detect a few incidents. Then, over the next several months I cleaned up formatting issues, improved the user interface, and fixed more IDS bugs.

In the meantime, I have been able to give the site some feedback on their environment. I have not created a lot of rules, but there is a fair amount of variety to those that are in use, and some were in response to information that they wanted to collect. The most interesting ones, for all of us, have been:

• Non-http traffic on port 80: This reported very few hits, but the ones it did report gave them enough info to correct the use of a couple of applications


• Brute force FTP logins: This just gave them more detail than what they were already seeing in logs, but at least it showed that none of the attempts were successful


• Activity at unusual times: By monitoring Email servers between midnight and 5:00 am, we have seen a few cases of spam from site hosts, and some other activity that led to them discuss policy


• Invalid TCP options: We are working on this one, stay tuned


• Rules for specific exploits: Between the low number of these and the sysadmins' efforts to harden their site, there have not been any serious (or from my perspective, spectacular) hits on these, which from their perspective is good news
  

Overall, I count this experience as a huge success. And the best news is that a couple of months ago, I finally fixed the main buffering issue. So while there are still some bugs, the system has been stable enough to run for days in a row (as opposed to hours). And now it is reassembling and analyzing as many as 35,000 simultaneous TCP sessions.

I never would have reached this point without the help of the people at the test site. So once again, I want to say, "Thanks."

Later . . . Jim