There is a new download available for the Realeyes IDS version 0.9.5. Read the release notes for details, but essentially, this release is about new user interface features. The screenshots have been updated and there are two new demos, one on installation and configuration, and the other on the new features. There are links to all of this on the technology page
The college where I have been running a pilot project was able to provide an upgraded host system. This led to the discovery of a couple of issues with the IDS sensor software that I had not seen before. One was the signal handler for user interrupts, and the other was in releasing locks if shutting down. Both of these are fixed, although I might make some further improvements before the next release.
The big news is that the college security software faculty heard about the pilot project and is going to use Realeyes in the curriculum. This means that I am now running the system in a lab environment. And all of this means it gets tested more thoroughly. I did find a couple configurations that would not run successfully, so hopefully, the installation process is more solid.
I am hoping that this will help build the community around the project. I have had some interest expressed in working on code, as well as testing it in other environments. So it is moving in the right direction, and with a little luck and some elbow grease, the next release will be 1.0.
Later . . . Jim
Monday, May 25, 2009
Wednesday, April 29, 2009
Command Line vs. GUI Reality Check
I was reading Kyle Rankin and Bill Childer's Point/Counterpoint column on "Mutt vs. Thunderbird" in Linux Journal over breakfast the other day. It mostly boiled down to the perennial text vs. graphical user interface argument. And since I don't have a strong opinion about either mail client, it got me to thinking about the real difference between the two interfaces.
Before I dive into the firepit, let me explain that I spent the first half of my career in the IBM mainframe world, which meant writing Job Control Language (JCL) to submit batch jobs. Part of that time, I spent maintaining a TCP/IP stack written in assembler language. Compared to that, the distance between the command line and the GUI is much smaller than their advocates seem to realize.
To be honest, I didn't use GUIs for a quite a while after they became available. I didn't fight them so much as I just didn't find them intuitive or efficient. However, as usability studies improved the interfaces and processor speeds increased their capabilities, I gradually came to appreciate what they provide.
I think that much of the bias in favor of GUIs comes from the saying, "A picture is worth a thousand words." And certainly, pictures of cityscapes or family reunions or battlefields convey much more information than even the best description. However, imagine having to draw a picture to say, "I'm thirsty," or, "Turn left at the next light," or even to describe how to use a graphical user interface to a first time user. Very often, a few words are more efficient than any picture.
Obviously, each interface has its strengths and weaknesses. So what I was wondering, as I read Bill and Kyle, was whether it is possible to generalize the strengths of each. I'm not suggesting that I have it completely figured out, but I think there are, in fact, some common elements for the situations in which each is superior to the other.
To begin with, I am assuming that the interface in question has gone through enough testing and real-world use to be very good at what it is supposed to do. The Linux/Unix command line's many utilities that can be piped or scripted to create miniature applications is the best example of that interface. And I prefer the GNU collection because of the extra arguments that have been added by developers scratching an itch. Meanwhile, the current state of the art web browsers are good examples of GUIs.
Command line interfaces are best for data that is concise and reasonably well known in advance. For example, comparing the dates of two copies of the same file in different directories:
Graphical user interfaces are best for displaying somewhat large amounts of data that needs to be analyzed to determine what to do next. When a web page is loaded in a browser, it is usually not known in advance what link(s) will be selected. A window with a helpful design to guide the user to the proper selection and a mouse pointer that can be moved directly to the appropriate target is a highly effective use of this interface.
With this in mind, I started thinking about the use of both interfaces in the Realeyes IDS.
The installation is performed by executing several shell scripts on the command line. Many of the responses have a default, which allows the Enter key to be pressed for a majority of the responses. And where that is not possible, simple information, such as an IP address or a single name, is entered. To begin with, a graphical environment is not required. And second, there is no switching between the keyboard and the mouse. This makes deploying a new sensor or updating the application very fast after the first couple of times.
In contrast, the analysis window displays large amounts of information that may not be selected in a sequential order. Having the data in a scrollable list allows me to focus on one column, while watching another with my peripheral vision. This allows me to see patterns that might not be apparent in a text display.
Another advantage of the GUI comes when I want to view the session data. This is done by right clicking on a report and selecting Playback from the popup menu displayed. When the reports are sorted by Event, all of those with the same Event are similar enough that a quick glance at the session data is sufficient to determine whether it warrants more attention. Then a single click closes the window. This means that I can often rip through reports, spending only 2 or 3 seconds on most of them.
The GUI also provides tabs that allow me to search for trends or get summary reports without losing my focus on the original display. And the status frames automatically notify me if there is something that needs attention, without my having to query for it.
There are some administrative functions that could be performed from the command line as easily as the GUI, and in the early versions of the Realeyes IDS, it was the only way to do that. However, having them incorporated into the GUI is very convenient.
The downside of this is a lack of flexibility. In order for a capability to be available, there must be code in the GUI application. The command line gives an administrator complete control of maintenance procedures, and under certain circumstances, this is the only option.
From a design perspective, the choice of command line vs. GUI seems pretty straightforward. First, how quickly does the code need to be produced? Second, which interface makes the user most productive? While there is plenty of room for different points of view on the answers to these questions, it is simply not true that one is always better than the other.
Later . . . Jim
Before I dive into the firepit, let me explain that I spent the first half of my career in the IBM mainframe world, which meant writing Job Control Language (JCL) to submit batch jobs. Part of that time, I spent maintaining a TCP/IP stack written in assembler language. Compared to that, the distance between the command line and the GUI is much smaller than their advocates seem to realize.
To be honest, I didn't use GUIs for a quite a while after they became available. I didn't fight them so much as I just didn't find them intuitive or efficient. However, as usability studies improved the interfaces and processor speeds increased their capabilities, I gradually came to appreciate what they provide.
I think that much of the bias in favor of GUIs comes from the saying, "A picture is worth a thousand words." And certainly, pictures of cityscapes or family reunions or battlefields convey much more information than even the best description. However, imagine having to draw a picture to say, "I'm thirsty," or, "Turn left at the next light," or even to describe how to use a graphical user interface to a first time user. Very often, a few words are more efficient than any picture.
Obviously, each interface has its strengths and weaknesses. So what I was wondering, as I read Bill and Kyle, was whether it is possible to generalize the strengths of each. I'm not suggesting that I have it completely figured out, but I think there are, in fact, some common elements for the situations in which each is superior to the other.
To begin with, I am assuming that the interface in question has gone through enough testing and real-world use to be very good at what it is supposed to do. The Linux/Unix command line's many utilities that can be piped or scripted to create miniature applications is the best example of that interface. And I prefer the GNU collection because of the extra arguments that have been added by developers scratching an itch. Meanwhile, the current state of the art web browsers are good examples of GUIs.
Command line interfaces are best for data that is concise and reasonably well known in advance. For example, comparing the dates of two copies of the same file in different directories:
- ls -l /a/b/c/file /d/e/f/file
- ps aux | grep program
- /etc/init.d/daemon start
/etc/init.d/daemon stop
Graphical user interfaces are best for displaying somewhat large amounts of data that needs to be analyzed to determine what to do next. When a web page is loaded in a browser, it is usually not known in advance what link(s) will be selected. A window with a helpful design to guide the user to the proper selection and a mouse pointer that can be moved directly to the appropriate target is a highly effective use of this interface.
With this in mind, I started thinking about the use of both interfaces in the Realeyes IDS.
The installation is performed by executing several shell scripts on the command line. Many of the responses have a default, which allows the Enter key to be pressed for a majority of the responses. And where that is not possible, simple information, such as an IP address or a single name, is entered. To begin with, a graphical environment is not required. And second, there is no switching between the keyboard and the mouse. This makes deploying a new sensor or updating the application very fast after the first couple of times.
In contrast, the analysis window displays large amounts of information that may not be selected in a sequential order. Having the data in a scrollable list allows me to focus on one column, while watching another with my peripheral vision. This allows me to see patterns that might not be apparent in a text display.
Another advantage of the GUI comes when I want to view the session data. This is done by right clicking on a report and selecting Playback from the popup menu displayed. When the reports are sorted by Event, all of those with the same Event are similar enough that a quick glance at the session data is sufficient to determine whether it warrants more attention. Then a single click closes the window. This means that I can often rip through reports, spending only 2 or 3 seconds on most of them.
The GUI also provides tabs that allow me to search for trends or get summary reports without losing my focus on the original display. And the status frames automatically notify me if there is something that needs attention, without my having to query for it.
There are some administrative functions that could be performed from the command line as easily as the GUI, and in the early versions of the Realeyes IDS, it was the only way to do that. However, having them incorporated into the GUI is very convenient.
The downside of this is a lack of flexibility. In order for a capability to be available, there must be code in the GUI application. The command line gives an administrator complete control of maintenance procedures, and under certain circumstances, this is the only option.
From a design perspective, the choice of command line vs. GUI seems pretty straightforward. First, how quickly does the code need to be produced? Second, which interface makes the user most productive? While there is plenty of room for different points of view on the answers to these questions, it is simply not true that one is always better than the other.
Later . . . Jim
Tuesday, March 17, 2009
Database Security
I saw the following on the webappsec list at Security Focus:
I don't disagree with Taufiq's assessment. However, I do disagree with his acceptance of the status quo. I wrote a rant on this blog responding to a complaint that security professionals are not taken seriously. In it, I pointed out that the security industry should promote improving the security climate, not just react to it with solutions 'for a price'. The example I gave was *DBC libraries.
The JDBC package, java.sql, does not supply any security parsing. This is not the real workhorse, but it should at least provide a method for this. Each database supplies a jar that java.sql classes call to access the specific database. This is where security parsing must be handled.
The thing is that parsing input is tricky. The first step is to validate that the input is correct for the column data type. This is reasonably straightforward for simple types like integer and varchar. But the way different databases support binary data and very large fields is not consistent. There is also support for non-standard data, such as PostgreSQL's support for the inet data type.
The JDBC Connection interface includes the getMetaData method, which returns the information supplied by the specific database library, some of which is unique to that database. There are not only differences between databases, it is even possible that there are differences between versions of the same database. This could be an issue for an application because:
The next step is to escape all characters that have special meaning, such as single quote and backslash. But again, each database has its own special characters that must be accounted for, such as ampersand in Oracle, and the E'...' escape pattern in PostgreSQL.
Update: Eric Kerin points out in his comments that the PreparedStatement interface does this, and after some testing I have found that this is the case. My excuse is that there is nothing in the javadoc for the SQL package or the PreparedStatement interface that explains this. Instead the documentation promotes it for optimizing frequently used statements. See my reply below for further responses to comments.
Also, there is a good article on this issue at the Open Web Application Security Project, which I found by googling for java and "sql injection".
The current situation places the responsibility for security on the thousands of application programmers, who must now dig into the internals of the database(s!) on the backend of their applications. If instead, the database development teams provided a parser for each field of data, it would be possible to determine if the input would result in a message, like this one that I was able to create from testing various input sequences:
I'm still working on parsing that construct and reworking it in a way that does not reject the data out of hand, because it might be a legitimate description of an Event, or possibly a Trigger definition. I am fortunate, because the input is not read from the network. You might not be so lucky.
And before I leave this topic/rant, I must point out that application programmers need to work closely with their DBAs to be sure that permissions are set on tables to allow only as much access as absolutely necessary and no more. If you don't have a DBA and/or maintain the database yourself, you need to become very familiar with the levels of GRANTing access and the use of roles to at least limit the damage when SQL injection attacks succeed. In my own experience, as well as reports from others, the attacks on applications, and databases especially, is continuing to increase.
If anyone is interested in the database security in the Realeyes UI, check out the Database and ValidatorDBForm modules, and then see how they are used in any of the Window*Base.java modules at the UI subversion repository. The ValidatorDBForm class includes the InterfaceAdminExtendedProcessing interface to do extra contextual error checking, which really is the job of the application. There are some pretty good examples of its use in the WindowsRules(Triggers/Actions/Events).java modules.
I'm pretty sure I'm talking for all application developers when I say (as a security guy), "Hey database developers, a little help!"
Later . . . Jim
| I've heard this preached before.
|
| Using JDBC properly can help protect against SQL Injection.
|
| What protections does JDBC provide?
|
| Does java encode the input to not be malicious?
|
| I'm curious where in the java source/libraries does jdbc help
| to mitigate malicious input when using jdbc.
|
This preach is applicable for any programming language. It
all depends on how well you have done input & output
validation. As in what input you expect & what input is
malicious for your app. If all goes well you can make SQL
injection very difficult or even impossible . The reason I
say difficult, because it all depends on how well the SQL
injection is crafted. As far as I recollect I don't think
JDBC or for that case even java gives you predefined class
for doing that. But there is quite a possibility that some
one on the internet must have surely written these classes.
--
Taufiq
http://www.niiconsulting.com/products/iso_toolkit.html
I don't disagree with Taufiq's assessment. However, I do disagree with his acceptance of the status quo. I wrote a rant on this blog responding to a complaint that security professionals are not taken seriously. In it, I pointed out that the security industry should promote improving the security climate, not just react to it with solutions 'for a price'. The example I gave was *DBC libraries.
The JDBC package, java.sql, does not supply any security parsing. This is not the real workhorse, but it should at least provide a method for this. Each database supplies a jar that java.sql classes call to access the specific database. This is where security parsing must be handled.
The thing is that parsing input is tricky. The first step is to validate that the input is correct for the column data type. This is reasonably straightforward for simple types like integer and varchar. But the way different databases support binary data and very large fields is not consistent. There is also support for non-standard data, such as PostgreSQL's support for the inet data type.
The JDBC Connection interface includes the getMetaData method, which returns the information supplied by the specific database library, some of which is unique to that database. There are not only differences between databases, it is even possible that there are differences between versions of the same database. This could be an issue for an application because:
Some DatabaseMetaData methods return lists of information in the form of ResultSet objects. Regular ResultSet methods, such as getString and getInt, can be used to retrieve the data from these ResultSet objects.
The next step is to escape all characters that have special meaning, such as single quote and backslash. But again, each database has its own special characters that must be accounted for, such as ampersand in Oracle, and the E'...' escape pattern in PostgreSQL.
Update: Eric Kerin points out in his comments that the PreparedStatement interface does this, and after some testing I have found that this is the case. My excuse is that there is nothing in the javadoc for the SQL package or the PreparedStatement interface that explains this. Instead the documentation promotes it for optimizing frequently used statements. See my reply below for further responses to comments.
Also, there is a good article on this issue at the Open Web Application Security Project, which I found by googling for java and "sql injection".
The current situation places the responsibility for security on the thousands of application programmers, who must now dig into the internals of the database(s!) on the backend of their applications. If instead, the database development teams provided a parser for each field of data, it would be possible to determine if the input would result in a message, like this one that I was able to create from testing various input sequences:
WARNING: nonstandard use of \' in a string literal
ERROR: relation "foobar" does not exist
STATEMENT: select foo from foobar
I'm still working on parsing that construct and reworking it in a way that does not reject the data out of hand, because it might be a legitimate description of an Event, or possibly a Trigger definition. I am fortunate, because the input is not read from the network. You might not be so lucky.
And before I leave this topic/rant, I must point out that application programmers need to work closely with their DBAs to be sure that permissions are set on tables to allow only as much access as absolutely necessary and no more. If you don't have a DBA and/or maintain the database yourself, you need to become very familiar with the levels of GRANTing access and the use of roles to at least limit the damage when SQL injection attacks succeed. In my own experience, as well as reports from others, the attacks on applications, and databases especially, is continuing to increase.
If anyone is interested in the database security in the Realeyes UI, check out the Database and ValidatorDBForm modules, and then see how they are used in any of the Window*Base.java modules at the UI subversion repository. The ValidatorDBForm class includes the InterfaceAdminExtendedProcessing interface to do extra contextual error checking, which really is the job of the application. There are some pretty good examples of its use in the WindowsRules(Triggers/Actions/Events).java modules.
I'm pretty sure I'm talking for all application developers when I say (as a security guy), "Hey database developers, a little help!"
Later . . . Jim
Saturday, March 14, 2009
Java Search
I think I'm finally getting the hang of Object Oriented programming. I have been working on the user interface to provide all administration from it and add quite a bit of usability.
Over the past few days, I added search to the playback window. Because the playback window has two frames (Text classes in Java), this is a bit trickier than your standard text search. To begin with, I am allowing the search to be limited to one or the other frame, as well as using both. This means that the search class has to be aware of each frame.
To be able to highlight the text, the Text class actually needs to be defined as StyledText. When the text is found, the replaceStyleRanges method is called to highlight it. For now, I am leaving it highlighted, thinking that it is more helpful to be able to see all of the found selections. The current found text is displayed in reverse video by using the setSelection method. This one has to be reset by setting the selection range to zero before setting the new selection.
I thought about being able to use a single find window to search multiple playback windows, but this made my head hurt. However, it did seem friendly to share the search strings between playback windows. So I created a string array in the global variables class, and store the strings there. I even save them to the preference store so that they are maintained over application restarts.
If you are interested in this code, check out the WindowPlayback*.java source modules in the subversion repository. The preference store is defined and initialized in the Globals.java and MainWindow.java modules.
The beauty of the OOP style is that almost all of my code is spent managing indexes. The heavy lifting is done by other classes and their methods. So I am hopeful that these GUI enhancements will be finished pretty soon, and I will build another set of download packages by the end of the month.
Later . . . Jim
Over the past few days, I added search to the playback window. Because the playback window has two frames (Text classes in Java), this is a bit trickier than your standard text search. To begin with, I am allowing the search to be limited to one or the other frame, as well as using both. This means that the search class has to be aware of each frame.
To be able to highlight the text, the Text class actually needs to be defined as StyledText. When the text is found, the replaceStyleRanges method is called to highlight it. For now, I am leaving it highlighted, thinking that it is more helpful to be able to see all of the found selections. The current found text is displayed in reverse video by using the setSelection method. This one has to be reset by setting the selection range to zero before setting the new selection.
I thought about being able to use a single find window to search multiple playback windows, but this made my head hurt. However, it did seem friendly to share the search strings between playback windows. So I created a string array in the global variables class, and store the strings there. I even save them to the preference store so that they are maintained over application restarts.
If you are interested in this code, check out the WindowPlayback*.java source modules in the subversion repository. The preference store is defined and initialized in the Globals.java and MainWindow.java modules.
The beauty of the OOP style is that almost all of my code is spent managing indexes. The heavy lifting is done by other classes and their methods. So I am hopeful that these GUI enhancements will be finished pretty soon, and I will build another set of download packages by the end of the month.
Later . . . Jim
Monday, February 23, 2009
Whatcha Doin'?
As soon as I put the latest download on SourceForge, I started working on the user interface. I am hoping to bring it up to a version 1.0 level of usability. At the rate I am going, I expect to have a download ready in about a month.
So that's what's coming. But what I barely mentioned, was a new feature in the last package that I added a week or so before it was built. The reason was, I wanted to see how well it worked in the pilot project before explaining it. But now I have done that, and I am pretty happy with it.
The feature is a comparison of the client and server session size. When a TCP session ends, the client data is multiplied by a factor, and if that is larger than the server data, it is reported. The multipliers are specified by port, so each protocol can be handled uniquely.
In the pilot, I have set port 80 to have a multiplier of 1, just as a proof of concept. There have not been too many reports on it, and those can be grouped in two categories:
Later . . . Jim
So that's what's coming. But what I barely mentioned, was a new feature in the last package that I added a week or so before it was built. The reason was, I wanted to see how well it worked in the pilot project before explaining it. But now I have done that, and I am pretty happy with it.
The feature is a comparison of the client and server session size. When a TCP session ends, the client data is multiplied by a factor, and if that is larger than the server data, it is reported. The multipliers are specified by port, so each protocol can be handled uniquely.
In the pilot, I have set port 80 to have a multiplier of 1, just as a proof of concept. There have not been too many reports on it, and those can be grouped in two categories:
- Document uploads: This was what I was hoping to see. And I believe that it means if there is an exploit that loads a rootkit, that would be reported, as well.
- Large cookies: I never realized how much is saved in some cookies. I have seen several (Facebook, I'm looking at you!) that are over 4Kbytes.
- Miscellaneous: The rest of the reports are usually client requests for many files, but some of them don't get sent for some reason.
Later . . . Jim
Subscribe to:
Posts (Atom)
Posts
