<body><iframe src="http://www.blogger.com/navbar.g?targetBlogID=14550857&amp;blogName=David+Francis+personal+web+log&amp;publishMode=PUBLISH_MODE_FTP&amp;navbarType=BLUE&amp;layoutType=CLASSIC&amp;homepageUrl=http%3A%2F%2Fblog.davidfrancis.org%2Fdefault.asp&amp;searchRoot=http%3A%2F%2Fblogsearch.google.com%2F" marginwidth="0" marginheight="0" scrolling="no" frameborder="0" height="30px" width="100%" id="navbar-iframe" title="Blogger Navigation and Search"></iframe> <div id="space-for-ie"></div>

Friday, August 18, 2006

IP Range Compare I finally figured it out!

This key component was designed for client website statistics but also for my own domain intelligence application and "anti-stalking" needs. It will simplify the banning of known abusers as well as undesirables who are simply on my sites to disrupt, harass, or worse. I've been trying to come up with a method of storing and comparing IP ranges in a database for the visitor tracker application I've been working on. Last night, I finished testing on the new comparing scripts and it's all good!

See, internet service providers are assigned a range of IP numbers, some of which are broken down into regions. All are broken down into countries. A range can include tens of thousands IP numbers and I didn't want to have to make a whois request if I already have the range. Simply put, I've broken the lower and upper range into their decimal equivalent and do the compare from there. This was a key component because of the number of compares that were required for 12 sets of numeric variables. Now there are only 3; the originating IP address; the lower range; the upper range.

This new method can also be used as another layer of security on sites or sections that I would prefer to control access. Say I have a "sandbox" website that I use for development and testing. I really don't want the search engines or prying eyes to view the site, but I need the site on the production server for testing. I know what my own IP range is so I now can limit access to only my range. I know, it's not perfect and I will have to use other methods for secret information, but this can be one more level of authentication.

Online forums and discussion board are also plagued with undesirable visitors like e-mail harvesting robots (or humans), abusers of the system, people who have been asked to leave and the like. In the worst case scenario, some members are banned but they continue to return with new email address and logins. This new script will allow me to ban a range of IP's either permanently or temporarily so as to frustrate the efforts of the condemned.

The key application that this new script will be most useful for is my own domain intelligence application. I have been wanting to create a private database of whois information for statistical analysis and information gathering. This one little script will help speed up database queries as well as offer cross reference for deeper analysis.

I am automating the whois lookup next. I have a couple of places I can retrieve the data but I am looking for one that will not be "insulted" by my queries. I don't intent to abuse the system and I will be throttling the queries to prevent overuse of other's resources. My interest in the data is for statistics and monitoring of websites under my responsibility, but as I dig deeper to glean more information, I may have to make a number of requests to the whois server. I must find a way to do this that is within the terms of service for the data provider. There is one that I could use for a fee, but it is very expensive for the number of queries that could result. So my search continues for that perfect data source.

The application is the "visitor tracker" but the deeper data is yet to be branded, but something cool to reflect the depth and breadth of the data. It will expand as needs and features arise. Currently it is simply two tables in a SQL database. 1 for the IP range and id and another for cross reference of the IP and registrant information. From there I can create a contact table of registrants, technical contacts, administrative and billing as well. Domain Name Servers, and more for deeper mining of data and cross reference.

There used to be a great resource for this information online but for some reason they killed the project. I wrote to domainsdb.net but they never responded. It would appear they were a company in Russia. The entire site was in ASP and I was willing to continue the project, help with development and/or hosting or buy it outright. Since I didn't hear back, I guess I'll have to develop it for myself.

I've been testing the other aspect of the visitor tracker and it appears to be working as well. A simple readout of active users but with a little "more" information. This new piece will be added, perhaps as an AJAX popup. This will help speed up the review of raw data. Also it will help filter out search engine and personal hits so as to find exactly what I'm looking for.

It's very exciting and will be a great asset to the SoKy.net project as well as one more value added service to my web clients.