When adding new virtual hosts in your nginx configuration file, you can experience this error message:

# nginx -t
2008/11/13 09:37:03 [emerg] 12299#0: could not build the server_names_hash, you should increase server_names_hash_bucket_size: 32
2008/11/13 09:37:03 [emerg] 12299#0: the configuration file /etc/nginx/nginx.conf test failed

server_names_hash_bucket_size controls the maximum length of a virtual host entry (ie the length of the domain name).

In other words, if your domain names are long, increase this parameter.

You need to add this flag in the http context:

http {
    server_names_hash_bucket_size 64;
    ...
}

After increasing the value, test your configuration file and reload nginx:

# nginx -t
2008/11/13 09:48:06 [info] 12315#0: the configuration file /etc/nginx/nginx.conf syntax is ok
2008/11/13 09:48:06 [info] 12315#0: the configuration file /etc/nginx/nginx.conf was tested successfully
# kill -HUP `cat /var/run/nginx.pid`

Tags: nginx

Sun JRE is available in the non-free repository. You need to update your repositories configuration file (/etc/apt/sources.list).

Add the following line:

deb http://ftp.us.debian.org/debian/ etch main contrib non-free

Update apt with apt-get update command and you’re ready to install the JRE.

$ apt-get install sun-java5-jre

Check your java binary:

$ java -version
java version "1.5.0_14"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_14-b03)
Java HotSpot(TM) Client VM (build 1.5.0_14-b03, mixed mode, sharing)

Tags: debian, etch, Java, jre

While reading this interresting article on benchmarking disks, Benjamin Schweizer pointed out a link to a python tool he wrote: benchmark disks IOs.

I made a try on different available servers.

Server 1

• PowerEdge 2950 - Intel(R) Xeon(R) CPU 5160  @ 3.00GHz
• Dell PowerEdge Expandable RAID controller 5 (LSI Logic SAS based MegaRAID driver, PERC 5/i Integrated)
• 6 disks RAID 10
• uname -a Linux … 2.6.22.1 #1 SMP Wed Aug 8 11:02:15 CEST 2007 x86_64 GNU/Linux

# python iotest-2008-10-15 /dev/sda4 10
/dev/sda4, 397 GB, 512B blocks: 169.8 IOs/s, 84 kB/s
/dev/sda4, 397 GB, 1024B blocks: 160.7 IOs/s, 160 kB/s
/dev/sda4, 397 GB, 2 kB blocks: 153.2 IOs/s, 306 kB/s
/dev/sda4, 397 GB, 4 kB blocks: 140.2 IOs/s, 560 kB/s
/dev/sda4, 397 GB, 8 kB blocks: 133.0 IOs/s, 1 MB/s
/dev/sda4, 397 GB, 16 kB blocks: 113.2 IOs/s, 1 MB/s
/dev/sda4, 397 GB, 32 kB blocks: 85.7 IOs/s, 2 MB/s
/dev/sda4, 397 GB, 64 kB blocks: 80.0 IOs/s, 4 MB/s
/dev/sda4, 397 GB, 128 kB blocks: 74.2 IOs/s, 9 MB/s
/dev/sda4, 397 GB, 256 kB blocks: 70.9 IOs/s, 17 MB/s
/dev/sda4, 397 GB, 512 kB blocks: 62.3 IOs/s, 31 MB/s
/dev/sda4, 397 GB, 1024 kB blocks: 53.8 IOs/s, 53 MB/s

Server 2

• PowerEdge 2950 - Intel(R) Xeon(R) CPU 5130  @ 2.00GHz
• LSI Logic SAS based MegaRAID driver
• 2 disks RAID 0
• uname -a: Linux … 2.6.18-5-686 #1 SMP Fri Jun 1 00:47:00 UTC 2007 i686 GNU/Linux

# python iotest-2008-10-15 /dev/sda4 10
/dev/sda4, 220 GB, 512B blocks: 78.7 IOs/s, 39 kB/s
/dev/sda4, 220 GB, 1024B blocks: 76.9 IOs/s, 76 kB/s
/dev/sda4, 220 GB, 2 kB blocks: 77.5 IOs/s, 154 kB/s
/dev/sda4, 220 GB, 4 kB blocks: 77.9 IOs/s, 311 kB/s
/dev/sda4, 220 GB, 8 kB blocks: 76.6 IOs/s, 613 kB/s
/dev/sda4, 220 GB, 16 kB blocks: 75.5 IOs/s, 1 MB/s
/dev/sda4, 220 GB, 32 kB blocks: 73.9 IOs/s, 2 MB/s
/dev/sda4, 220 GB, 64 kB blocks: 68.5 IOs/s, 4 MB/s
/dev/sda4, 220 GB, 128 kB blocks: 58.9 IOs/s, 7 MB/s
/dev/sda4, 220 GB, 256 kB blocks: 44.7 IOs/s, 11 MB/s
/dev/sda4, 220 GB, 512 kB blocks: 33.3 IOs/s, 16 MB/s
/dev/sda4, 220 GB, 1024 kB blocks: 23.4 IOs/s, 23 MB/s

Server 3

• PowerEdge 1950
• Intel(R) Xeon(R) CPU E5410  @ 2.33GHz
• LSI Logic / Symbios Logic - SAS1078 PCI-X Fusion-MPT SAS
• 2 disks RAID 1
• uname -a: Linux … 2.6.18-6-686 #1 SMP Tue Jun 17 21:31:27 UTC 2008 i686 GNU/Linux

# python iotest-2008-10-15 /dev/sda4 10
/dev/sda4, 220 GB, 512B blocks: 88.6 IOs/s, 44 kB/s
/dev/sda4, 220 GB, 1024B blocks: 89.6 IOs/s, 89 kB/s
/dev/sda4, 220 GB, 2 kB blocks: 87.7 IOs/s, 175 kB/s
/dev/sda4, 220 GB, 4 kB blocks: 86.4 IOs/s, 345 kB/s
/dev/sda4, 220 GB, 8 kB blocks: 86.4 IOs/s, 690 kB/s
/dev/sda4, 220 GB, 16 kB blocks: 83.6 IOs/s, 1 MB/s
/dev/sda4, 220 GB, 32 kB blocks: 79.9 IOs/s, 2 MB/s
/dev/sda4, 220 GB, 64 kB blocks: 71.9 IOs/s, 4 MB/s
/dev/sda4, 220 GB, 128 kB blocks: 59.2 IOs/s, 7 MB/s
/dev/sda4, 220 GB, 256 kB blocks: 54.2 IOs/s, 13 MB/s
/dev/sda4, 220 GB, 512 kB blocks: 34.6 IOs/s, 17 MB/s
/dev/sda4, 220 GB, 1024 kB blocks: 22.1 IOs/s, 22 MB/s

Links

• LSI Logic MegaRAID – Emergency Cheat Sheet

Tags: Benchmarks, disks, iostat

As you may know, when using memcached on 32-bit servers, processes can only address 4GB of virtual memory making the memcached daemon handle only 2 or 3 Gb of memory.

If you want to address more memory, you need to launch multiple memcached daemons.

I wrote an rc script , mmemcached, for FreeBSD to manage this.

To use it, you need to add this in /etc/rc.conf:

mmemcached_enable="YES"
mmemcached_servers="1 2"
mmemcached_user="nobody"
mmemcached_args="-c 8192 -t 2 -m 2048 -d"

Copy the mmemcached script in /usr/local/etc/rc.d for example and launch it with the start argument. With the above configuration it will launch two memcached daemons listening on ports 11211 and 11212 (mmemcached_servers parameter) with 2Gb of memory.

Configuration

Details of the rc.conf configuration:

• mmemcached_enabled: let this script “startable”,
• mmemcached_servers: List of the daemons you want to start. Each number is used to generate the port number of the daemon. For example, 2 will launch a daemon listening on port 11212,
• mmemcached_user: user that will start the daemon,
• mmemcached_args: flags to pass to the daemon.

Download

Download the script on github.

Tags: FreeBSD, memcached

I was surprised when I saw the length of the Chrome user agent string last week:

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.X.Y.Z Safari/525.13

And in our logs:

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.2.149.29 Safari/525.13

a user agent string of 119 characters. It looks quite a waste of space but is Google Chrome the only one? Surprisingly, Chrome is far from the worst.

Best of one of our log file:

• 641 characters: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4 GoogleToolbarFF 3.0.20070420 GoogleToolbarFF 3.0.20070420 GoogleToolbarFF 3.0.20070525 GoogleToolbarFF 3.0.20070525 GoogleToolbarFF 3.0.20070525 GoogleToolbarFF 3.0.20070525 GoogleToolbarFF 3.0.20070525 GoogleToolbarFF 3.0.20070525 GoogleToolbarFF 3.0.20070525 GoogleToolbarFF 3.0.20070525 GoogleToolbarFF 3.0.20070525 GoogleToolbarFF 3.0.20070525 GoogleToolbarFF 3.0.20070525 GoogleToolbarFF 3.0.20070525 GoogleToolbarFF 3.0.20070525 GoogleToolbarFF 3.0.20070525 GoogleToolbarFF 3.0.20070525 GoogleToolbarFF 3.0.20070525 GoogleToolbarFF 3.0.20070525
• 337 characters: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; DA4BB049-ADVLOVER|0001|DSL; C:\DOCUME~1\everey\CONFIG~1\Temp\; C:\DOCUME~1\zulcan\CONFIG~1\Temp\; C:\DOCUME~1\nilfer\CONFIG~1\Temp\; C:\DOCUME~1\mirmor\CONFIG~1\Temp\; C:\DOCUME~1\ASTNU~1\CONFIG~1\Temp\; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) ; .NET CLR 2.0.50727)
• 290 characters: Mozilla/5.0 (Windows; U LupinV2.u2/20080827 LupinV2.u2/20080828 LupinV2.u2/20080829 LupinV2.u2/20080830 LupinV2.u2/20080831 LupinV2.u2/20080902 LupinV2.u2/20080903 LupinV2.u2/20080909 LupinV2.u2/20080911 LupinV2.u2/20080912; Windows NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1
• 272 characters: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; FunWebProducts; SU 3.011; User-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; http://bsalsa.com) ; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 1.1.4322; .NET CLR 3.5.30428; .NET CLR 3.0.30422)
• 202 characters: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.0.3705; .NET CLR 1.1.4322; Media Center PC 4.0; IE7-01NET.COM-1.1; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; InfoPath.2; IE7-01NET.COM-1.1)

The full list

• Download the user agent log (gzip format)

How to extract user agent strings from a HTTP log file?

• Print user agent strings with its length:

awk -F\" '{print length($6)" "$6}'  access.log

• print user agent strings that are more than 200 characters length:

awk -F\" '{if ($6 > 200) print length($6)" "$6}'  access.log

In those examples, the access.log file has this log format:

xxx.xxx.xxx.xxx \
www.domain.com - \
[15/Sep/2008:00:00:00 +0200] \
"GET / HTTP/1.1" 200 4242 \
"http://www.domain.com/" \
"Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1"

About bandwith

If you take an average user agent string likes the Firefox one, you have a 91 charaters string.

• How many entries with a user agent string longer than 120 characters: 249586

awk -F\" '{if (length($6) > 120) print length($6)}' access.log | wc -l

• Size waste with string longer than 120 characters: 5.67 M

awk -F\" '{if (length($6) > 120) SUM += length($6)-120} END {print SUM/1024/1024" Mo"}'  access.log

• Bandwidth waste per month for this server: 170M…

Tags: awk, bandwidth, user agent

Logcheck is a tool to parse system logs and send summaries by email. It filters out logs with a regular expressions database to suppress common/normal entries.

Are you reading / checking your log files? Too many servers? logcheck will help you in this task and eliminates the noise.

Installing on Debian

# apt-get install logcheck
Reading package lists... Done
Building dependency tree... Done
The following extra packages will be installed:
lockfile-progs logtail
Suggested packages:
syslog-summary
Recommended packages:
logcheck-database
The following NEW packages will be installed:
lockfile-progs logcheck logtail
0 upgraded, 3 newly installed, 0 to remove and 6 not upgraded.
Need to get 110kB of archives.
After unpacking 428kB of additional disk space will be used.
Do you want to continue [Y/n]?

Also install logcheck-database which contains lots of rules

# apt-get install logcheck-database

Config files

• /etc/logcheck/logcheck.conf
  • SENDMAILTO=”root” - your email address
• /etc/logcheck/logcheck.logfiles
  • configure which logfiles to analyze
• /etc/cron.d/logcheck
  • logcheck cron (by default, logcheck runs every hour)

You can try it by executing the following command:

# su -s /bin/bash -c "/usr/sbin/logcheck" logcheck

Your mailbox should now contains a report from logcheck if some unusual log entries have been found.

Links

• logcheck wiki

Tags: logcheck

This post is a quick ref on the linux touch command. All the examples have been tested on Linux.

This command is used to update the access and modification times of files.

• touch’s syntax

touch [option] file_name(s)
touch file1 file2 file3

• Here some examples:

# touch /tmp/file
# ls -l /tmp/file
rw-r--r-- 1 charlybr charlybr 0 Sep 10 16:13 /tmp/file

• Update access and modification time to current time:

# ls -l /tmp/file
rw-r--r-- 1 charlybr charlybr 0 Sep 10 16:13 /tmp/file
# touch /tmp/file
# ls -l /tmp/file
rw-r--r-- 1 charlybr charlybr 0 Sep 10 16:14 /tmp/file

• Update access and modification time to a specified timestamp ([[CC]YY]MMDDhhmm[.ss] format):

# touch -t 09091842 /tmp/file
# ls -l /tmp/file
-rw-r--r-- 1 charlybr charlybr 0 Sep  9 18:42 /tmp/file

• Update access and modification time to a specified date

# touch -d '9 Sep' /tmp/file
# ls -l /tmp/file
-rw-r--r-- 1 charlybr charlybr 0 Sep  9 00:00 /tmp/file
# touch -d '9 Sep 2008 13:14' /tmp/file
# ls -l /tmp/file
-rw-r--r-- 1 charlybr charlybr 0 Sep  9 13:14 /tmp/file

Links

• touch is part of the Coreutils package
• touch documentation page
• man page on FreeBSD

Tags: coretutils, shell

On MacOSX, if you have installed macports with the package installer, you may encounter the sudo: port: command not found problem.

Macports binaries are installed in /opt/local/bin, so you just need to add this path to your PATH environment variable. Example with your user, add to your $HOME/.profile :

export PATH=$PATH:/opt/local/bin

You can source your profile file to update your environment:

$ source .profile

You are now able to use the port command:

$ sudo port -v selfupdate

Tags: MacOSX, port, shell

The dmidecode command gives you all informations available about your memory.

With the special parameter “-t 16″, you can see the maximum (physical) memory that your server can have:

$ dmidecode -t 16
# dmidecode 2.8
SMBIOS 2.4 present.

Handle 0x1000, DMI type 16, 15 bytes
Physical Memory Array
Location: System Board Or Motherboard
Use: System Memory
Error Correction Type: Multi-bit ECC
Maximum Capacity: 32 GB
Error Information Handle: Not Provided
Number Of Devices: 8

Here we can see that your server can handle up to 32Gb

Check empty memory slots

To know which slots are used or not use the “-t 17″ flag.

dmidecode -t 17 | grep Size
        Size: 2048 MB
        Size: 2048 MB
        Size: No Module Installed
        Size: No Module Installed
        Size: No Module Installed
        Size: No Module Installed
        Size: No Module Installed
        Size: No Module Installed

Links

• DMI specification
• SMBIOS specification

Tags: dmidecode, memory

apticron is a shell script that send you an email report when new packages are available on your debian server.

Install apticron

As root, do:

apt-get install apticron

apticron configuration via /etc/apticron/apticron.conf

To receive reports on your email change the EMAIL variable in the configuration file. By default, reports are sent to the root user.

EMAIL="root" -> EMAIL="you@domain.com"

Reports received

Now apticron will send you reports like this:

apticron report [Sat, 26 Jul 2008 06:45:47 +0200]
==============================================
apticron has detected that some packages need upgrading on:

        localhost.localdomain

        [ 127.0.0.1 XXX.XXX.XXX.XXX ]

The following packages are currently pending an upgrade:

        lighttpd 1.4.13-4etch10

========================================================================

Package Details:

Reading changelogs...

--- Changes for lighttpd ---

lighttpd (1.4.13-4etch10) stable-security; urgency=low

  [ Pierre Habouzit ]

  * Non-maintainer upload.

  * Fix [CVE-2008-1531] patches mess, and add a missing hunk of the patch.

 -- Thijs Kinkhorst thijs debian org  Tue, 22 Jul 2008 12:19:10 +0200

========================================================================

You can perform the upgrade by issuing the command:

        aptitude dist-upgrade

as root on localhost.localdomain

It is recommended that you simulate the upgrade first to confirm that the actions that
would be taken are reasonable. The upgrade may be simulated by issuing the command:

        aptitude -s -y dist-upgrade

--
apticron

Apticron files

For more informations, you can have a look to those files:

/etc/cron.daily/apticron
/usr/sbin/apticron

Tags: apticron, debian, linux