Weniger Wissenswertes für die Welt

After finishing my Homeserver project I wanted to get the maximum of performance. At the end of my research in the web i have created following script:

# Get Arrays and physical Disks for active Raid5 Volumes

ARRAYS=""
DISKS=""

for field in `cat /proc/mdstat | grep "active raid5"`
do
  if [[ "x${field:0:2}x" == "xmdx" ]]
  then
    ARRAYS="$ARRAYS $field"
  elif [[ "x${field:0:2}x" == "xsdx" ]]
  then
    DISKS="$DISKS ${field:0:3}"
  fi
done

# Changes on physical Devices
for disk in $DISKS
do
  #Set read-ahead to 16MB
  blockdev --setra 16384 /dev/$disk
  #Device Queue Tuning
  echo 1024 > /sys/block/$disk/queue/read_ahead_kb
  echo 256 > /sys/block/$disk/queue/nr_requests
  #Disable NCQ
  echo 1 > /sys/block/$disk/device/queue_depth
done

# Changes on Raid Arrays
for array in $ARRAYS
do
  #Set read-ahead to 64MB
  blockdev --setra 64536 /dev/$array
  #Increase stripe-cache size
  echo 16384 > /sys/block/$array/md/stripe_cache_size
done

This are the optimal and tested settings for my system. The numbers can differ from system to system. You have to test.

Network Layout:

There is one server which is only connectable via SSH. This server hosts two additional services: git and http. We have no access to the firewall and can’t forward the ports. The server is a linux system, as clients windows and linux are a option.

We can use the comon git port on our client, but we can’t use port 80 on the client because there is already a webserver running.

Server configuration:

Changes in /etc/ssh/sshd_config:

....
Port 22
....
AllowTCPForwarding yes
...

Linux Client:

Only a few parameters are needed to tunnel our git and web service:

me@client$ ssh -L 10080:localhost:80 -L 9418:localhost:9418 user@server
user@server$ 

As soon as the SSH session starts, you can connect to localhost:10000 for access to the web server. The git service listens on the same port as on the server, so you only have to change the server configuration to localhost in your git client.

Windows Client:

I use putty a very common ssh client:

Open putty and enter username and destination server:

In Connection->SSH->Tunnels configure the tunnels:

Just click open and start the session.

Now you should connect to the tunneled services as shown in the linux config.

I found it annoying for a long time, that i have only single threaded compress programs under linux when using compress from stdin.
Today i noticed a command line switch for rar which enables this for rar.

   -si[name]
     Read data from stdin (standard input), when creating
     an archive. Optional 'name' parameter allows to specify
     a file name of compressed stdin data in the created
     archive. If this parameter is missing, the name will be
     set to 'stdin'. This switch cannot be used with -v.

Now i can replace my MySQL Database Backup like this

[old]# mysqldump bigdb | bzip2 -c > bigdb.sql
[new]# mysqldump bigdb | rar a -sibigdb.sql bigdb.sql.rar

To uncompress to stdout the commands change like the following:
Das Entpacken nach stdout funktioniert dann wie folgt:

[old]# bzcat bigdb.sql.bz2 | mysql bigdb
[new]# rar -inul p bigsb.sql.rar bigdb.sql | mysql bigdb

A sub project of Homeserver 2.0 was the virtualization of my home router.
I wanted to use pass-through of the additional nic for the external interface, so my host system can’t see any external traffic.

The steps should be easy, started virt-manager and followed these simple steps:

After a reboot of the vm i got a error: Connection reset by peer. The qemu process for the wasn’t started.

I got a more detailed error message with the virsh console:

server # virsh
virsh # start router-linz --console
error: Failed to start domain router-linz
error: internal error process exited while connecting to monitor: qemu-kvm: 
-device pci-assign,host=03:00.0,id=hostdev0,configfd=24,bus=pci.0,addr=0x5: 
No IOMMU found.  Unable to assign device "hostdev0"
qemu-kvm: -device pci-assign,host=03:00.0,id=hostdev0,configfd=24,bus=pci.0
,addr=0x5: Device 'kvm-pci-assign' could not be initialized
2013-10-01 17:44:52.566+0000: shutting down

Here it is: No IOMMU found. But i bought hardware that supports VT-d, a i5-4570 and a Z87 board. I rebooted my host system and checked the bios. VT-d was activated already.

Ok, I started googling and found a article about xen and HowTo activate VT-d. This article mentions special boot parameters you have to add to use this feature.

I’ve tested iommu=1 but haven’t got luck. Then i tried different combinations of the other kernel parameters in the article but none works.

Shortly before my surrender i found a mailing list post. In this post was explained to use intel_iommu=1.

Grub menu.lst edited, a short reboot of the host and then it worked.

Summary:

To activate VT-d under linux you have to add the kernel boot parameter intel_iommu=1. The VT-d/IOMMU support in the kernel alone is only half the battle.