Ludo's notes

Host information

[root@smtp1 ~]# cat /etc/redhat-release
Fedora release 15 (Lovelock)
[root@smtp1 ~]#

How to add a new mysql user

[root@smtp1 ~]# mysql -u postfix -p
Enter password:

mysql> use postfix
mysql> INSERT INTO users VALUES ('new_user', ENCRYPT('password'));

MySql Setup

Install MySql

[root@smtp1 ~]# yum install mysql mysql-server

Turn on the service

[root@smtp1 ~]# chkconfig mysqld on

Start the service (or reboot your server)

[root@smtp1 ~]# /etc/init.d/mysqld start
Starting mysqld (via systemctl):                           [  OK  ]
[root@smtp1 ~]# 

Check the mysql server is working correctly

[root@smtp1 ~]# mysql
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.5.18 MySQL Community Server (GPL)

Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| test               |
+--------------------+
4 rows in set (0.00 sec)

mysql>

Assign the root password

[root@smtp1 ~]# mysqladmin -u root password password

Try this new password

[root@smtp1 ~]# mysql -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 6
Server version: 5.5.18 MySQL Community Server (GPL)

Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| test               |
+--------------------+
4 rows in set (0.00 sec)

mysql>

Create a postfix database in mysql (to store our virtual users)

[root@smtp1 ~]# mysqladmin create postfix -p
Enter password:
[root@smtp1 ~]#

Assign the postfix user to the postfix database

[root@smtp1 ~]# mysql -p

Enter password:
mysql> grant all on postfix.* to postfix@"localhost" identified by "postfix";

Create the table dedicated to store smtp users information

[root@smtp1 postfix]# mysql -u postfix -p
Enter password:
mysql> use postfix
mysql> CREATE TABLE users ( user varchar(255) NOT NULL, password varchar(255) NOT NULL, PRIMARY KEY (user) ) ;
Query OK, 0 rows affected (0.04 sec)

mysql> show tables;
+-------------------+
| Tables_in_postfix |
+-------------------+
| users             |
+-------------------+
1 row in set (0.00 sec)


mysql>

Postfix setup

Install Postfix

[root@smtp1 ~]# rpm -qa | grep postfix
postfix-2.8.2-2.fc15.x86_64
[root@smtp1 ~]#

Check mysql is built-in in this postfix version

[root@smtp1 postfix]# postconf -m | grep mysql
mysql
[root@smtp1 postfix]# 

Turn on postfix service

[root@smtp1 ~]# chkconfig --list | grep postfix

Note: This output shows SysV services only and does not include native
      systemd services. SysV configuration data might be overridden by native
      systemd configuration.

postfix             0:off     1:off     2:off     3:off     4:off     5:off     6:off

[root@smtp1 ~]#

[root@smtp1 ~]# chkconfig postfix on

[root@smtp1 ~]# /etc/init.d/postfix start
Starting postfix (via systemctl):                          [  OK  ]
[root@smtp1 ~]#

Check the daemon is running

[root@smtp1 ~]# telnet localhost 25
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 smtp1.rack.airband.net ESMTP Postfix
HELO idefix
250 smtp1.rack.airband.net
MAIL FROM: ludo@epita.fr250 2.1.0 Ok
RCPT TO: ludo@epita.fr
250 2.1.5 Ok

DATA
354 End data with <CR><LF>.<CR><LF>
test
.
250 2.0.0 Ok: queued as 78F0A3841E
^]
telnet> close
Connection closed.
[root@smtp1 ~]# 

Check the queue works correctly

[root@smtp1 log]# mailq
-Queue ID- --Size-- ----Arrival Time---- -Sender/Recipient-------
78F0A3841E      308 Mon Jan  9 15:26:34  ludo@epita.fr
(host mx-2.ig-iit.com[163.5.42.12] refused to talk to me: 450 See http://en.wikipedia.org/wiki/Greylisting - Try later)
                                         ludo@epita.fr

-- 0 Kbytes in 1 Request.
[root@smtp1 log]#

Send a test email

[root@smtp1 log]# ( echo subject: test; echo ) | sendmail -Am -v lfrancois@gmail.comMail Delivery Status Report will be mailed to <root>.
[root@smtp1 log]# mailq
Mail queue is empty

[root@smtp1 log]#

Create postfix configuration file defining how to pull user information from mysql

[root@smtp1 mysql]# pwd
/etc/postfix/mysql
[root@smtp1 mysql]# cat mysql-virtual_mailboxes.cf
user = postfix
password = postfix
dbname = postfix
table = users
select_field = CONCAT(SUBSTRING_INDEX(user,'@',-1),'/',SUBSTRING_INDEX(user,'@',1),'/')
where_field = user
hosts = 127.0.0.1
[root@smtp1 mysql]# 

Modify main.cf postfix configuration file

Save a copy of the original file

[root@smtp1 postfix]# cp -a main.cf main.cf.$(date -I)

Define how to access the user accounts

[root@smtp1 ~]# postconf -e 'virtual_mailbox_maps = proxy:mysql:/etc/postfix/mysql/mysql-virtual_mailboxes.cf'

Enable SASL Auth

[root@smtp1 ~]# postconf -e 'smtpd_sasl_auth_enable = yes'
[root@smtp1 ~]# postconf -e 'broken_sasl_auth_clients = yes'

Make sure only authenticated clients have access to the smtp server

[root@smtp1 ~]# postconf -e 'smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination'

Turn on TLS and define where are the keys

[root@smtp1 ~]# postconf -e 'smtpd_use_tls = yes'
[root@smtp1 ~]# postconf -e 'smtpd_tls_cert_file = /etc/postfix/smtpd.cert'
[root@smtp1 ~]# postconf -e 'smtpd_tls_key_file = /etc/postfix/smtpd.key'

Below the final configuration file once settings applied

[root@smtp1 ~]# diff -u /etc/postfix/main.cf.2012-01-09 /etc/postfix/main.cf
--- /etc/postfix/main.cf.2012-01-09     2011-03-23 18:51:43.000000000 +0000
+++ /etc/postfix/main.cf        2012-01-10 00:24:11.000000000 +0000
@@ -113,7 +113,7 @@
 #inet_interfaces = all
 #inet_interfaces = $myhostname
 #inet_interfaces = $myhostname, localhost
-inet_interfaces = localhost
+inet_interfaces = all
 
 # Enable IPv4, and IPv6 if supported
 inet_protocols = all
@@ -248,7 +248,7 @@
 #
 #mynetworks_style = class
 #mynetworks_style = subnet
-#mynetworks_style = host
+mynetworks_style = host
 
 # Alternatively, you can specify the mynetworks list by hand, in
 # which case Postfix ignores the mynetworks_style setting.
@@ -674,3 +674,10 @@
 # readme_directory: The location of the Postfix README files.
 #
 readme_directory = /usr/share/doc/postfix-2.8.2/README_FILES
+virtual_mailbox_maps = proxy:mysql:/etc/postfix/mysql/mysql-virtual_mailboxes.cf
+smtpd_sasl_auth_enable = yes
+broken_sasl_auth_clients = yes
+smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination
+smtpd_use_tls = yes
+smtpd_tls_cert_file = /etc/postfix/smtpd.cert
+smtpd_tls_key_file = /etc/postfix/smtpd.key
[root@smtp1 ~]# 

Modify master.cf file to enable 465 and 587 ports

[root@smtp1 postfix]# cp -a master.cf master.cf.$(date -I)
[root@smtp1 ~]# diff -u /etc/postfix/master.cf.2012-01-09 /etc/postfix/master.cf
--- /etc/postfix/master.cf.2012-01-09   2011-03-23 18:51:42.000000000 +0000
+++ /etc/postfix/master.cf      2012-01-09 19:14:08.000000000 +0000
@@ -13,16 +13,16 @@
 #smtpd     pass  -       -       n       -       -       smtpd
 #dnsblog   unix  -       -       n       -       0       dnsblog
 #tlsproxy  unix  -       -       n       -       0       tlsproxy
-#submission inet n       -       n       -       -       smtpd
-#  -o smtpd_tls_security_level=encrypt
-#  -o smtpd_sasl_auth_enable=yes
-#  -o smtpd_client_restrictions=permit_sasl_authenticated,reject
-#  -o milter_macro_daemon_name=ORIGINATING
-#smtps     inet  n       -       n       -       -       smtpd
-#  -o smtpd_tls_wrappermode=yes
-#  -o smtpd_sasl_auth_enable=yes
-#  -o smtpd_client_restrictions=permit_sasl_authenticated,reject
-#  -o milter_macro_daemon_name=ORIGINATING
+submission inet n       -       n       -       -       smtpd
+  -o smtpd_tls_security_level=encrypt
+  -o smtpd_sasl_auth_enable=yes
+  -o smtpd_client_restrictions=permit_sasl_authenticated,reject
+  -o milter_macro_daemon_name=ORIGINATING
+smtps     inet  n       -       n       -       -       smtpd
+  -o smtpd_tls_wrappermode=yes
+  -o smtpd_sasl_auth_enable=yes
+  -o smtpd_client_restrictions=permit_sasl_authenticated,reject
+  -o milter_macro_daemon_name=ORIGINATING
 #628       inet  n       -       n       -       -       qmqpd
 pickup    fifo  n       -       n       60      1       pickup
 cleanup   unix  n       -       n       -       0       cleanup
[root@smtp1 ~]# 

ssl key generation

Generate the key

[root@smtp1 postfix]# openssl req -new -outform PEM -out /etc/postfix/smtpd.cert -newkey rsa:2048 -nodes -keyout /etc/postfix/smtpd.key -keyform PEM -days 3650 -x509Generating a 2048 bit RSA private key
..............................+++
......................................................+++
writing new private key to '/etc/postfix/smtpd.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:US
State or Province Name (full name) []:
Locality Name (eg, city) [Default City]:Los Angeles
Organization Name (eg, company) [Default Company Ltd]:MyCompany
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server's hostname) []:
Email Address []:support@zoxx.net
[root@smtp1 postfix]#

Make sure the files are at the correct place and change the permissions

[root@smtp1 postfix]# ls -altr smtpd.*
-rw-r----- 1 root postfix 1704 Jan  9 19:11 smtpd.key
-rw-r----- 1 root postfix 1281 Jan  9 19:11 smtpd.cert
[root@smtp1 postfix]# 

saslauthd configuration

Install saslauthd with the sql module

[root@smtp1 postfix]# yum install cyrus-sasl cyrus-sasl-sql

Setup saslauthd

[root@smtp1 postfix]# cd /etc/sasl2
[root@smtp1 sasl2]# diff -u smtpd.conf.2012-01-10 smtpd.conf
--- smtpd.conf.2012-01-10       2011-03-23 18:51:44.000000000 +0000
+++ smtpd.conf  2012-01-10 00:33:18.000000000 +0000
@@ -1,2 +1,9 @@
 pwcheck_method: saslauthd
 mech_list: plain login
+allow_plaintext: true
+auxprop_plugin: mysql
+sql_hostnames: 127.0.0.1
+sql_user: postfix
+sql_passwd: postfix
+sql_database: mail
+sql_select: select password from users where user = '%u'
[root@smtp1 sasl2]# 

Turn on sasl

[root@smtp1 ~]# chkconfig saslauthd on
[root@smtp1 ~]# mkdir -p /var/run/saslauthd
[root@smtp1 ~]# /etc/init.d/saslauthd start
Starting saslauthd (via systemctl):                        [  OK  ]
[root@smtp1 ~]#

Configure PAM to get smtp information on MySQL

[root@smtp1 pam.d]# ls -altr | grep smtp
-rw-r--r--.  1 root root   76 Mar 23  2011 smtp.postfix
lrwxrwxrwx.  1 root root   25 Aug 25 13:15 smtp -> /etc/alternatives/mta-pam
[root@smtp1 pam.d]#

[root@smtp1 pam.d]# cat smtp
auth required pam_mysql.so user=postfix passwd=postfix host=127.0.0.1 db=postfix table=users usercolumn=user passwdcolumn=password crypt=1
account sufficient pam_mysql.so user=postfix passwd=postfix host=127.0.0.1 db=postfix table=users usercolumn=user passwdcolumn=password crypt=1
[root@smtp1 pam.d]#

Restart all services a last time

[root@smtp1 ~]# /etc/init.d/mysql restart
[root@smtp1 ~]# /etc/init.d/saslauthd restart
[root@smtp1 ~]# /etc/init.d/postfix restart

How to add a new mysql user

[root@smtp1 ~]# mysql -u postfix -p
Enter password:

mysql> use postfixmysql> INSERT INTO users VALUES ('new_user', ENCRYPT('password'));

Test with Swaks the server is working correctly

falabala:~# swaks --from ludo@zoxx.net --to lfrancois@gmail.com --server smtp1.zoxx.net -a -au ludo -ap password -tls --port 25
=== Trying smtp1.zoxx.net:25...
=== Connected to smtp1.zoxx.net.
<-  220 smtp1.zoxx.net ESMTP Postfix
 -> EHLO falbala.dedibox.fr
<-  250-smtp1.zoxx.net
<-  250-PIPELINING
<-  250-SIZE 10240000
<-  250-VRFY
<-  250-ETRN
<-  250-STARTTLS
<-  250-AUTH LOGIN PLAIN
<-  250-AUTH=LOGIN PLAIN
<-  250-ENHANCEDSTATUSCODES
<-  250-8BITMIME
<-  250 DSN
 -> STARTTLS
<-  220 2.0.0 Ready to start TLS
=== TLS started w/ cipher DHE-RSA-AES256-SHA
=== TLS peer subject DN="/C=US/L=Los Angeles/O=MyCompany/emailAddress=support@zoxx.net"
 ~> EHLO falbala.dedibox.fr
<~  250-smtp1.zoxx.net
<~  250-PIPELINING
<~  250-SIZE 10240000
<~  250-VRFY
<~  250-ETRN
<~  250-AUTH LOGIN PLAIN
<~  250-AUTH=LOGIN PLAIN
<~  250-ENHANCEDSTATUSCODES
<~  250-8BITMIME
<~  250 DSN
 ~> AUTH LOGIN
<~  334 VXNlcm5hbWU6
 ~> bHVkbw==
<~  334 UGFzc3dvcmQ6
 ~> cGFzc3dvcmQ=
<~  235 2.7.0 Authentication successful
 ~> MAIL FROM:<ludo@zoxx.net>
<~  250 2.1.0 Ok
 ~> RCPT TO:<lfrancois@gmail.com>
<~  250 2.1.5 Ok
 ~> DATA
<~  354 End data with <CR><LF>.<CR><LF>
 ~> Date: Tue, 10 Jan 2012 03:14:12 +0100
 ~> To: lfrancois@gmail.com
 ~> From: ludo@zoxx.net
 ~> Subject: test Tue, 10 Jan 2012 03:14:12 +0100
 ~> X-Mailer: swaks v20100211.0 jetmore.org/john/code/swaks/
 ~>
 ~> This is a test mailing
 ~>
 ~> .
<~  250 2.0.0 Ok: queued as B7ACF3841C
 ~> QUIT
<~  221 2.0.0 Bye
=== Connection closed with remote host.
falabala:~# 

Check the server is not in open relay and denied

ludo@idefix:~ % telnet smtp1.zoxx.net 25             
Trying smtp1.zoxx.net...
Connected to smtp1.zoxx.net.
Escape character is '^]'.
220 smtp1.zoxx.net ESMTP Postfix
HELO idefix
250 smtp1.zoxx.net
MAIL FROM: ludo@epita.fr
250 2.1.0 Ok
RCPT TO: lfrancois@gmail.com
554 5.7.1 <lfrancois@gmail.com>: Relay access denied
^]
telnet> close
Connection closed.
ludo@idefix:~ %

Debian version

ludo@blackbox:~ % cat /etc/debian_version          
wheezy/sid
ludo@blackbox:~ % uname -a                               
Linux blackbox 3.0.0-12-generic #20-Ubuntu SMP Fri Oct 7 14:50:42 UTC 2011 i686 athlon i386 GNU/Linux
ludo@blackbox:~ %     

btrfs package install

root@blackbox:~# apt-get install btrfs-tools

Reinitialize all LUNs in use for this new file system

root@blackbox:~# dd if=/dev/zero of=/dev/sdb bs=1M count=512
512+0 records in
512+0 records out
536870912 bytes (537 MB) copied, 9.07636 s, 59.2 MB/s
root@blackbox:~# dd if=/dev/zero of=/dev/sdc bs=1M count=512
512+0 records in
512+0 records out
536870912 bytes (537 MB) copied, 9.06802 s, 59.2 MB/s
root@blackbox:~# 

Check if any btrfs file system are already present on the system

root@blackbox:~# btrfs-show
Btrfs Btrfs v0.19
root@blackbox:~# 

Create btrfs file system (metadata and data replicated)

root@blackbox:~# mkfs.btrfs -m raid1 -d raid1 -L btrfs01 /dev/sdb /dev/sdc

WARNING! - Btrfs Btrfs v0.19 IS EXPERIMENTAL
WARNING! - see http://btrfs.wiki.kernel.org before using

adding device /dev/sdc id 2
fs created label btrfs01 on /dev/sdb
     nodesize 4096 leafsize 4096 sectorsize 4096 size 298.02GB
Btrfs Btrfs v0.19
root@blackbox:~#

mount btrfs01

root@blackbox:~# mkdir /mnt/btrfs01
root@blackbox:~# mount /dev/sdb /mnt/btrfs01/

check the file system is mounted (it shows up twice the capacity really available)

root@blackbox:~# df -h -F btrfs
Filesystem            Size  Used Avail Use% Mounted on
/dev/sdb              299G   56K  296G   1% /mnt/btrfs01
root@blackbox:~#

make a test to understand how the data are stripped across the LUNs

root@blackbox:~# dd if=/dev/zero of=/mnt/btrfs01/file.10G bs=1M count=$((10*1024))
10240+0 records in
10240+0 records out
10737418240 bytes (11 GB) copied, 176.188 s, 60.9 MB/s
root@blackbox:~#

root@blackbox:~# df -h -F btrfs
Filesystem            Size  Used Avail Use% Mounted on
/dev/sdb              299G   21G  276G   7% /mnt/btrfs01
root@blackbox:~#

root@blackbox:~# btrfs-show
Label: btrfs01  uuid: d2b77569-a8be-4ec7-862a-d0fc1e6c8511
     Total devices 2 FS bytes used 10.01GB
     devid    1 size 149.01GB used 12.03GB path /dev/sdb
     devid    2 size 149.01GB used 12.01GB path /dev/sdc

Btrfs Btrfs v0.19
root@blackbox:~#

root@blackbox:~# btrfs filesystem df /mnt/btrfs01/
Data, RAID1: total=11.00GB, used=10.00GB
Data: total=8.00MB, used=0.00
System, RAID1: total=8.00MB, used=4.00KB
System: total=4.00MB, used=0.00
Metadata, RAID1: total=1.00GB, used=14.11MB
Metadata: total=8.00MB, used=0.00
root@blackbox:~#

I am playing in CherryPy these days, the most helpful documentation is directly from the python command line.

[ludo@falbala ~]$ python2.6
Python 2.6.3 (r263:75183, Oct  9 2009, 13:58:37) 
[GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import cherrypy
>>> help(cherrypy)
>>> help(cherrypy.config)

Configure a simple relay-only mail transport agent with sendmail

df -k

This command allows you to know where we are in the file system and to find back from where do we mount this file system, specially if you use a lot of automount crossed or nfs shares.

obelix:/var/www# df -k .
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mapper/vg01-var   3096336   2713092    225960  93% /var
obelix:/var/www# 

showmount -e

When you want to troubleshoot any nfs issue, you have to know the command showmount, if this command fails you can be sure you will fail to mount your network file system.

obelix:~ # showmount -e localhost
Export list for localhost:
/goinfre 192.168.1.44
obelix:~ #          
idefix:~ # showmount -e obelix
mount clntudp_create: RPC: Program not registered
idefix:~ #

In this specific case, it means the nfs client idefix doesn't have rpc.statd started, to fix it on debian you need just to run it:

idefix:~ # /etc/init.d/nfs-common start
Starting NFS common utilities: statd
idefix:~ # showmount -e obelix

Installation of flashplayer9 plugin with Firefox on Debian.

Create a debian package with dh_make and dpkg-buildpackage

• generate a debian template directory
• editing configuration files
• create a debian apt repository

IPv6 dns configuration with bind

Configuration of an IPv6 tunnel with Hexago's migration broker on Debian GNU/Linux

• Register
• Kernel configuration
• Hexago tspc tool