rspamd with mailman

When setting up spam filtering for mailman, it is important to make sure you don't accidentically filter too much. Things like bounces need to go thru, and you probably don't want to filter out mails for your users.

We have set up rspamd in front of dovecot.org mailman. As you are not permitted to send mail unless you are subscribed, we had quite a lot of administrative workload, and setting up rspamd reduced fair share of that. But, since it's a mailing list, we needed to make some adjustments.

First of all, we do not want to accidentically classify our subscribers as spam. Luckily this was almost trivial step.

  1. Setup cron job that will export members from mailing list to file.
  2. Configure symbol for senders that are in list using multimap module
  3. Configure negative score for this symbol
#!/bin/bash

/usr/lib/mailman/bin/list_members dovecot > /etc/rspamd/local.d/maillist_members.txt
restorecon /etc/rspamd/local.d/maillist_members.txt
# local.d/multimap.conf
MAILLIST_MEMBER {
  type = "header";
  header = "From";
  filter = "email:addr";
  map = "file:///etc/rspamd/local.d/maillist_members.txt";
}
# rspamd.conf.local
metric "default" {  
  symbol "MAILLIST_MEMBER" {
     score = -5.0;
     description = "Member of a mailing list";
  }
}

The next hurdle is to permit bounces to work. Initial attempt was just with multimap.conf and do prefilter that accepts anything sent to bounces.

# local.d/multimap.conf
WHITELIST_RCPT {
  type = "rcpt";
  filter = "email:user";
  map = "file:///etc/rspamd/local.d/bounces.txt"; 
  prefilter = true;
  action = "accept";
} 
Turns out this was not such a good idea, because if someone sends spam to mailing@list mailing-bounces@list, it will end up matching here and bypasses the spam filter. A more refined solution was to configure a conditional whitelisting, that permits bounces if there is only one recipient.
# local.d/multimap.conf
WHITELIST_RCPT {
  type = "rcpt";
  filter = "email:user";
  map = "file:///etc/rspamd/local.d/bounces.txt"; 
} 
# local.d/force_actions.conf
rules {
	WHITELIST_BOUNCES {
		action = "accept";
		expression = "WHITELIST_RCPT && RCPT_COUNT_ONE && !FORGED_RECIPIENTS";
	}
}
This, while not as efficient as the first one, does the job more carefully, and prevents spam from getting thru. It would be probably useful to have multimap option that specifies that it only matches for single recipient, not if any of them matches.

Appsuite & Dovecot installation

I thought I'd write up a small guide for setting up Open-Xchange App Suite and Dovecot using free versions.

There are lots of these already, but I thought I'd freshen this up, to latest avavailable versions. But first, Full Disclosure, I work for Open-Xchange (at time of writing). This guide is for Debian Stretch, but with some small changes, it should be appliable to some other OSes.

I also assume you are capable of configuring your MTA yourself, and that you are using virtual user setup.

Feedback is welcome to aki.tuomi at dovecot.fi

Setting things up

Before installation, one needs to set up repositories, since we want to use packaged versions.

First we import PGP geys to get things started.

curl https://software.open-xchange.com/oxbuildkey-community.pub | gpg --import
curl https://software.open-xchange.com/oxbuildkey.pub | gpg --import
curl https://repo.dovecot.org/DOVECOT-REPO-GPG | gpg --import

Then, export the imported keys into trusted key file.

gpg --export CD6B53ED ED409DA1 DD1A5E9CEED949F0 > /etc/apt/trusted.gpg.d/open-xchange.gpg

Then we setup the repository list.

cat<<EOF > /etc/apt/sources.list.d/open-xchange.list
deb http://repo.dovecot.org/ce-2.3-latest/debian/stretch stretch main
# this is not a typo, but please use Stretch repos when they become available
# 2018-11-18 - Updated to use Stretch repositories
deb http://software.open-xchange.com/products/appsuite/stable/appsuiteui/DebianStretch/ /
deb http://software.open-xchange.com/products/appsuite/stable/backend/DebianStretch/ /
EOF

Now you are ready to install the server. Run the following commands to get the necessary packages. It is recommended to upgrade your system before installation.

apt update
apt install mariadb-server mariadb-client
apt install open-xchange open-xchange-appsuite open-xchange-smtp \
open-xchange-imap open-xchange-authentication-imap dovecot-imapd \
dovecot-lmtpd dovecot-managesieved dovecot-mysql apache2

Add group and user vmail, create directory /srv/vmail/ and chown it to vmail:vmail, chmod the directory to 0700.

Configuring things

Lets start by configuring dovecot using minimal setup. As stated in the start of this blog, this is for small installations. We use MySQL for user authentication. You can choose to use either Dovecot or App Suite for authentication, both work. You can replace /etc/dovecot/dovecot.conf with this, or find out relevant places in the split config to make doveconf -n look like this.

mail_home=/srv/vmail/%Ld/%Lu
mail_location=sdbox:~/Mail
 
mail_uid = vmail
mail_gid = vmail
mail_attribute_dict = file:%h/Mail/dovecot-attributes

## this is sometimes needed
#first_valid_uid = uid-of-vmail-user
 
ssl=yes
# you might want to change these to match whatever you are using
ssl_cert=<cert.pem
ssl_key=<key.pem
# generate this file on system with lots of entropy
# openssl dhparam 4096 (or if you are in hurry, 1024 or 2048)
ssl_dh=<dh.pem
ssl_cipher_list = ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384
ssl_min_protocol = TLSv1
ssl_prefer_server_ciphers = yes
protocols = imap lmtp
# Needed for appsuite
imap_capability = +XDOVECOT    
## if you are using dovecot for authentication, use
#passdb {
#  driver = mysql
#  args = /etc/dovecot/dovecot-mysql-auth.conf
#}

namespace inbox {
  inbox = yes
  list = yes
  location =
  mailbox Drafts {
    special_use = \Drafts
    auto = create
  }
  mailbox Spam {
    special_use = \Junk
    auto = create
  }
  mailbox Sent {
    special_use = \Sent
    auto = create
  }
  mailbox Trash {
    special_use = \Trash
    auto = create
  }
  prefix =
  separator = /
  subscriptions = yes
  type = private
}

Then restart dovecot and make sure it works.

telnet localhost 143
* OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE LITERAL+ STARTTLS AUTH=PLAIN] Dovecot ready.
1 LOGOUT
* BYE Logging out
1 OK Logout completed.

Authenticating with MySQL

Create a database with following schema. If you get problems with the primary key, you can either shorten the username and domain field, or you can configure mariadb using following things.

innodb_file_format=Barracuda;
innodb_file_per_table=ON;
innodb_large_prefix=1;
CREATE DATABASE users (
  username VARCHAR(255) NOT NULL,
  domain VARCHAR(255) NOT NULL,
  password VARCHAR(255) NOT NULL,
  PRIMARY KEY(username, domain)
) ROW_FORMAT=DYNAMIC;

Create file /etc/dovecot/dovecot-auth-sql.conf

driver = mysql
connect = host=/var/run/mysqld/mysqld.sock dbname=mails user=admin password=pass
password_query = SELECT username, domain, password \
  FROM users WHERE username = '%n' AND domain = '%d'
iterate_query = SELECT username, domain FROM users;

Authenticating with App Suite

Using MySQL

driver = mysql
connect = host=/var/run/mysqld/mysqld.sock dbname=mails user=admin password=pass
password_query = SELECT CONCAT(passwordMech, userPassword) AS password FROM user WHERE imapLogin = '%u';
user_query = SELECT imapLogin AS user FROM user WHERE imapLogin = '%u';
iterate_query = SELECT imapLogin AS user FROM user WHERE imapLogin IS NOT NULL;

Configuring App Suite

Stop open-xchange service. Then change following settings and start open-xchange. Give it few minutes to start.

server.properties

com.openexchange.forceHTTPS=true
com.openexchange.servlet.contentSecurityPolicy="script-src 'self'; object-src 'none'"
com.openexchange.connector.networkListenerHost=localhost
com.openexchange.server.backendRoute=APP1
com.openexchange.server.knownProxies=127.0.0.1
com.openexchange.rest.services.basic-auth.login=rest-login
com.openexchange.rest.services.basic-auth.password=<random string>

Configuring Apache

Put this file into webmail.conf

<VirtualHost *:80>
       ServerAdmin webmaster@domain.name
       ServerName webmail.domain.name

       Redirect permanent / https://webmail.domain.name/
</VirtualHost>

<VirtualHost *:443>
       ServerAdmin webmaster@domain.name
       ServerName webmail.domain.name

       DocumentRoot /var/www/html
       <Directory /var/www/html>
               Options None
               AllowOverride None
               Order allow,deny
               allow from all
               RedirectMatch ^/$ /appsuite/
       </Directory>

       <Directory /var/www/html/appsuite>
               Options None +SymLinksIfOwnerMatch
               AllowOverride All
       </Directory>

       RequestHeader set X-Forwarded-Proto "https"
       SSLCertificateFile      fullchain.pem
       SSLCertificateKeyFile   privkey.pem
<IfModule mod_proxy_http.c>
   ProxyRequests Off
   ProxyStatus On
   # When enabled, this option will pass the Host: line from the incoming request to the proxied host.
   ProxyPreserveHost On
   # Please note that the servlet path to the soap API has changed:
   <Location /webservices>
       # restrict access to the soap provisioning API
       Order Deny,Allow
       Deny from all
       Allow from 127.0.0.1
       # you might add more ip addresses / networks here
       # Allow from 192.168 10 172.16
   </Location>

   # The old path is kept for compatibility reasons
   <Location /servlet/axis2/services>
       Order Deny,Allow
       Deny from all
       Allow from 127.0.0.1
   </Location>

   # Enable the balancer manager mentioned in
   # http://oxpedia.org/wiki/index.php?title=AppSuite:Running_a_cluster#Updating_a_Cluster
   <IfModule mod_status.c>
     <Location /balancer-manager>
       SetHandler balancer-manager
       Order Deny,Allow
       Deny from all
       Allow from 127.0.0.1
     </Location>
   </IfModule>

   <Proxy balancer://oxcluster>
      Order deny,allow
      Allow from all
      # multiple server setups need to have the hostname inserted instead localhost
      BalancerMember http://localhost:8009 timeout=100 smax=0 ttl=60 retry=60 loadfactor=50 route=APP1
      # Enable and maybe add additional hosts running OX here
      # BalancerMember http://oxhost2:8009 timeout=100 smax=0 ttl=60 retry=60 loadfactor=50 route=APP2
      ProxySet stickysession=JSESSIONID|jsessionid scolonpathdelim=On
      SetEnv proxy-initial-not-pooled
      SetEnv proxy-sendchunked
   </Proxy>

   # When specifying additional mappings via the ProxyPass directive be aware that the first matching rule wins. Overlapping urls of
   # mappings have to be ordered from longest URL to shortest URL.
   #
   # Example:
   #   ProxyPass /ajax      balancer://oxcluster_with_100s_timeout/ajax
   #   ProxyPass /ajax/test balancer://oxcluster_with_200s_timeout/ajax/test
   #
   # Requests to /ajax/test would have a timeout of 100s instead of 200s
   #
   # See:
   # - http://httpd.apache.org/docs/current/mod/mod_proxy.html#proxypass Ordering ProxyPass Directives
   # - http://httpd.apache.org/docs/current/mod/mod_proxy.html#workers Worker Sharing
   ProxyPass /ajax balancer://oxcluster/ajax
   ProxyPass /appsuite/api balancer://oxcluster/ajax
   ProxyPass /drive balancer://oxcluster/drive
   ProxyPass /infostore balancer://oxcluster/infostore
   ProxyPass /publications balancer://oxcluster/publications
   ProxyPass /realtime balancer://oxcluster/realtime
   ProxyPass /servlet balancer://oxcluster/servlet
   ProxyPass /webservices balancer://oxcluster/webservices

   #ProxyPass /documentconverterws balancer://oxcluster_docs/documentconverterws

   ProxyPass /usm-json balancer://eas_oxcluster/usm-json
   ProxyPass /Microsoft-Server-ActiveSync balancer://eas_oxcluster/Microsoft-Server-ActiveSync
</IfModule>
</virtualHost>

Setting up App Suite

Before you can use your new App Suite installation, it needs to be configured. Begin by ensuring 'hostname -f' works.

Initialize config database

/opt/open-xchange/sbin/initconfigdb --configdb-pass=password --configdb-dbname=ox_configdb -a

Then run following command

# if you want to use dovecot as authentication source, use --mail-login-src mail
/opt/open-xchange/sbin/oxinstaller --servername oxserver --mail-login-src login --mail-server-src global --transport-server-src global \
-configdb-user openexchange --configdb-pass password --configdb-dbname ox_configdb --master-pass password --no-license --servermemory 1024
service open-xchange restart

Then we need to register the server.

/opt/open-xchange/sbin/registerserver -n oxserver -A oxadminmaster -P password

Then we need to setup container for your users. Each user in the container has same domain name, if you want multiple domains, create multiple containers. You can add alias addresses for your users from other domains.

/opt/open-xchange/sbin/createcontext -c 1 -A oxadminumaster -P password -N domain.name 

Open vSwitch

No L2 for you

We started experimenting with UpCloud Ltd. cloud services. Our goal was to move most of our infrastructure there behind a vFirewall, such as pfSense. Unfortunately we found out that even within same region, you can end up in more than one L2, so putting routes, default or any kind, towards your pfSense is not really going to work.

After discussing with UpCloud people they suggested creating overlay network, using e.g. Open vSwitch to it. So, how hard can that be? Well. Turns out it can be quite hard. At least to me, the documentation appears to be, as usual, highly fragmented, partially out of date and conflicting, and everything is done by combining little bits together.

Outline of vSwitch

What does vSwitch do? Well, it's a switch. At most simple case, you have a bridge, where you can attach one or more interfaces. The idea in most setups is to run the switch in your host and put the guest interfaces to your vSwitch bridge along with, say, vxlan tunnel for making logical L2 topology.

So, I setup openvswitch-switch, and found a handy guide on creating vxlan tunnel between two switches. The command is

# ovs-vsctl add-br ovsbr0
# ovs-vsctl add-interface ovsbr0 vxlan0 -- set interface type=vxlan remote_ip=some.other.ip
# ip addr add 10.123.0.1/24 dev ovsbr0
and then you provide IP for ovsbr0 on both sides, and lo presto, you get virtual L2 network. In correct usage, you would not need necessarly even add an IP for your interface, you could just attach your vnet's or docker0 device to it and it would probably work out just fine.

Our case though is that we can't run switch in host, we need to run it on the guest. Nothing wrong with it, but it's bit more complicated. Well, only because it becomes pretty tedious to manually create all those vxlan tunnels, it's not going to automate at all.

Ryu controller to rescue

Clearly automation is needed. If you look for a OpenFlow controller you'll find some. Not that documentation is very useful, as you are mostly expected to DIY everything. I chose ryu, because it's python. Unfortunately ryu seems to be having some pause in development, since there are pull requests from 2015 unmerged.UPDATE: ryu development seems to be quite active, based on their mailing list.

I started experimenting with simple_switch_13.py and quickly got a controller up and running, but the downside is that it doesn't really do anything useful in my case. What I understood was that I need something that changes configuration on the switch automatically.

Automation for lazy

Armed with grep, guesswork and mostly just staring at the example code for few days, I understood that what I need is a manager. Manager can configure a vSwitch. Manager can be either active (pull) or passive (push) type. Because I don't need to constantly do things, and because it turned out to be a lot easier, I opted for push configuration. Only to find out that ryu's library had a bug. Turns out, the bug is easy to fix by synchronising the copy of Idl component in ryu from ovs, but it took quite long moment to figure out, because even though ryu's Idl inherits the one in ovs, it does not actually call the parent constructor.

That sorted, I started to think how to make the configuration happen when switch comes online. It should happen on it's own, without any provoking, and while you could make a REST API to call, I thought, why? It will connect to the controller, so I should hook to that, somehow, and then automatically setup the VxLAN pair.

After many frustrating hours, I was able to figure this out. So now whenever a vSwitch connects to a controller, it automatically connects to it, and ensures it has VxLAN for the hub, and connects to hub as well, and adds VxLAN pair for the spoke.

Last hurdle

The hub is special case, because it needs modified ryu code to work. But the spokes, they must work without customized scripts as much as possible. This was somewhat hard to figure out, mostly due to lack of documentation, but if you install openvswitch-switch package in Debian, it also installs useful helpers for configuring openvswitch, so it was actually possible to put everything into /etc/network/interfaces file. I also added dhcpd for automating IP distribution.

In the end, I had very good, functional system, that works with debian jessie, at least. You can find all this at https://github.com/cmouse/ryu-auto-vxlan. README.md has the interfaces file described.

New look on site again

A new look on site, and some new tricks. Not using PHP at all on this iteration, because I could not find any use for it here. Site layout made with twitter bootstrap and added a theme for the fun of it to get it dark. Why dark? Because it's easier on my eyes.

Matching text

While C/C++ provides the power of Regular Expressions, sometimes a slightly more simple matching is desired. FnMatch provides * and ? based mask support, but for non-filename use I find it bit suprising, and it is not nil safe. Following code is released to public domain (in countries where public domain is not applicable, CC BY 4.0 is applied).

The code is fashioned after fnmatch, but removes any special cases from it. You might be worried about the recursion there, but it must be evaluated against the SUBJECT and MASK, not just mask. And if it worries you, add some recursion prevention measures there, like counter.

#include <string>
#include <cctype>

class SimpleMatch
{
public:
  SimpleMatch(const std::string &mask, bool caseFold = false)
  {
    this->d_mask = mask;
    this->d_fold = caseFold;
  }

  bool match(std::string::const_iterator mi,
             std::string::const_iterator mend, 
             std::string::const_iterator vi,
             std::string::const_iterator vend)
  {
    for(;;mi++) {
      if (mi == mend) {
        return vi == vend;
      } else if (*mi == '?') {
        if (vi == vend) return false;
        vi++;
      } else if (*mi == '*') {
        while(*mi == '*') mi++;
        if (mi == d_mask.end()) return true;
        while(vi != vend) {
          if (match(mi,mend,vi,vend)) return true;
          vi++;
        }
        return false;
      } else {
        if ((mi == mend && vi != vend)||
            (mi != mend && vi == vend)) return false;
        if (d_fold) {
          if (std::tolower(*mi) != std::tolower(*vi)) return false;
        } else {
          if (*mi != *vi) return false;
        }
        vi++;
      }
    }
  }

  bool match(const std::string& value) {
    return match(d_mask.begin(), d_mask.end(), 
                 value.begin(), value.end());
  }

private:
  std::string d_mask;
  bool d_fold;
};

Managing NLNOG ring keys in git repository

NLNOG ring is a community which provides visibility within other AS to it's members by having each member provide a server with access. This access is done with OpenSSH public keys. If you are a small operator with, say, 1 or 2 keys it is not really difficult to manage these keys. When you are a larger company with tens or hundreds of keys to maintain, it becomes more difficult. To solve this, I wrote this script for our users so that keys are kept in git repository, in gitlab and also in NLNOG ring. You keep one (or more) key per user in a per-user file, which is then concatenated when you update git repository in manage.ring.nlnog.net.

First, create a git repository in manage.ring.nlnog.net. (You can use whatever repo name you want, I used keys. It can also be under some directory)

~$ mkdir keys
~$ cd keys
# we need a bare repository for this to work
~/keys$ git init --bare
Initialized empty repository in /home/user/keys

Then create hooks/post-update under repository directory with content from https://gist.github.com/cmouse/4dc3c24d5cdcaad0f681. Remember to set executable bit on it.

This script will basically concatenate any *.key files into $HOME/ssh-keys file. Now, if you are using some VCS in your company, you can keep the files there, and have people who understand what they are doing push the keys into ring management.

On your local system, execute following commands to setup your key repository

~$ mkdir keys
~$ cd keys
~/keys$ git init
Initialized empty repository in /home/user/keys/.git/
~/keys$ git remote add ring username@manage.ring.nlnog.net:keys

After this, your workflow is basically

  • create user.key with openssh public key(s)
  • git add user.key
  • git commit user.key
  • git push ring master

This will cause keys to be updated on ring and once their puppet runs, you're up to date.

2015-03-23 12:53 Update: added information about how to use git.

Formless buttons in Rails

I wanted to make a simple toggle button inside a form in Rails, and I tried to do it with <%= button_to .. %>, only to find out that nested forms are not accepted. So, I made a small helper script to allow buttons work without a nested form.

$(document).ready(function(){
$(document).delegate(".btn-remote", "click.rails", function(e){
var button = $(this);
if (!$.rails.allowAction(button)) return $.rails.stopEverything(e);
$.rails.handleRemote(button);
return false;
});
});

Fixing ubuntu/debian json

A simple way to get the original code back. For the ones that just want it fixed fast, head to https://cmouse.fi/php-json-orig/ for packages. To build the sources yourself, run

dpkg-buildpackage -b -uc -us

How to make it yourself then?

First, grab yourself php sources. I used 5.5.9 as that is the base they use in ubuntu today. This will change eventually.

Then you make directory /tmp/php-orig-json-1.0-1 and copy everything under php-5.5.9/ext/json/ to it. After this, compress the /tmp/php-orig-json-1.0-1 directory into /tmp/php-json-orig_1.0.orig.tar.xz.

Now, to debian stuff.

Make debian directory, and put following files into it

changelog

php-json-orig (1.0-1) trusty; urgency=medium

  * Original version of PHP json

 -- your name <your@email>  output of date -R

Note that there are two spaces between email and date. Important.

Then

echo 9 > compat

You want to grab the original copyright message for the json code and toss it into copyright file.

json.ini

; configuration for php json module
; priority=20
extension=json.so

php5-json-orig.dirs

usr/lib/php5
usr/lib/php5/json

php5-json-orig.install

*/php_json.h usr/include/php5/ext/json/

php5-json-orig.php5

mod debian/json.ini

rules

#!/usr/bin/make -f
# This has to be exported to make some magic below work.
export DH_OPTIONS

DEB_HOST_GNU_TYPE    ?= $(shell dpkg-architecture -qDEB_HOST_GNU_TYPE)
DEB_BUILD_GNU_TYPE   ?= $(shell dpkg-architecture -qDEB_BUILD_GNU_TYPE)
SOURCE_VERSION        = $(shell dpkg-parsechangelog | grep ^Version | sed "s/Version: //")

CPPFLAGS:=$(shell dpkg-buildflags --get CPPFLAGS)
CFLAGS:=$(shell dpkg-buildflags --get CFLAGS)
CXXFLAGS:=$(shell dpkg-buildflags --get CXXFLAGS)
LDFLAGS:=$(shell dpkg-buildflags --get LDFLAGS)

# CPPFLAGS not used by makefile
CFLAGS += -O2 -Wall -fno-strict-aliasing $(CPPFLAGS)

ifeq ($(DEB_HOST_GNU_TYPE), $(findstring $(DEB_HOST_GNU_TYPE), ia64-linux-gnu powerpc64-linux-gnu aarch64-linux-gnu))
  CFLAGS += -g
else
  CFLAGS += -gstabs
endif

build: build-php5-stamp
build-php5-stamp: configure-php5-stamp
        dh_testdir
        # Add here commands to compile the package.
        cd build-php5 && $(MAKE) CFLAGS="$(CFLAGS)" && $(MAKE) test NO_INTERACTION=1

        touch build-php5-stamp

configure: configure-php5-stamp
configure-php5-stamp:
        dh_testdir
        rm -rf build-php5 && mkdir build-php5
        cp -r config.m4 config.w32 CREDITS json.c json.dsp JSON_parser.c JSON_parser.h package.xml php_json.h README tests utf8_decode.c utf8_decode.h build-php5
        cd build-php5 && phpize && \
            CFLAGS="$(CFLAGS)" CPPFLAGS="$(CPPFLAGS)" LDFLAGS="$(LDFLAGS)" \
            ./configure --build=$(DEB_BUILD_GNU_TYPE) --host=$(DEB_HOST_GNU_TYPE) \
                --prefix=/usr \
                --with-php-config=/usr/bin/php-config5 \
                --disable-rpath \
                --disable-static \
                --with-json=shared,/usr

        touch configure-php5-stamp

clean:
        dh_testdir
        dh_testroot
        rm -f configure-php5-stamp
        rm -f build-php5-stamp
        rm -f install-stamp

        # Add here commands to clean up after the build process.
        rm -rf build-php5

        dh_clean

install: DH_OPTIONS=
install: build
        dh_testdir
        dh_testroot
        dh_clean -k
        dh_installdirs

        # Add here commands to install the package into debian/php5-json-orig.
        (ext=`/usr/bin/php-config5 --extension-dir`;mkdir -p debian/php5-json-orig/$${ext};install -m 644 -o root -g root build-php5/modules/json.so debian/php5-json-orig/$${ext}/json.so;)
        mkdir -p debian/php5-json-orig/etc/php5/mods-available
        install -m 644 debian/json.ini debian/php5-json-orig/etc/php5/mods-available/json.ini

        touch install-stamp

# Build architecture-independent files here.
binary-indep:

# Build architecture-dependent files here.
binary-arch: DH_OPTIONS=
binary-arch: build install
        # Need this version of debhelper for DH_OPTIONS to work.
        dh_testdir
        dh_testroot
        dh_installdebconf
        dh_installdocs

        dh_installchangelogs
        dh_strip
        dh_link
        dh_compress
        dh_fixperms
        dh_installdeb
        dh_shlibdeps
        echo "php:Depends=phpapi-`php-config5 --phpapi`" >> debian/php5-json-orig.substvars

        dh_gencontrol
        dh_md5sums
        dh_builddeb

binary: binary-arch binary-indep
.PHONY: build clean binary-indep binary-arch binary install configure

Now that you have functional debian directory, you can go into /tmp/php-orig-json-1.0_1/ and issue dpkg-buildpackage. Add -uc -us if you don't want signed changes. Then you can install package with dpkg and it should replace the json module provided by ubuntu.

Converting PCKS#12 PFX into Java keystore

You have a key and certificate in PFX format, and would like to get them into a java keystore with least hassle. Here"s how I do it:

#!/bin/sh
# (c) Aki Tuomi 2012 
# You are free to use it as you like, no warranty or guarantee
#

base=`basename $1 .pfx`

if [ -z "$1" ] || [ -z "$2" ]; then
  echo "Usage $0 file.pfx alias [password]"
  exit 1
fi

# extract key
openssl pkcs12 -nodes -nocerts -in $1 -out $base.key

# extract certificate
openssl pkcs12 -nodes -nokeys -in $1 -out $base.crt

# recode key into pkcs8
openssl pkcs8 -topk8 -nocrypt -in $base.key \
-outform DER -out $base.pk8

# recode certs into pkcs7
openssl crl2pkcs7 -nocrl -certfile $base.crt \
-outform DER -out $base.p7b

# just in case
rm -f $base.jks

# create jks
java ImportKey $base.pk8 $base.p7b "$2" $base.jks

if [ "$3" != "" ]; then
   keytool -keypasswd -alias "$2" -keypass changeme \
-new "$3" -keystore $base.jks -storepass changeme
   keytool -storepasswd -new "$3" -keystore $base.jks \
-storepass changeme
fi

# cleanup
rm -f $base.key $base.crt $base.pk8 $base.p7b

You also need a Java snippet for inserting keys into java keystore, as this is not supported by keytool. https://cmouse.fi/ImportKey.java

Jul. 9th, 2012

Getting rid of problem waste in Finland

Recycling is easy, or so they claim here. And complain that people are not recycling properly. Wonder why?

Last weekend, when cleaning up my mother-in-law's cellar, I had to dispose a set of old winter tyres. Simple thing, eh?

There are few ways to get rid of them:

  • Take them to Kuusakoski, which is nearest, and open from 8-17 every week day. Handy for anyone working normal day.
  • Take them to HSY Sortti-station, open from 8-21, every week day. Better, but not really usable during weekends

This applies to many other types of difficult to dispose waste as well. The places you can take these are open during week, only, and at least the nearest Kuusakoski was so damn difficult to find that it took me 10 minutes riding around before I could figure out how to get there.

Why is it so impossible to do this sensibly, by having few places that are open also on weekends, when people actually need them, and have that place somewhere you can get with public transportation, or by car easily. Now it's been made so difficult that most people opt to leave their stuff in any place they dare and then the victims complain that people do not know how to recycle. They do, but it's just too damn difficult.

Apr. 1st, 2012

Creating map cache for Neongeo

Now that I have android phone I decided to give geocaching software Neongeo a go. Works like charm, but I really like having my maps downloaded for faster operations and less waiting for downloads. Not to mention the cost.

Anyways, Neongeo wiki has no instructions, so I decided to write 'em here, and see if I can get write permissions there to copy these there then.

Steps to perform

Download and install MObile Atlas Creator. You can get it from http://mobac.sourceforge.net/

Create map cache

Create new atlas with OpenStreetMap or Google maps. The working map type is Rmaps SQLite. Choose layers from 10-18. You can choose multiple rectangles for your cache. I did it for the region I normally stay. I used OpenStreetMap MapQuest because I like the way the mark paths and terrain to the map, making cache hunting easier.

Wait until map downloads

Copy the resulting SQLite database into Card\Android\data\com.neongeo.app\files\map

Open Neongeo, SettingsMap ServersEdit mapserverGoogleMaps or OpenStreetMaps (depending your choice). Click on 'Offline Map', and choose your SQLite database. Click 'ok'.

And you are done. Works like a charm.

Reinforcing computer table

I've noticed that my computer table from Ikea had sagged and started to keep disturbing noise. It also wobbled when poked, quite disturbingly. After having watched it for some time, I decided to actually do something about it. I decided to reinforce it.

First, I went to Ikea to see if they still sell it, but unfortunately they no longer did. Oh well, had to do what I had. Next stop was hardware shop where I bought some angle irons and a box of screws, and then to home.

First up, removal of all stuff from the table, sort what goes back.

Quite a lot of stuff, that. The white piece of board with the extension cord screwed on is the white cover plate that goes under the table, but it's been shortened a bit to make room for exhaust air from the PC.

Then, it's time to rebuild the table, and add glue to strategic places. The black thing is a mount rail for my desktop PC, so that I can keep it off the floor, and off the table. The yellow cord is for holding my sub woofer up.

Then it's time to add the white cover plate and screw in the angle irons. They really steady the table a lot. The ones on the cover plate are really quite mandatory, otherwise the cover plate would no longer stay in place as it has been removed a chunk at the end.

Then, the only thing left is to install the "wire rack", and put all the wires in place, along with everything else that goes on the table.

And so, quite pleasing end result is achieved. This last picture was taken later on, as I had to wait for a card reader having lost my connector cable for the camera.

The total cost was about 10 EUR, and I think it was money well spent.

Fear and doubt in Europe

It has been interesting to notice how media is a tool for bulk dissemination for fear, doubt and uncertainty. I do not know if they do it intentionally, or is it just by-product of greed.

Take, for example, the recent account on financial crisis in Europe and USA. The headlines are absolutely stunning, ranging from "it is all over now" to "salvation reached". And not to forget to chide politicians for the same decision that will be promoted the very next issue, or previous one.

This misinformation that media spews about the financial crisis causes more fear in the markets, provoking the issue. And normal citizens cannot even understand how the finance works these days, so giving them these incomplete, and in most cases, dangerously wrong simplifications usually causes just more problems than anything else. I am not saying that people should be kept in the dark, but, well, there is no moral in scaring people stiff for profit.

The same happens with the latest killing in Norway. Media has been relentless in disseminating absolute fear and terror from the case, by connecting the case with whatever they can get away with, and making every unrelated act seem like a dangling step into chaos. Just the other day, Finnish media proclaimed with horror that some 18 yr old boy had ordered the same kind of fertilizer component from Poland than the Norway killer. No mind that you actually need bit more than just some fertilizer component to make a bomb, or that the guy had actually tried to cancel the order, it is proclaimed as one step before chaos in the headlines.

Is it then wonder that we find lots of extreme solutions to these so called "problems". Just to mention few, banning of violent games, limitations to free speech, limitation to anonymity, forming of fertilizer registry, etc. etc. The bodies in Norway have barely had time to cool, when politicians, and their groups, all over the world, start to proclaim dangerous and very dangerous limitations to people's freedom in the name of "security". The slippery slope into totalitarian world.

Be scared, what you are lead to, when you are afraid.