zen.org Communal Weblog

I post it here so I can find it again if needed again. Sometimes I just want to see everything that is going on in a perl object. Sort of what Java’s toString() function does when printing out maps or lists. This is what I ended up with. Now it’s public domain!

our $join = ' ';
sub to_string{ # ug, java
    my $s = shift;
    my $out;
    if (ref $s) {
	if ($s =~ m/ARRAY/) {
	    $out .= '[' . join($join, map {
		to_string($_);
	    } @$_) . ']‘;
	} elsif ($s =~ m/HASH/) {
	    $out .= ‘{’ . join($join, map {
		“$_=” . to_string($s->{$_});
	    } keys(%$s)) . ‘}’;
	} else {
	    $out .= $s;
	    warn “How do i display ‘$s’?”;
	}
    } else {
	$out .= quotemeta($s);
    }
    return $out;
}

A friend of mine is set up with a satellite Internet connection to his home in a not-all-that-rural part of Ireland. He’s been hosting his domain from there, with all email traffic and such going to his local server. Until recently, it was a perfectly workable solution, even with the normal supply of spam, virus, and other junk mail arriving.

But nearly two weeks ago, his domain came under attack from a bunch of spam botnets. Uncountable messages were forged to various places, all of which set up with the Sender: header to be totally random addresses @domain.ie. Unfortunately his ISP said they would not help block the traffic. (As opposed to could not.)

The workaround we came up with pushed his traffic through a virtual-hosted system I have set up over in the US with johncompanies.com (yes, a blatant plug, but I really like their service). There were a few steps I had to take in configuring Postfix before they added the MX record for his domain to reroute everything. (This is on a system running Debian GNU/Linux version 4.0, codenamed etch, using postfix 2.3.7.)

• In main.cf, add his domain to relay_domains (which already existed for other domains I MX with).
• Since he uses a lot of different email addresses (to make it easy to catch re-use and selling of them), I didn’t set up a relay_recipient_maps hash table. That would have been even cooler with its ability to block every single address except for the few that are in fact valid. In this case, however, he had a number of variants of addresses he used so it wasn’t a practical choice.
• Add to smtpd_recipient_restrictions the line
  

  check_recipient_access hash:/etc/postfix/maps/access_recipient

  and created the file /etc/postfix/access_recipient containing
  

  postmaster@domain.ie  REJECT
  MAILER-DAEMON@domain.ie       REJECT

  and then ran postmap access_recipient as root. I should note I did not put a line like domain.ie OK which would have let all other mail for the domain go through—but usurped any other rules that smtpd_recipient_restrictions may try to do after my access_recipients entry.

• I created a /etc/postfix/access_sender file with the lines below. The first was used because his server will never receive mail from someone in his domain.
  

  domain.ie       REJECT
  MAILER-DAEMON@  REJECT
  MailerDaemon@   REJECT
  abuse@          REJECT
  admin@          REJECT
  Administrator@ REJECT
  autoresponder@  REJECT
  bounce@         REJECT
  info@           REJECT
  majordomo@      REJECT
  Majordomo-Owner@ REJECT
  nobody@         REJECT
  postmaster@     REJECT
  savrequest@     REJECT
  senderchallenge@ REJECT
  spam@   REJECT
  vacation@       REJECT
  

  Then I had to run postmap access_sender as root. In main.cf, for smtpd_sender_restrictions I added
  

  check_sender_access hash:/etc/postfix/access_sender

  as well.

• I found I wanted to add some rules that used regular expressions. After installing the postfix-pcre Debian package, I created a new file /etc/postfix/access_sender.pcre with the lines
  

  /.*bounces\@/   REJECT
  /confirm-return.*\@/    REJECT

  and in main.cf gave smtpd_sender_restrictions yet another entry of
  

  check_sender_access pcre:/etc/postfix/access_sender.pcre

• Following the hints from a post by Justin Mason, I created a new file /etc/postfix/header_checks and gave it the lines
  

  /^Content-Type: multipart\/report; report-type=delivery-status\;/       REJECT no third-party DSNs
  /^Content-Type: message\/delivery-status; /     REJECT no third-party DSNs

  A second file, /etc/postfix/null_sender, had
  

  <>      550 no third-party DSNs

  In main.cf I gave the smtpd_sender_restrictions list the new entry of
  

  hash:/etc/postfix/null_sender

  and also added a new line defining header_checks as
  

  header_checks = regexp:/etc/postfix/header_checks

  Finally I had to run postmap null_sender as root.

• In master.cf I had to adjust the smtp unix and relay unix entries to only do 2 processes, not the default of 20, since having my machine try 20 simultaneous connections to his machine wouldn’t help. Under each, respectively, I had to add
  

  -o smtp_destination_concurrency_limit=2

  and
  

  -o relay_destination_concurrency_limit=2

  I’m still not positive if the maximum of 2 processes would make these options necessary. I should note that this particular system I was setting up did no other mail delivery, so this change was okay. If you’re doing this on a fully production-level host, you might find a different way to throttle the delivery connections going to a specific host, instead of this change which makes all outgoing mail connections happen only two-at-a-time.

• He’s closed port 25 on his router to try to at least stop the flood. Instead, he’s opening a random port number (like 1767) and having it listen there for new mail. I’ve made postfix deliver it by creating a /etc/postfix/transport file with the lines
  

  # 20080327 help fight the flood, tunnel the mail to its real destination, e.g., his server is 1.2.3.4
  domain.ie     :[1.2.3.4]:1767
  .domain.ie    :[1.2.3.4]:1767

  and ran postmap transport as root. Into main.cf I added
  

  transport_maps = hash:/etc/postfix/transport

• After all of this was done, I had to do postfix restart

The end result, with Justin’s rules in particular, has had thousands and thousands of attempts get blocked trying to get through the door. Some still trickle through, even after the amavis/clamav/spamassassin content filter has processed them.

This is the final accumulation (with a few I already had):

smtpd_sender_restrictions = check_sender_access hash:/etc/postfix/access_sender,
check_sender_access pcre:/etc/postfix/access_sender.pcre,
hash:/etc/postfix/null_sender

header_checks = regexp:/etc/postfix/header_checks

## Steps from http://www.akadia.com/services/postfix_spamassassin.html
smtpd_recipient_restrictions = permit_sasl_authenticated, permit_mynetworks,
reject_unauth_destination,
reject_unauth_pipelining,
reject_invalid_hostname,
reject_non_fqdn_hostname,
reject_non_fqdn_sender,
reject_non_fqdn_recipient,
reject_unknown_sender_domain,
reject_unknown_recipient_domain,
check_recipient_access hash:/etc/postfix/access_recipient,
check_recipient_access pcre:/etc/postfix/access_recipient.pcre,
check_policy_service inet:127.0.0.1:60000,
permit


(The check_policy_service line is for my use of postgrey, another simple step which drastically reduced the amount of spam my own server was getting.)

Please let me know if any of the instructions above prove to not work out properly for you.

P.S. A command I found handy watching the logs to see what was getting through for attempted delivery, even after everything above:

sudo tail -f /var/log/mail.log | egrep -v '((RCPT|connect(ion)?).* from |smtpd_peer_init)'

For the longest time I’ve been sticking with having to only ever visit www.ros.ie using W1ndow$ on my laptop. Being self-employed, every two months I have to give some tax to The Man.

This time, I decided to look again to see if anyone has discovered a way to do this without that other OS. Luckily, I found some notes by Andrew S. Townley explaining exactly how. He’s found the link into the ros.ie site to get at the actual KCrypto Java applet that it uses (and claims fails to start).

As described, I put it into /usr/lib/jvm/java-6-sun-1.6.0.03/jre/lib/ext and restarted Firefox. Now the login page on the site worked fine, and I could get in. Yay!

P.S. I’m doing this under Ubuntu 7.10 (Gutsy Gibbon).

We recently moved zen.org to a different server, and in the process my dump and reload of our MySQL database worked—mostly. However any posts with UTF-8 Unicode characters didn’t get displayed correctly.

After spending too much time trying to figure out how to make mysql and mysqldump help me, I realized I should look around for others who’ve had the same problem.

Voila, Jonkepon in Japan gave the fix for exactly the problem we had. The fix has to do with the collation of the entries in the database, not the actual dumping and importing of the content itself.

Since the newer Wordpress already does their first step with SET TABLE, I just had to go in via phpMyAdmin. For each of post_content in wp_posts and comment_content in wp_comments, I changed the collation of each to binary (noting the type of LONGTEXT or TEXT) and saved it. Then I edited them again and set each to utf8_unicode_ci, and saved them.

Bingo! All is happy and good again. The other tables are all still latin1_swedish_ci (?!), but I’ll leave them alone until we bump into somewhere else that it’s a problem.

I’m happy with my Dell Dimension C521 running Kubuntu, and now that I can write to the NTFS partition I can rip all kinds of CDs (NTFS is good for something). Four tops, no problem. India.Arie, Jill Scott, no ploblem. Lars Winnerbäck, Mel Tormé, Moby, Pete Fountain, Sublime, easy. Eagles, Nirvana, buzz! Out of a whim I try in on the old Quantex PII, was zen before the tree, no problem! I even tried it on the Dell at ×2. What does the old PII have that the AMD64 don’t?

Where I work we have to processes files from many sources, we have found three different line endings: “\n”, “\r\n”, and “\r”. We didn’t want to process all the files before, er, processing them so I wrote the following:

#!/usr/bin/perl
use warnings;
use strict;

until (($/ = getc(STDIN)) =~ m/[\r\n]/) {
die “no new line!” if (eof(STDIN));
};
if ($/ eq “\r”) {
$/ .= “\n” if (getc(STDIN) eq “\n”);
}
seek(STDIN,0,0); # rewind()

This will set $/ to which ever of the three file endings we have so you can then process them without thinking. Well, almost no thinking, you will need to use chomp instead of chop. I’m sure it could be nicely abstracted too, bit I didn’t.

This arrived in my mailbox just a few minutes ago; couldn’t resist sharing:

After a little difficulty, I’ve got it working! The music on the Mac Mini upstairs is coming out of the speakers of our 15 year-old Aiwa stereo in the diningroom downstairs.

My laptop’s got a Belkin TuneCastII FM Transmitter plugged into its headphone jack. The laptop (booted into WindowsXP) is running iTunes, playing a song—right now, “The Devil Went Down to Georgia” by The Charlie Daniels Band. And the laptop’s connected over our wireless network.

The Linksys WRT54G in the livingroom is running DD-WRT, a replacement firmware giving it a lot more oompf than what comes on the box by default. I specifically put it on to solve the continuous problem of Mac laptops losing their association with it after a few hours or a day.

Anyway, it turns out the only thing preventing the iTunes on my laptop from seeing the shared music off the mini upstairs was that I’d left the “SPI Firewall” enabled. Since I’ve got the wireless already as tight as I can get it (no broadcast, mac filter, wep encrypted and soon wpa), it’s probably not particularly useful since anything connected via an Ethernet cable wouldn’t be protected by it either.

With that firewall disabled, the packets (to whatever port(s)) made it through and can finally play music on my laptop!

Our 9 month-old boy doesn’t seem to like “Let It Roll”, the next song in that album. Let’s see if he likes The Eagles…

My brother sent me a funny joke recently, and I thought about throwing it up on the blog. But I figured I should first make sure it’s not one that’s done too much traveling around the Net. The basic idea is:

A husband and wife are traveling by car from Atlanta to New York. After almost twenty-four hours on the road, they decide to stop at a nice hotel and take a room. They only plan to sleep for four hours and then get back on the road. When they check out four hours later, the desk clerk hands them a bill for $350. The man explodes and demands to know why the charge is so high. He tells the clerk although it’s a nice hotel, the rooms certainly aren’t worth $350. When the clerk explains that $350 is the standard rate, the man insists on speaking to the manager.

The manager enters the conversation and explains that the hotel has an Olympic-sized pool and a huge conference center which were available for the husband and wife to use.

He also explains that they could have taken in one of the shows which the hotel is famous for. “The best entertainers from New York, Hollywood and Las Vegas perform here,” explains the manager.

No matter what facility the manager mentions, the man replies, “But we didn’t use it!”

The manager is unmoved. Eventually the man gives up and agrees to pay. He writes a check and hands it to the manager. “But sir,” the managers says, “this check is only made out for $100.”

“That’s right,” replies the man. “I charged you $250 for sleeping with my wife.”

“What! I didn’t sleep with your wife!” exclaims the manager.

“Well,” the man replies, “she was here, and you could have.”

Holy cow, it’s apparently pretty popular. So frequently used, in fact, that there exist a wide variety of adjusted versions involving:

• an unnamed husband and wife; Yonkel and Sara; Steve and Sarah; a Sardarji and his wife; a Uclutian and his wife; Sordarji and his wife; a biker and his wife;
• driving from Key West to Boston; Miami to Boston; Atlanta to New York; Florida to Canada; somewhere to Tofino; London to Southampton; Edinburgh to London; Delhi to Mumbai; Yorkshire to Scotland; Johannesburg to Cape Town; Sydney to Cairns;
• $350 US dollars; £350 British pounds; Rs. 5000/- Indian rupee; R750 South African rand;
• husband charges for: sleeping with his wife; kissing his wife.

I could actually take the time to turn it into an Ireland joke, but it sure seems almost as pointless as the last hour I spent looking at all of those sites.

I had too many passwords to remember at work, MacOS X did a good job of remembering them but it kept them in a way that was difficult to extract. At home, with Kubuntu, KWallet kept things encrypted but also gave me access to them with only one password. So I installed KDE of my University surplus system and got osx2x to work to share the keyboard with my much newer Mac. So now when I need one of my LDAP passwords in a place I have not used it before I can read it and not contact systems to reset my password.

The odd thing is, slowly, I’m doing more and more on my slow Debian box with KDE then on the duel processor Mac. MacOS X: iTunes, iCal, GNU Emacs 22, web browser testing. KDE: KMail, web browser use, OO.o, RSS, command line. I still keep the Mac monitor in front of me, hence Emacs. Before I installed KDE, I longed for a GUI sftp access in the Finder, now I use KDE’s file browser to browse my Mac files more then the Mac finder, even though it is right in front of me!

Gotta go, meeting.