Jeremy's Weblog

A few days ago, Linode.com announced native IPv6 roll out in their datacenters. Now, while I haven’t wrote about Linode in the past 6 months, I am still a happy customer. I am documenting the steps I took to migrate away from my HE.net tunnel.

1. Set the TTL low on any DNS addresses that you will be changing. Ideally, do this a fair amount ahead of time.
2. Send in a support ticket to get your /64 allocated. Sidenote: response time: 4 minutes
3. Reboot ‘node so the backend system deploys your IPv6 after it was allocated. Verify IPV6 status on your ‘node.
4. From a different IPv6 host, run nmap -6 on the existing address to verify listening services.
5. Update DNS, define static networking, be happy.

It should be no secret that this site and my other co-location needs are hosted at Linode.com, for which I am a happy customer running Gentoo Linux.

The reason for this post is that after an announcement today. All I had to do was reboot and then I received a 42% RAM increase. Yay.

Thanks Linode, you are exceeding your competition!
(Shameless plug for my referral code )

For some reason, Linode.com (my review) sets up their hosts to use dhcpd to grab the static IPv4 address on boot. This is in contrast to Host Virtual which uses the “Gentoo-way” to set static addresses. Now, there isn’t anything exactly wrong with using dhcpd on hosts with static addresses, actually, it may be simpler (and this is probably why they did it). However, I don’t like it for a few reasons, booting takes longer as it probes for IPs and it uses extra space for dhcpd binary on a low resource host – this includes extra time for updating. I know these are minor issues, but they bother me, at least. So, let’s take the easy way and assign eth0 the IP is should have:

Snippet from: /etc/conf.d/net
config_eth0=(
    "69.164.197.24 netmask 255.255.255.0 broadcast 69.164.197.255"
    )
routes_eth0=(
    "default via 69.164.197.1"
    )

So, that makes sense for all the right reasons and there is not much more to say. Let’s shift the attention to IPv6. Linode doesn’t offer IPv6 by default unlike their competition. To be honest, I don’t need IPv6, but it is something fun to play with and I have been learning something. It turns out that my tunnel from HE.net (free) is actually lower latency to some parts of the world than my IPv4 route and almost always less hops. Using the great examples from Robin (robbat2), I was able to put my IPv6 tunnel in /etc/conf.d/net too, so that is it created on reboot as tun0. Makes sense to do, right?

Snippet from: /etc/conf.d/net
HE="2001:0470" # 2001:470::/32 is the HE.net allocation
v6net64="${HE}:1f0f:2a0" # your initial /64 allocation from HE.net

# HE.net tunnel configuration
link_tun0="eth0" # tunnel IFACE (internet-facing iface, eg ppp0/eth0)
# tunnel IPv4 endpoint, remote, HE.net tells you this
iptunnel_tun0_remote="216.218.224.42"
# tunnel IPv4 endpoint, local
# this is the address of IFACE ${link_tun0}
iptunnel_tun0_local="69.164.197.24"

iptunnel_tun0="mode sit remote ${iptunnel_tun0_remote} local
${iptunnel_tun0_local} ttl 255 dev ${link_tun0}"
mtu_tun0=1280
config_tun0="${v6net64}::2/64" # /126
routes_tun0="default via ${v6net64}::1"

This is not exactly perfect, because I am using my tunnel’s /64 as my IPv6 address. The purists might say something about this practice, I respect that, but don’t really mind for my personal use. Of course, if you didn’t have a tunnel and instead had native IPv6, it would look a lot simpler because you just add the IP and route to the interface it is on, probably eth0.

My linode is now my personal DNS resolver. I have officially ditched the ISP nameservers from this point forward now that I found unbound. Unbound is a lightweight, recursive resolver that is perfect for your LAN, co-located host, or even a single host.

For your single host, emerge unbound, start the service, add 127.0.0.1 to the first nameserver in /etc/resolv.conf. Unbound is setup (by default) to accept connections from localhost and refuse anything else. If you are using dhcp at home (likely) then also emerge openresolv and uncomment name_servers=127.0.0.1 in /etc/resolvconf.conf, openresolv then “intercepts” dhcpcd when it tries to write to /etc/resolv.conf and adds 127.0.0.1 as your first nameserver For your LAN, just configure your router to look to the host that you setup unbound on, with additional configuration.

Finally, you can also have unbound run on your co-located host. Just edit /etc/unbound/unbound.conf to a) listen on an outside interface and b) allow your other host to query it. This will be left as an exercise for the reader, it is easy to figure out.

Lastly, a shout-out to Linux Gazette for an excellent write-up on GoogleDNS (and why you should use something like unbound) and DNS/LAN metaphors. Suggested reading if you feel out of your league with DNS internals, like me.

A quote from the above linked article: “Why outsource to anyone, when you can do a better job locally, at basically no cost in effort?“ and really, that is the truth. Have fun.

If you are running PHP on a limited-resource box, like a VPS then you may have seen your PHP pages randomly hang. I was able to trace this issue down because the PHP pages were hung up and the normal html pages were still being served. The problem was ‘solved’ when I restarted the web server. Some research later, and talking to Thilo (bangert), I found out about PHP_FCGI_MAX_REQUESTS. This is an environment variable that PHP respects, it basically tells how many requests to serve before respawning fcgi. In my case, 500 seemed like a good number after testing. Your mileage may vary, but it is worth a try if you have those symptoms.

%% cat /etc/lighttpd/mod_fastcgi.conf
server.modules += ("mod_fastcgi")
fastcgi.server = ( ".php" =>
    ( "localhost" =>
        (
            "socket"   => "/var/run/lighttpd/lighttpd-fastcgi-php-" + PID + ".socket",
            "bin-path" => "/usr/bin/php-cgi",
            "max-procs" => "2", # default 4
            "bin-environment" => (
                "PHP_FCGI_CHILDREN" => "2", # default 1
                "PHP_FCGI_MAX_REQUESTS" => "500" #default 1000
            )
        )
    )
)

About a month ago, I installed Gentoo on the new-to-me Acer Aspire1. Installation went like anything else, it is just a normal x86 host after all. I don’t have everything on it working, because I don’t care. If you are looking for additional resources on getting the extras working, you may want to look here or here.

The exciting part, that I got working and am ready to announce publicly, is my new atom-x86 binpkg repo. What makes this repo different than the binpkgs located on tinderbox.dev.gentoo.org/default-linux is that this repo has CFLAGS specific to the Intel Atom processor. I identified the compiler flags by using the following gcc command: gcc -Q --help=target -march=native and set the following -march=prescott -mtune=generic -msahf. On my linode (review) host, I have a chroot that builds all new packages in my world file once a day which comes from the aspire1. In this manor, I am able to always have binary packages available to me whenever I update my aspire1. Now, I have all the benefits of a source distro and the speed of a binary distro.

If you would like to use this repo, set PORTAGE_BINHOST in /etc/make.conf and add ‘getbinpkg’ to FEATURES (or use the emerge options directly). Be advised, that thought this works for me, I make no guarantees for you.

PORTAGE_BINHOST="http://tinderbox.jolexa.net/atom-x86/"
FEATURES="${FEATURES} getbinpkg"

I also have an html view of the packages available.

I had this genius idea about using sshfs with rtorrent. I thought that this use case would fit best in situations where you have good bandwidth but not much diskspace, such as my linode VPS (review). So, I’ll attempt to share my findings in this regard.

If you are not familiar with rtorrent. You just need to know that it is a powerful, lightweight bittorrent client. It has a “watch” feature that watches a directory for new torrents, and obviously it can put downloaded files in a specified location. I tried both of these with sshfs.

First, I was having trouble with rtorrent just ‘freezing’ up when I put a torrent file in the sshfs accessible watch dir. I didn’t quite know what was wrong here. Research led me to rtorrent bug 322 and that sshfs did not support filesystems without mmap properly. Darn. More research led me to a recent kernel commit that looked promising. Low and behold, reboot my host with 2.6.31.x kernel and rtorrent works with sshfs watch and destination directory. Yay.

Well, not so fast…

The performance is quite poor with the destination directory on sshfs. This is to be expected because now your download speed for torrents is limited to the download speed of your final destination. But, rtorrent was only giving me a sustained speed of 1/4 of that demonstrated with a simple file copy to the destination. I speculate that this is from the rtorrent overhead or maybe fragmenting? Not sure exactly and I don’t care. My solution to this was to use the rtorrent “move on finished” feature that downloads the file to local disk and then moves it to sshfs destination after it is finished. Amazingly, this works quite well.

My testing scenario was the following:
-79MB Gentoo 2008.0 install cd torrent. With the complete sshfs solution, it took ~6 minutes to download (to the sshfs destination) and then 5 minutes to check the hash. So, roundtrip of 11 minutes from start download to seeding. With the on_finished solution, it took 1 minute to download (to local disk) and 1 minute to check the hash and move to the sshfs destination. For a roundtrip of ~2 minutes from start of download to seeding.

In conclusion, this isn’t the perfect solution because you impose a large bottleneck into the mix and unintended I/O activity on the local disk. However, it works for me and what I am doing. Maybe it will give someone else some ideas in the future.

I recently had a bit of downtime on my linode. If you are wondering what a ‘linode‘ is, check out my review or the website. And a big thank you to the folks that used my referral code when they got setup with linode themselves, you guys rock!

So, about my recent 1/2 day downtime. It was self-inflicted because I wanted to move to a different datacenter. I moved my linode from Newark, NJ to Dallas, TX. It is quite a long story, but it boils down to a problem with my ISP (Comcast). I was only able to pull 100K/s from the Newark datacenter and 2-3M/s from the others. This was unacceptable. I tried to get it escalated past Comcast’s frontline support but they kept asking me questions like “Do you use a router? If so, each computer only gets 1/2 the speed” & “Every computer is different. I’m glad that you can get 3M/s from another host, that is really good” Sigh.

At least Linode’s customer server was helpful and allowed me to work around the ISP. The steps to move a linode are as follows:

1. File a support request. (My initial request was answered in 11 minutes)
2. Shutdown your linode
3. Hit the ‘migrate’ button, after support sets up your migration
4. Wait for the transfer. My total transfer time was ~43 minutes (~6G to transfer). This was pretty fast throughput, in my opinion
5. Meanwhile, update your DNS for your new IP.
6. Since you can queue up a boot job, I just let it go and checked in on it a couple hours later. Magic, it was online.

So, to finish the story off. Linode++, Comcast–. I wish I didn’t need to do something like this, I wish my ISP was…I don’t know…smart?