False Prophets

I have just committed a patch for inlined function support in GDB.

This is pretty neat stuff, if you ask me (but don’t - I’m obviously biased :-). GDB now treats inlined functions like normal functions whenever the optimized code - and debug information - permit it. For instance:

• The backtrace shows inlined functions. You can see their arguments and local variables. You can use up to look at the inlined function’s call site, instead of having to guess.
• If you’re stopped at a call to an inlined function, step will take you into it. next will skip it. next in particular is a little flaky; the logical operation doesn’t have a clear meaning in the presence of scheduling optimizations, which interleave the inlined function’s code with the code of its caller. So you may have to next a good number of times to get where you’re going, but at least it shouldn’t ever take you into the function you’re nexting over.
• finish will take you out of the inlined function. That may mean back to its call site, or on to the next line of code if the call site had no more work to do.

Of course, all of this neat functionality relies on debug information to understand the mapping from optimized binary back to user source code. The debug info used is DWARF, which is a modern and mostly cross-platform standard (obvious exception: Microsoft). None of this was possible using the older stabs debug format. (At least not stabs as understood by GDB; Sun may have had similar extensions to stabs for DBX.)

The more optimized your program is, the stranger that mapping and the worse using GDB on your program. There are two major causes of this: deficiencies in GDB’s understanding of the debug info, and deficiencies in GCC’s generation of it. This is a big step forward on the GDB side. Similarly huge steps are in progress in GCC, for instance Alexandre Oliva’s VTA (”variable tracking assignments”) branch. And having better GDB support and test cases will, I hope, help the GCC developers to fix more debug info generation bugs.

This patch is one of many recent improvements to GDB. I think there’s been a real sea change in the GDB community; GDB development was slow for many years, with much of the attention on avoiding bit-rot and occasional ports to new CPU architectures. But now exciting things are happening again! There’s a number of factors I can point at to explain the change. A big one is the recent work on multi-process and non-stop debugging, by Pedro Alves and Stan Shebs (and others!). That has convinced others, both individuals and corporations, that the GDB project is worth spending time on. There’s been a cascade of interest, leading to e.g. Python scripting support in the development tree. Finally, STL pretty-printers! Another big factor has been the emergence, and re-emergence, of active maintainers with a good global understanding of GDB; people like Pedro, Stan, Joel Brobecker, Ulrich Weigand, and Tom Tromey. They’ve made the lists a friendlier and much more responsive place than I can ever remember since I started working on GDB in 2001. There’s a host of other new contributors, too - sorry for not listing you all, guys, but please don’t think you aren’t appreciated.

It’s a good time to get involved! Ever been frustrated by something GDB did, or didn’t do? Drop us a line at gdb@sourceware.org, file a bug (thanks to Tom Tromey, we now have our own bugzilla instance instead of GNATS), or lend a hand.

One recently recurring topic is source control. Yes, GDB is still in CVS. The reason is modules - nothing else has modules that work quite like CVS’s, and the “src” repository relies heavily on them. It’s going to take some changes to the top level build scripts, some developer training, and some twisty management scripts to get us into another source control system. But I’m holding out hope it will happen in the next year.

When will you see these neat things in Debian, you ask? Pretty soon. GDB 7 will hopefully come out this Fall, and I think it’s ready to go into Debian unstable at this point. I’ve had packages of the Python scripting branch in experimental for a while; not all of that branch has been merged to trunk yet, but enough has to be handy. Once this is in unstable, hopefully we can make any needed tweaks to the libstdc++ packaging to help GDB find the new pretty-printers that ship with it.

My trusty Shuttle barebones desktop finally gave up the ghost on Monday. I got back from a week’s vacation, sat down to read email, and after about an hour it shut itself off and refused to turn on again (fans spin up, lights come on, then everything shuts off again within five seconds). My best guess is that the power supply is shot. I could probably have gotten a replacement, but the thing is four years old so this seemed like an auspicious moment to replace it.

I mentioned this died in the middle of the work day. So I needed to replace it promptly. Rather than building from components as I used to, I drove down to the local Best Buy and looked around. They had a plausible HP Phenom quad-core system, and one Dell Core i7 based system. Literally one - they don’t stock any other i7 based systems here, and this one was discontinued and sold out. So I got a nice discount on the floor model.

The system is a Dell Studio XPS 435MT (since it’s discontinued, I can’t find a shiny product page to link to, but here’s the system documentation - other Studio XPS systems like the Studio XPS 435 should be similar).

Installing Linux

I didn’t do a real install on it. Instead I left the internal 640GB drive with Windows, and transplanted the two drives of my Shuttle’s RAID array into the new case. There’s only one empty 3.5″ bay; I propped the second drive in the empty 5.25″ bay and ordered a mounting bracket from Newegg. While I was there I ordered 6G RAM to go with the 4G included (which involved pulling out one of the 1G sticks).

Getting it to boot in this setup took me several tries. I had a working grub configuration on what was now /dev/sdc ((hd2) in grub speak). But I needed to put the MBR on (hd0). First I fixed up drive names in /etc/fstab and /boot/grub/menu.lst using System Rescue CD. The live CD almost but not quite handles LVM on top of RAID; it starts them in the wrong order. So after booting, vgscan and vgchange -ay, and away we go.

Also from the rescue CD, I needed to build or install a new kernel - I didn’t have the e1000e network driver or the ata_piix SATA driver enabled in the Shuttle’s kernel. It was easier to build a new kernel than to transfer the standard kernel .deb onto the system, so I did that. I also regenerated the initrd (see previous rant) about how hard initrd makes this sort of transplant).

Several times I got dumped into the initrd while I was getting this right and my USB keyboard didn’t work. I never did figure out why; the right modules were there, but perhaps they aren’t loaded early enough.

Unfortunately grub’s “setup” command runs “install” with “/boot/grub/menu.lst” instead of “(hd2)/boot/grub/menu.lst”. Eventually I figured out what I needed to add, and ran first setup and then a modified version of the install command that setup echoed. And now it booted.

Graphics acceleration

X was very, very slow when the system first came up. Eventually I figured out that only current git kernels had the PCI IDs listed for the Radeon RV770 - 2.6.29 was too old. An update brought working DRM, though see below for more about X. I haven’t used X without 2D acceleration since I worked on the Power Mac controlfb driver; I’d forgotten how disgustingly slow it is!

Fans, temperature sensors, and power management

Windows utilities like SpeedFan and Everest report three internal fans. Everest labels them as CPU, Chassis, and Power Supply. Linux lm-sensors detects the IT8720F chip but only shows two fans; the power supply fan is presumably on the same sensor chip, but Linux is detecting it as not present for some reason. I’m going to come back to this.

There are two temperature sensors (three show up, but -55C is pretty unlikely, so I presume it’s not connected). The motherboard sensor typically reads 42C in Windows and the “Aux” sensor (which I haven’t located) reads about 74C in Windows… and 84C in Linux.

The most likely source of heat in the case is the GPU (Radeon HD 4850, RV770 chip). After rebooting from Linux to Windows, Everest showed three additional temperature sensors on the GPU, which were falling rapidly. So Linux seemed to be driving the GPU unnecessarily hard. This seems to be a known issue; here’s some detailed information about Radeon power management, from Alex Deucher (written just last week).

So until DRM KMS (kernel mode switching) lands, or at least until X.org 6.13, I get to either ignore the extra heat and noise or else run the ATI fglrx driver. I reluctantly downgraded from 2.6.30-rc9 to Debian’s 2.6.29 kernels and installed fglrx. I had to remove fglrx-atieventsd to stop my disk churning although unfortunately I can’t reproduce the problem now (bug filed anyway, in hopes someone recognizes it).

Now the temperatures are stable, although fglrx leaves scary kernel messages suggesting its memory management is buggered (#524871). *sigh*.

Also, those temperature sensors are apparently on a standard i2c bus on the Radeon. It looks like radeonfb is capable of exposing the bus to userland i2c tools or lm-sensors modules, but only for chips it supports - which the RV770 definitely isn’t. Hopefully this is another thing which can be fixed after DRM KMS lands. The current version only seems to expose the DDC devices though maybe I’m misreading it.

Sound almost works

This system uses hda-intel. Sound does work - but any channel comes out all speakers! I haven’t tracked down the problem yet. I’m used to having trouble with 5.1 surround sound and ALSA, but I’ve never had trouble with stereo before. It’s probably a configuration problem.

Disk temperature

The hard drive in the spare 5.25″ bay (with or without the mounting bracket) gets very warm - 49C instead of the 41C/42C my other drives run at. So I went looking for ways to keep it cool. I ended up with a clever plan: this machine spends most of its time in Linux, and there’s nothing Linux needs on that drive. So can I keep it spun down?

The answer is “yes, with effort”. With any recent kernel, hdparm can spin down SATA hard drives (hdparm -y /dev/sda). It spun right back up again with nasty messages in the kernel log, though. When searching for similar errors on google I discovered that probing SMART data - including the temperature sensor in the drive - would cause the platters to spin up (irritating!). I eventually tracked this to hddtemp checking the temperature of both /dev/sda and /dev/sg0, and filed bug #531849, which has an easy workaround. Once hddtemp is configured to only check /dev/sda, it knows not to spin up the drive.

Next I added this to /etc/rc.local:

# Spin down the drive with only Windows on it, and make it spin down
# again if idle.
hdparm -S120 /dev/sda
hdparm -y /dev/sda

That doesn’t work, for some reason; it does run, but either the disk doesn’t spin down or else something spins it back up shortly afterwards. So right now I have the hdparm commands in my $HOME/.xsession instead, and now it stays spun down all the time.

Inspired by a question from Zack, I’ve published the hacked-up script I use to read LiveJournal entries using liferea: ljfriendfeed-local.

I’m trying to reach Alan Curry, pacman@TheWorld.com, about some GDB work he did last year. That address now bounces. He’s a regular Debian bug submitter, though none of his bugs have a newer address for him listed. So the best idea I could come up with was to ask the lazyweb - Alan, are you out there? Or does anyone else reading Planet Debian know how to reach him? Leave a comment or send me email if you do. Thanks.

Updated: Yes, I did try Google. Thanks to all those who pointed out world.std.com (alias for TheWorld.com), and clss.net (most recently used in 2005; no active nameservers); sorry I forgot to mention that.

An anonymous reader asked for a Debian package of the font I recently posted about. One is now available here should you want it. I’m not planning to upload this to the archive, although if anyone wants to add it to an existing fonts package you are more than welcome to.

I am considering merging this with GNU Unifont, so that the many missing characters show up looking out-of-place and non-matching instead of completely blank.

Also, see xterm bug 456236, which should be fixed in unstable soon. Or mostly fixed, anyway; it still seems to glitch once in a while but it’s harder to reproduce now.

I have always had some trouble with my eyes. One bizarre side effect of this is that switching fonts is very unpleasant. I recently got fed up with fighting things to live in the stone age of pre-Unicode fonts and locales and started hunting for a new console font. First I looked for nice new ones already designed for Unicode, with a selection of accented characters and so forth. The nicest I found was Terminus, but I’ve had trouble adjusting to it. So eventually I decided to stick with the font I’d been using - at least for now.

I have been using the same font for my consoles since before 1997. This is, or at least was, the default Linux console font back when I did all my work on a VT on Linux/PPC: /usr/share/consolefonts/default8x16.psf.gz. When I started using X routinely I missed it quite a lot. Mostly that was because I used BitchX, and BitchX is written for an IBM437 code page set of line drawing characters. So I opened up that console font in xmbdfed and converted it to a PCF file. If you have Debian’s BitchX package installed, you’ve got a copy of this work in /usr/share/fonts/X11/misc/default8x16.pcf.gz.

So I’ve done the same trick again. Here’s the steps I followed:

• Open the PCF font file in FontForge.
• Choose Add Enconding Name… from the Encoding menu.
• Enter IBM437, which is the iconv name for the traditional PC ROM character set.
• Choose Force Encoding and select IBM437.
• Choose Reencode and select ISO 10646 (BMP).
• Choose Generate Fonts… and save it as a BDF file, since PCF wasn’t an option.
• Create a fonts.dir file by running mkfontdir.
• Add the directory to the font path using xset +fp.
• Attempt to start an xterm.

Actually, after all that I was having trouble with the font’s name and eventually getting scary resource errors from X. I opened the font in gbdfed and saved it, which added some attributes and modified the bitmaps. Then I tried again and it worked.

And now, I can use a UTF-8 locale and gcc’s error messages come out with `foo’ (”angled” quotes) instead of display corruption!

The resulting font is uni8×16, if anyone else would like it. I have switched to using it for all my terminals other than the one I run BitchX in; BitchX still needs the IBM437 version in order to display correctly.

Of course, as soon as I finished this Paul Brook informed me that Konsole includes basically the same sort of font. I do not know the origins of KDE’s console8×16, but I do know that it’s a slightly different font (the @ has a smaller center, for example). And I know that I’m mighty annoyed it didn’t get added to my font path correctly when I installed konsole ages ago, since I might otherwise have found it!

The gloom from my last entry has more or less lifted. To refresh, my desktop died shortly after a lightning storm; no definite cause and effect relationship, but a strong correlation. I shut it down, and after the next reboot I got scary errors all over the place, starting with this disclaimer:

HARDWARE ERROR. This is *NOT* a software problem!
Please contact your hardware vendor

Now, that may be true, or it may not be. Certainly the disclaimer is sensible, since machine checks are usually caused by failing components. And it persisted for most of a day every time I tried to boot the machine, but there was quickly a new addition: different scary errors on ata1. And it got rapidly worse. On Friday morning, having let the machine cool and relax overnight, I brought it up on a rescue CD and could get at /boot (a separate partition), but not at /etc (from inside my RAID/LVM setup). I went out and bought a new SATA drive and attempted to recover /boot onto it, but I had waited too long; at this point I could still get at the partition table, but not the contents of /boot.

Some things to note here.

• As far as I know, you still need a separate /boot if your root uses RAID + LVM.
• Therefore your /boot is obviously not backed up by your carefully constructed RAID.
• But that’s fine because all that really mattered was your kernels, and you have them still lying around on your mirrored disk. Right?
• Of course, right. But dynamically generated initrds, while technically awesome, are awkward to recreate from a rescue CD. Especially a 32-bit rescue CD when your root filesystem is all 64-bit.
• It can’t be hard to find a rescue CD with a 64-bit kernel, right? No, not right either. I did eventually find a 50MB Gentoo install CD that worked serviceably well.
• Your laptop has a convenient CD burner, right? No, not right, turns out it doesn’t write CDs after all. Oh well, Windows file sharing plus the wireless plus the Mac Mini in the other room… there you go.
• You didn’t need anything else from /boot, right? Like your carefully constructed voodoo incantation to boot Windows… whoops… hopefully you remember how that worked…

So, after some futzing around I was able to boot the Gentoo rescue CD, partition the new hard drive, mount the remaining disk in degraded mode, reinstall the handy Debian packages of my kernels, have it magically recreate ramdisks, and then reinstall grub. By the way, --no-floppy obviously means something different to grub than it does to me, as most of the reinstalling grub process was still spent waiting for my non-existant fd0 to time out. Then I added the new disk to the array, let it reconstruct for two hours, and rebooted.

Lo and behold… no more machine checks! I am a happy camper. Despite the tribulations above I came out remarkably well, since I seem to have lost nothing but my grub menu.lst file. And now I have rdiff-backup configured to back up /boot onto the array in case I ever need to do this again.

My desktop seems to have melted something precious in this morning’s lightning storm, despite the recent and rather hefty UPS sitting behind it. I am, to say the least, annoyed. It was a nice self-built Shuttle system from 2005, but now all it is is a machine check generator:

HARDWARE ERROR. This is *NOT* a software problem!
Please contact your hardware vendor
CPU 0 4 northbridge TSC 6eb92ec370b
  Northbridge Watchdog error
       bit57 = processor context corrupt
       bit61 = error uncorrected
  bus error 'generic participation, request timed out
      generic error mem transaction
      generic access, level generic'
STATUS b200000000070f0f MCGSTATUS 4

It’s not even bad RAM or a bad RAM socket, the way most memory errors seem to be; I had two DIMMs in two sockets so I was able to try for both possibilities. Unless it fried both somehow, of course. It seems to boot a little further when left cold for a while, and not as far just after crashing. I am inclined to suspect the motherboard.

I think I’m going to have to write it off as a loss. Which means I need a replacement. I can get by with just my laptop for a little while, but not indefinitely.

Are there any good options for pre-built Linux systems yet? I was somewhat underwhelmed by Dell’s. I usually build from parts, so looking at pre-built systems always has a certain amount of sticker shock. My main concerns are CPU / memory performance (a lot of compiling here), room for at least three hard drives, and reasonably quiet.

Otherwise it’s off to newegg again…

Or I could always buy an equivalent computer to my current one, for a fraction of the original price, at a local consumer electronics megastore! How’s that for a different idea.

I was checking idly this morning whether there was a 64-bit version of Macromedia Flash yet (there isn’t but they claim to be working on one; yes, I do know about gnash). While searching I learned about nspluginwrapper, which was quite gratifying; I have always thought that ought to be possible, but the one time I took a stab at it I got hopelessly lost in the plugin API and had to be evacuated by helicopter.

If nspluginwrapper works, it will kill two of the reasons I run a 32-bit firefox on my amd64 desktop (flash and the Acrobat plugin). I have firefox working quite nicely using schroot, but there are some inconveniences - like having to mount my sshfs filesystems once outside the chroot where I want them and once inside the chroot so that I can access them in firefox. And having to keep a ton of stuff installed in the chroot, too.

Of course that leaves the one other gaping omission: there’s no official 64-bit Sun Java browser plugins yet. Blackdown had a 64-bit browser plugin, but there were a number of reports of its crashiness. And there was a lot of ambiguity in the web pages I found about whether IBM’s JRE had one or not. So I fought with IBM’s registration today (their database server must’ve been horked this morning…) and downloaded their Jave 5 JRE. Well, it doesn’t. No plugin except for IA32 and PPC32. I’m guessing from the links in the documentation that the guts of their browser plugin still come from Sun. Sun claims that JDK 1.7 will have such a plugin, but no one seems to have seen it yet.

I think the only realistic way this will work out is to try building one from OpenJDK, and that’s more work than I have time for this evening. Back to the chroot for now…

The machine (formerly) hosting this blog was a dedicated server. I’ve got nothing bad to say about the hosting company, but it had the usual pitfalls of dedicated hosting. It was very expensive, which is why I split the service with two friends. We underutilized it. They didn’t provide backup services, so I ran my own using rdiff-backup - I won’t tell you how long I ran it before I bothered to set that up, though. And when something went wrong it could take quite a while to get the machine fixed and running again. Fortunately that only happened twice in all the time we were there. Anyway, our contract with them was up in February, and we decided not to renew. Since all of us were insanely busy at the time we decided to pay the higher monthly rate for three months to give us some time to migrate off. Which of course I left to the last minute.

This blog is one of the first services I’ve moved to my new hosting at Slicehost. I don’t have a lot to say about these guys yet, since I haven’t had any trouble. The only downside was catching them at a somewhat turbulent and resource-starved time as they dealt with a wait list for orders, so it took a month for me to get one. They’re working on that problem, though. Towards the end I was checking every two days or so how long it said I’d have to wait. Note to everyone, progress bars that don’t progress are just infuriating. But I happened to check one day during the five minute window between when they sent me confirmation and when my mail client checked for new mail, and poof there it was.

This is virtual hosting - Etch on Xen - thus the “slice” naming. They’re a heavy on automation shop, which fits right in with my style. And there is something so viscerally awesome about entering your credit card number, clicking OK, having their software allocate and create a virtualized disk, bootstrapping Etch onto it, and showing you its IP and root password. It doesn’t get much better than that. So far I’m happy.

I took the opportunity to move to WordPress, away from my old hacked up Blosxom installation. WordPress imported entries easily, although I had to manually fudge the RSS file to prevent it from inserting a log of bogus <br> tags that I didn’t want in the middle of my paragraphs. I probably could have gotten comments imported too, but I didn’t bother - sorry folks, well, it’s not like there were a whole lot of them. The Debian WordPress packages work fine, although the configuration is a bit messy; my biggest complaints were the centralized upload and themes directories so that I had to mess around with symlinks in /usr to get everything going.

Maybe I’ll actually use it more :-) I hacked up the GUIDs a bit by hand, so this shouldn’t flood planet, but here’s hoping.

Next step: migrating email service to the new system. Always a joy.