Northern Colorado Linux Blogs

From: Sean Reifschneider

Last week my Raspberry Pi finally arrived... I have a lot of stuff going on right now, so I loaned it to my friend Stephen Warren who hasn't received his yet (scheduled 3 weeks out or so) until his comes in.

His project for the Pi is to do environmental sensors around his house. For that he needed an I2C driver to talk to the sensors. There was already an I2C driver out there, but it was ... not optimal.

So he wrote a new one this weekend:

Github Raspberry Pi Kernel: I2C Driver

Then today he had another person contact him that they were working on an I2C driver. Stephen had done a google search to try to find I2C drivers before he started on his own, so I figured some more pointers in Google would be good.

It's a fun looking device, and seemed to be real easy to get booting. I can't wait until they've cleared up all the sales back-log.

As I noted recently ( Fedora 17 and updates-testing ) In the run up to Fedora 17 release, the updates-testing repo has been disabled. If you are wanting to use ‘yum distro-sync’ to sync your pre-release machine up to stable updates/base repo packages only, there’s a few gotchas you might run into:

• First the fftw package decided to reorg how it’s subpackages were setup. (IMHO not a great idea right before release). The version in updates-testing is one libs package, but the base version is a bunch more subpackages.
• Next you may run into colord-gtk issues. colord grew a new colord-gtk subpackage (IMHO not a great change right before a release either).

To work around these, try:

yum shell
remove colord-gtk
remove fftw-libs-double fftw-libs-long fftw-libs-quad fftw-libs-single
downgrade colord
downgrade fftw-libs
distro-sync


I’ll try and update this post with any more of these I find… feedback welcome.

Xfce 4.10 was released to the world 2012-04-28 and Fedora rawhide was updated in the days right after that. (Thanks Christoph!)

Our Fedora 18 feature has been approved: https://fedoraproject.org/wiki/Features/Xfce-4.10

My side repo for existing Fedora releases has been updated also to 4.10 final, and I have also built for Fedora 16 as well now. See:
http://repos.fedorapeople.org/repos/kevin/xfce-4.10 for a repo file for your /etc/yum.conf.d/ directory.

Lots of nice new little bugfixes and additions in Xfce 4.10. Give it a try today.

If you have a Fedora 17 install and updated recently (since saturday), you may or may not have noticed that the updates-testing repo is now disabled.

It’s enabled for testing during most of the pre-release cycle of Fedora, but as we get close to release time (and we are), updates-testing is disabled.

So, if you are a tester/qa person/wanting to help test in the run up to release you may wish to re-enable updates-testing (a simple: ‘su -c ‘yum-config-manager –enable updates-testing” should do). You can then update and use the excellent ‘fedora-easy-karma’ tool to provide testing feedback.

If you are not interested in testing things out, and want to just have your install more or less as it will be when it’s released on the 22nd, you may want to run a ‘yum distro-sync’. This should downgrade your packages to the base versions from any you have installed from updates-testing. If you don’t do this you may run into odd problems about packages not available or mismatches when trying to install 32bit packages on a 64bit install or the like.

From: Sean Reifschneider

A year ago I started trying to use a desktop in addition to the Thinkpad laptop that had been my primary computer (in different models) over the previous decade. I really like the Thinkpad keyboard, and can't live without the stick-mouse...

So to make the transition as painless as possible, I got a "ThinkPad USB Keyboard with TrackPoint" (FRU: 55Y9053, Model No: SK-8855).

However, this had a well known issue with the caps key not responding quickly to being lifted. Like the key was stuck down for around another 0.2 seconds after you released it. If you type less than 60WPM it's not a problem, but above that it starts getting maddening...

I decided to just try buying another one, because the one I had was probably 4 years old (I just hadn't used it seriously until last year).

The new one, which is labeled as "REV: R01" (compared to the previous "R00"), completely fixes that problem.

Lenovo had released a driver fix for Windows that resolved this, but that doesn't help me, I'm on Linux.

Searching around the net, you still see a lot of reviews that say that this is a problem, which is why I wanted to get this post out there. If you get a R01 revision, it seems like this problem is fixed.

You can't believe how happy I am about this!

NOTE: This keyboard does not have a Touchpad, it only has the stick. Which I am happy with, I disable the touchpad on my laptop. There is a model that has a touchpad, but it does not have the Windows key (which my window manager uses extensively).

From: Sean Reifschneider

The AT&T Global Network Client, on a 12.04 Ubuntu Precise 64-bit system, is a 32-bit app. It initially doesn't work because of missing libraries. You can see this by running:

$ cd /opt/agns/bin
$ ldd NetVPN
[...]
libssl.so.4 => not found
libcrypt.so.4 => not found
[...]

To fix this, it needed a set of symlinks like this:

$ cd /lib/i386-linux-gnu
$ sudo ln -s libcrypto.so.0.9.8 libcrypto.so.4
$ sudo ln -s libssl.so.0.9.8 libssl.so.4

Now, just restart the agnclient and agnclientd processes:

$ sudo pkill -9 agnclient
$ sudo pkill -9 agnclientd
$ ps awwlx | grep agnclient
[shouldn't display any processes except maybe grep]
$ sudo /etc/init.d/agnclientd stop
$ sudo /etc/init.d/agnclientd start

Then starting the agnclient from the menu worked.

Kudos for Bill Tucker for giving me this opportunity to fix this. :-)

Xfce 4.10 pre1 was released on 2012-04-01 (no joke!

In the last week I have built and landed it in Fedora rawhide, including rebuilding all the plugins against the new libraries. The only application left I know of that has issues is xfburn. I’ll be working on that in the next few days, in the mean time you will probibly want to remove it so you can update the rest of the stack.

I’ve also setup a Fedora 17 side repo with all these packages built. You can find it at:

http://repos.fedorapeople.org/repos/kevin/xfce-4.10/

I’m running it here and it seems to be pretty stable so far. Of course it’s only a pre1, so there could well be problems. Save your work often. Please report bugs on these packages to the fedora-xfce list or directly to me, to avoid confusion with the normal 4.8 packages in Fedora 17.

Comments and corrections and welcome for our Fedora 18 Feature page: Xfce-4.10

I got a little email note from the Red Hat HR folks today congratulating me on my 1 year anniversary with Red Hat. It really has gone by so quickly, so much to do, so much to learn, so much to work on. Hopefully I’ve made a difference and kept things running smoothly in Fedora Infrastructure for everyone. I hope it will continue to be a fun and satisfying trip in the years to come.

A number of folks have asked about this in #fedora over the last few months, so I thought I would do a quick blog post about rc.local in the systemd world.

/etc/rc.d/rc.local is no longer shipped, but it still works fine once you set it up correctly:

Create /etc/rc.d/rc.local as root.
Add a “#!/bin/sh” to the top of it.
chmod 755 /etc/rc.d/rc.local

Once you do those things, systemd will see the file exists, is executable, is a shell script and will run it. No further action needed.
I do hope systemd maintainers keep this functionality around as it’s sometimes handy to run things in a simple script rather than create a new unit file for it.

From: Sean Reifschneider

We've been upgrading our mirrors.tummy.com server to add more storage space, and the new system got accidentally set up as a Linux software RAID-10 array rather than RAID-5. Since most of our use is reads, and we want capacity, RAID-5 is preferable.

So I decided to try "growing" the array from RAID-10 to RAID-5. Read on for more details about this operation. (read more)

This all started with my employer giving me a new laptop. They don't want the possible loss of productivity caused by an hardware failure, so they replace machines after four years of operation. Anyway, that new machine lead me to try Debian as I blogged a few months ago. Debian testing, unfortunately, turned out to be too unstable (ironic, eh?) changing too much stuff that I eventually felt like they were pulling the chair I was sitting on, so I gave up. I returned to Ubuntu. Sadly, Unity was unusable for me and I encountered the same problem Linus Torvalds did with GNOME 3 (see here). He settled on XFCE, I didn't. I tried a lot of stuff as my desktop, finding them all unsuitable for my needs. I planned to write a detailed report on this stuff, with the purpose of giving other suggestions about the workarounds I found, and (maybe) push the authors to fix some of the bugs I stepped into (most of them had already been reported, some long time ago). Then I read this, thought that maybe my post would have been only complaints. And I left for a busy conference.

When I was back, I found my notes covered in mouse's poop (yes, a mouse entered my cubicle, should I sue my employer? :-) Maybe that was a message...
So I completely gave up on the idea of posting all the details, and will post only the bottom line. What I did not like in Unity, GNOME 3 and KDE is that they shifted their focus from the desktop to the tablet/phone. Heck, Shuttleworth even announced a version of Ubuntu running on a phone! I mean, the idea is nice, but my current needs are different. I don't even own a tablet nor a smartphone, they would be greatly underspecified for what I do (the only feature I would use is the GPS, which by the way I have as a standalone device). My computer at work is a 13" laptop, but I use it almost only on a docking station hooked to two 24" screens, on top of which I use at least 4 other virtual desktop (sometimes more). I need it that way. Unity and GNOME 3 (both Gnome Shell and Fallback Mode) simply are unable to provide an useful user experience in that setup. GNOME 2 was fine. XFCE barely makes it, but it has a couple of deal-breaking bugs (well known and never fixed). KDE 4 is almost ok (and it is what I settled for), but it has a very nasty issue: if you "save" your monitor configuration (and I do, since I don't want to tell it that I have two 24" monitors each time I boot), you are stuck with it: if you turn the machine off, decide that you need to carry your laptop with you, take it out of the docking station and (later) turn it on, you'll always get the laptop's screen in black, because it stubbornly search for the monitors and does not fallback to the only actual screen it has connected (the laptop screen). The workaround I used was to create another user, for the only purpose of not using the two monitors and being able to log in while undocked. What a pain! But Unity, Gnome Shell v3, Gnome Fallback and XFCE were worse...

Now I realize that the problem is the desktop. Everybody wants a fancy tablet or smartphone and open (or closed, for what is worth) source developers fly there. Even KDE, see for example the Stark tablet (just renamed Vivaldi). Nobody to blame, everybody's free to do what they please. But it's sad: the year of the Linux Desktop will never come. Maybe there will be the year of the Linux Phone (if you count Android that's already happened) or the Linux Tablet. But that's not what I need today. My hopes are currently on http://mate-desktop.org/ but just for myself, this doesn't look like a project that could go mainstream.

UPDATE: a couple of pages in a review about GNOME3 which exactly explain why I don't like it

[en] I just learned from RadioVerda that GoogleTranslate can now translate Esperanto. This is great news. Google Translate has had the ability for a long time to be able to detect whether a page or a piece of text was written in Esperanto, but i always found it a bit strange (and sad) that it [...]

That is all.

I've been playing with Juju to deploy servers in the cloud. This means that Juju has come up in regular conversations with my devops friends (admittedly, I'm more dev than ops...). I had lunch with some devops folks from an unnamed startup while at PyCon, and I mentioned it to them. They use Ubuntu in the cloud very heavily, but the systems they use for deployments, etc. were running OS X (as glorified ssh terminals)1.

This wasn't the first time I'd had a conversation about Juju result in questions about its compatibility with OS X. After that lunch discussion, I decided it'd be my last. I sat down with two devops guys with those super expensive netbooksMacbook Air laptops and sorted out how to get Juju working on OS X.

First, install Zookeeper. The devops guys used Homebrew to do that. It looks like there's also a source distribution at the Zookeeper site, but I'm not familiar enough with OS X to know about anything outside of Homebrew.

brew install zookeeper

Once Zookeeper is installed, download Juju and go into the new branch.

bzr branch lp:juju
cd juju

Install juju dependencies2.

pip install pyOpenSSL pyyaml twisted txaws txzookeeper zkpython

Once that's complete, install juju.

python setup.py install

Now you've got Juju running on OS X. I suggest starting with the tutorial, and then moving on to writing your own charms.

1 The Mac builds of Precise nightly are most excellent, so if you've got a Mac, running Ubuntu on it without all the pain that usually comes with running Linux on a Mac. I highly recommend it.

2 I've submitted a patch to Juju so that juju knows about its own dependencies. As soon as that lands, this step can be skipped.

While on my weekly phone call with the rest of the Ubuntu One Web and Mobile team, my wife walked into my office and gestured for me to mute the phone so she could talk to me.

"Hey, I'm uploading my portfolio to Ubuntu One and it's taking a long time. It's a lot of files. Ubuntu One can handle a lot of files, right?"

"Wait, what?"

"I'm meeting someone today to talk about doing some art, and I need to send my portfolio to her. I'm putting it in Ubuntu One. Is that okay?"

"Absolutely! If you want, you can then put it on your tablet too, so you can have it mobile. Just install the app"

...an hour goes by...

"My tablet can't read my portfolio, so I uploaded the individual pictures, and now I have my portfolio on my tablet."

The original "portfolio" was a zip file, and her tablet didn't know how to open zip files. She soon had her phone and tablet syncing their pictures to Ubuntu One.

Sometimes I spend so much time working on Ubuntu One that I forget that I'm contributing to making cloud storage accessible to "normal people".1 Moriah doesn't care about filesystems and protocols and desktop environments. She just wants the artwork she did on the computer to be available to her on her tablet when she goes to meet someone at a coffee shop. She wants to be able to easily get to her phone's pictures on her computer (she's always taking pictures of things that she finds inspiring, to refer to later). I'm glad to be part of the movement to help facilitate that.

The Ubuntu One team has a lot of work ahead of us, but little encouragements like this go a long way. I'm now being accosted with feature requests from my wife, which is a good problem to have.

As a final note, sure, there's Dropbox and iCloud and all sorts of things. Moriah was familiar with Ubuntu, so Ubuntu One became the obvious choice for her. In fact, she had iCloud enabled on her iPhone and they were causing side-effects that caused her to curse profusely on a regular basis.

1 I use the term "normal people" because it's a running joke in our house that geeks are not normal. We do very peculiar things. This has become the subject of her weekly webcomic Citation Neeeded.

I've just cut and released Pariah 0.2, a reusable Django app for working with web comics. It's a small piece of code, but focuses on doing one thing and doing it well. You can download the source on Github, or you can install it from pypi with pip install django-pariah.

It's still a very early-stage release, but is now in production use on Citation Needed, which was previously using Comicpress, which is both terrible software and also apparently no longer maintained.

Why call it Pariah? Because my wife, for whom this software was written (the author of Citation Needed), is named Moriah. My name is Paul. If we were a celebrities, the tabloids would refer to us as "Pariah", which I find pretty fitting.

A few months ago, I traded my XO laptop for a CR-48 laptop. I've fallen in love with this little gadget. It's simplicity is what appealed most to me originally. It's just a browser. That's it. If it's not a webapp, it doesn't matter. What hooked me, however, is the new perspective on computing that I wouldn't have gotten had I kept working with the same tools that I always work with.

I've been an Ubuntu user for a long time. I've recently also started using OS X for tasks that require a Mac. I've always considered myself a pretty forward thinking guy. I like the direction that Ubuntu is going, including the polarizing Unity project¹. Using ChromeOS on the CR-48 showed me that I'm not forward-thinking enough.

Before ChromeOS, webapps were this cute thing that I could use when I didn't have a native app that did the job much better. Now I look at webapps as the default, and see my usage of native applications as a failure of the internet-makers to fill the need (this failure has sparked quite a fun little side project that I hope to do something with soon).

Sure, webapp performance will never beat native performance, but there's more to a User Experience than performance. They may not cover all cases. I suspect, though, that all those edge cases are usually people that want to "use computers" rather than just get things done.

Screw Android. When can I have a ChromeOS phone?

¹Unfortunately, I've recently "outgrown" Unity, which I think is okay. The addition of the second launcher on dual monitor screens did it. I have no desire to have a second launcher right at the seams of my two monitors.

Update (6 Mar 2012): thumper and I chatted about the Unity problem I was having on my dual monitor setup, and we've come to a solution that will work for both of us. I'm looking forward to going back to Unity, because Xfce and LXDE and all the other minimalist environments don't have nearly the polish that Unity has, and Ubuntu has spoiled my for so long that I don't feel like polishing anything myself.

When the Django/Rails revolution hit the scene years ago, I was working for a small development shop building something similar to the Ellington product that Django came from (or something like that). We were building a CMS using PHP for a small newspaper in California¹. The day we rolled out prettified URLs in our completely re-written version was the day we started to see our sites climb the Google ladder.

Django and Rails and CakePHP and all those other has-been-hipster frameworks all adopted prettified URLs and everyone rejoiced. Now you could guess URLs like twitter.com/rockstar_ or facebook.com/yourmom; you could actually read a URL out loud to someone and it wouldn't be that hard to replicate by hand somewhere else (I witnessed this happen in a college dorm a few years ago).

There's only one problem: Those URLs don't really match with the HTML5 APIs.

HTML5's History keeps track of state (and if you're not using it while building web apps, I hate you). The mismatch comes from the fact that History keeps track of state in key/value pairs. So, if you were looking at a Contacts app, and you clicked on a Contact, it might do something like this:

history.pushState(
    {contact: 'ipfreehly', view: 'contact'},
    'I.P. Freehly', '/contact/ipfreehly');

The third argument to pushState is a new url (with the same origin). How do you take those two key/value pairs and update a pretty URL accordingly? If you can use querystrings, it's easy, since querystrings are just key/value pairs as well.

The only sane solution (because reproducing URL building logic in the client-side is not sane) that I've been able to come up with is to make sure the href attributes of anchor tags are set correctly, and to just update the url to be the same as the href of the link. But what about actions that are not a:link, like buttons? In that case, we use any number of other attrs, or create our own data attribute for it.

I'd be curious to know if there are any cleaner solutions for the mismatch between pretty URLs and History.

¹ The software was called "Bakomatic" and from what I hear, it is no longer being maintained or implemented. It was funded by the Bakersfield Californian (thus the 'bako' name). It was a good experience to work in publishing, even if it was only for about 18 months.

Pre-cursor note (just in case): This isn't an official Ubuntu One post. For those, go to the Ubuntu One blog. I just work for them, and I have my own opinions.

Today I submitted and landed the branch that will kill the web UI for Ubuntu One notes. Users can still use Tomboy to sync, but you won't be able to access them from the web. I know a vocal minority is upset about this. To those, it's unfortunate. I'm sorry. However, I think the decision was a great one.

The work was done in two branches. The first removed the links on the website and python views and entries in urls.py (it's no secret that our web app uses Django). The second one removes everything else: css, javascript, javascript dependencies, and the rest of the python files. Here's the diffstat of that second branch: 103735 lines (+3/-103093) 92 files modified

Think about that: more than 100k lines of code removed there. Sure, we didn't write some of that, but we sure as hell needed to maintain it (it's our attitude that we own the stack). Think about all the hidden bugs in there. Speaking as someone who knew that code intimately (it was one of my first tasks to re-factor it when I first came to Ubuntu One), there's also a lot of technical debt there¹.

If code is liability, that's a lot of liability.

We didn't see that much usage of the Web UI for Notes. One of the reasons I was re-factoring it a year ago was so that we could make it less fragile. It broke a lot. Automated tests probably would have caught that, but the fact that users didn't notice when it broke is indicative of a feature that isn't being used. Even after it was refactored, there were browser bugs that would occasionally surface, but no one seemed to notice.

If features are an asset, Notes on the web were an asset that few people valued.

I'm not a business guy, but it makes total sense for us to reduce the liability with little real cost. Our team is small (but quick and amazing), so the less time we spend maintaining unused features, the more time we can spend making things that are awesome (and we are doing amazing things; stay tuned).

The last thing that I want to address is this question that keeps getting asked in some form or another: Why not make it open source and have the community maintain it? Here's how I respond to that: You don't get bored of a piece of software and punt it to open source. You also can't think "Oh, the community will pick this up." They won't. You can't just assume there will be a "community" for it. If you can't hold someone responsible for its maintenance, you can't expect it to be maintained.

¹ Technical debt is a fact of life on any development team, and specifically one a team that moves as quickly and kicks as much ass as the Ubuntu One team. We try to mitigate it where we can, but it's a balancing act.

A week or two back we got a bug report from someone that they were downloading Fedora from download.fedora.redhat.com via https and the ssl cert didn’t match up. Of course it wouldn’t, as download.fedora.redhat.com has pointed to our dl.fedoraproject.org servers for at least 2.5 years now.

Since no one should be using this old dns entry now, I asked for it to be removed. I probibly should have made more noise at the time, but didn’t think too many people would still be using it. I did make a pass at fixing anything I could see still using it in the wiki.

In any case, it’s gone now. Please take a few minutes to go and look and see if you have any docs or wiki pages or notes or anything else that refers to this old entry, and update it. Please point them to ‘dl.fedoraproject.org’ (if you want to point directly to our master mirrors) or ‘download.fedoraproject.org’ (If you want it to point to a mirrormanager mirror that contains the content you seek).

Sorry if this causes any issues for anyone, but after 2.5 years, we should really be able to move on.

I have a little weekend-project that I've been working on since December. It started as tinkering, got really ambitious, and then settled down into something I was just playing with to learn some new things. Learn new things I have. This new project is built on Node.js using the Express and Socket.io libraries to make things a little easier. I wrote the account creation/authentication code, and backed it with MongoDB. However, when it got to honoring that login, things got a little tricky.

The first thing you'll read about is the MemoryStore object in Express. It's used to store sessions. Take that MemoryStore object and throw it out the window. Let's use Redis instead (separation of concerns FTW). First, install the connect-redis library.

npm install connect-redis

...and stick it in your app.js...

var express = require('express')
    ...
  , RedisStore = require('connect-redis')(express)
  , sessionStore = new RedisStore()

...

app.configure(function() {
    <snip>
    app.use(express.cookieParser());
    app.use(express.session({
        secret: 'YOURSOOPERSEKRITKEY',
        store: sessionStore
    }));
});

Now you should have a session property on each routes' request object. I only stick the user id in the session and then just look it up when I need it. Here's the rub: if you're using Socket.io, your socket doesn't have access to that cookie by default. That's a real bummer. The Socket.io wiki talks about this a bit, but it's not entirely clear on how to hook it to the existing session store, and who wants to re-implement code.

One that I really like about Node is that basically everything is middleware. All we do is set the authorization function to add the session object to the socket's handshakeData.

var app = express.createServer()
  , io = require('socket.io').listen(app)

io.configure(function() {
    io.set('authorization', function(data, callback) {
        if (data.headers.cookie) {
            var cookie = parseCookie(data.headers.cookie);
            sessionStore.get(cookie['connect.sid'], function(err, session) {
                if (err || !session) {
                    callback('Error', false);
                } else {
                    data.session = session;
                    callback(null, true);
                }
            });
        } else {
            callback('No cookie', false);
        }
    });
});

Now, when you add your socket connection handler, you can do this:

io.sockets.on('connection', function(socket) {
    var session = socket.handshake.session;
    ...
});

Hooray, now you can fetch session data in the web socket.

I'm planning on "hiring" my younger brother to do some work for me during his Spring Break in a few months. He's 16, and has some tech skills, but his school is not very technical and so he could use more opportunities to learn from folks. I thought I'd have him do some QA and some light programming (he's a fairly new but competent pythonista). I say "hire" because I want to add some incentive to learning enough to get a summer job.

Can you think of tasks that might be good for him to do? Maybe you have an open source project that you could mentor him on, and in so doing, gain a contributor. If you can think of something that you wished you had known when you started hacking, I'd appreciate if you'd please leave me a comment.

From: Sean Reifschneider

Earlier this month we switched our bandwidth accounting system to use new software. Previously we used a kernel module on a system, which was conceptually very cool. But with CentOS 4 going end-of-life, that kernel module doesn't work on CentOS 6 and I wanted to go a different route rather than fix the module.

So earlier this month, after extensive testing in parallel with our normal traffic stats collection software, I flipped the switch and started using the new software. Except that I forgot to re-enable the database update code, so it was only updating the graphs. I had disabled it to prevent double-accounting when running the old and new code in parallel during testing.

But, I log all those updates to syslog. I was able to pull out the logs of all those updates and replay them into the database. Logging of updates is a great safety net.

Which leads me to my next point. The simple code I had written to replay the logs, just was going to do one big commit at the end. I didn't think about how many updates we were talking about, over 2.5 million... I also wasn't writing out status, so I wasn't sure how far along it was. So I eventually killed it after an hour of running, and added code to commit every 1,000 updates and then write out a progress message.

With these changes, it ended up finishing after 18 minutes, giving me progress all the way along. Of course, I could have consolidated the data down in the program and done only a few updates, but 18 minutes was totally acceptable.

After seeing Christoph’s post fudcon 2012 todo list I thought that making a post fudcon TODO list was a great idea. Here’s mine (help welcomed!):

• Need to drop our staging git branch in puppet and merge any staging machine changes into the master branch
• Need to work on making out applications more ‘container’ like. ie, reduce the things they depend on and make it easier to spin up an env they run in
• Check out cloud tech and filesystems to see if any will meet our needs
• See if we can automate spinning up a application env with all needed integration testing machines
• Get two factor auth working for sudo for sysadmin-main and deploy. Continue adding types of auth and services once it’s solid
• Talk to the Board about our torrent seeder.
• Make sure the folks who want to work on reviving smolt have all the info and help they need
• Figure out how best to deploy the new community/tagger applications to production
• Find the time to add the nightly live compose to the rawhide/branched scripts so it happens in a totally automated way
• Finish migrating everything to RHEL6

If there’s something I told you I would help you with/do at Fudcon and you don’t see it above, please let me know and I will add it to the list.

Day 3 started out over at Macbride hall again.

Some of us Fedora Infrastructure folks got together to hash out any issues and projects we are concerned about in the coming weeks, months or year. We may have some big challenges ahead with ARM possibly coming in as a primary arch, which will stretch our storage and networks and building code. We are going to take a harder look at torrents and see if they are still worth providing to the community. The new community packages and tagging stuff needs to move into production. Lots of high bandwith talks with involved folks in all those areas.

After a nice little lunch, and a few more discussions it was time to head out to the airport. Matt Domsch was able to take 3 of us to the airport, which was nice. At the airport, there was time to sign gpg keys and mail them out from the keysigning the other day as well as make another pass of the cacert.org assurances that I needed to enter into the system. Hopefully flights home will go smoothly.

Overall it’s been a great fudcon. Lots of things seemed to get discussed and worked out. I heard very little where people were asking for something that they were not willing to champion themselves, which I think is excellent.

Thanks to all the folks who worked so hard to put on a great event and those who attended.

Saturday started off super early: Scheduled for a 8:30 start (although we didn’t actually start until 9am).

We had some intro and then pitching of barcamp sessions. Lots of nice sounding sessions. Then a short State of Fedora talk from our fearless Fedora Project leader.

There were tons of sessions (as you can see from looking at the wiki: Fudcon wiki page for saturday. I went to the session on making Fedora packaging easier. This was a talk to unveil a few new cool applications that can help folks with maintaining packages.

Pretty slick stuff: Community development packages and community dev tagger. It’s fast and could be very very handy for users.

Lunch was a delightful burger and beefy miracle. Good stuff.

After lunch I met up with some IRC folks to discuss some issues and ideas for improvements there. We also did some cacert.org assurances with a few folks (there’s another session just for that at 4pm, but since we had a few minutes we went ahead and did it to save time later).

Then on to the session on redoing comps and package organization given by Bill Notthingham. There are still lots of details to work out, but comps will split into distro groups (xfce, kde, base, webserver, etc) and categories (web browsers, editors, etc). distro groups will stay in the same place, but categories still need to be hashed out. Some info can come from the package tagging application I saw this morning and some from package maintainers. More info to come, but overall a good plan.

Next was the talk and show and tell on the rasberry pi arm box. It’s a pretty cute little device. I might have to pick up one after they are available.

Then, a lovely cacert event. I assured about 7 people and was in turn assured by 5 more. Then a gpg keysigning which had quite a few folks at it, but we managed to organize and run through checking fingerprints and id’s in short order.

Only a few minutes to drop things off at the hotel and then it was time for FUDPub over at the student center. Lots of food and fun. Finally some more talk back in the hotel where amusingly a wedding party was gathered.

tomorrow will be the last day and travel back home.

Fudcon day 1 started with having to get up at 7:30am to get ready for the 9am starting time. Thats 5:30am my time, so that’s an excuse for me being groggy this morning.
Had no problem getting to the venue and getting my badge and t-shirt. Then, after some logistics we started in on the first session of the day:

Fixing Staging in Fedora Infrastructure.

Some background: Currently we have a some ‘staging’ machines that are supposed to be copies of production instances that we can use to test and integrate new things with. We have a seperate git branch in puppet that handles the staging instances, which seems neat, but turns out to be an annoyance in several ways.

There was a lot of information and debate on what production, dev, staging, integration, or the meant. How we could setup puppet. If we could on demand make a staging instance or a subset of those. How process should work. How we could go from here.

We came up with a plan of attack and some things to consider:

• Drop the ‘staging’ git branch. Everything is in the same git repo. ie, all machines are production.
• Try and make our apps more able to be ‘containers’. Ie, reduce dependence on other parts of Infrastructure so things can be tested in containers easier.
• Look at ways to build containers or integration staging machines on the fly as needed.

After a quick lunch (man the wind was nasty to/from lunch), it was time for a 2 factor auth session.

We’ve been talking about finishing off yubikey as a true two factor authentication method in fedora infrastructure. We had a lot of good input here and hashed out a plan here too:

Short term:

• Fork linotp’s pam module to a new project. This would be just the pam module, and we would enhance it to require a valid ssl cert from the server it’s talking to before sending it anything, prompting for pin and pass and other enhancements.
• First target is going to be sudo for all sysadmin-main users.
• Create a CGI that the pam module can talk to and send auth info to and return ok, bad, broken
• CGI will likely run on fas servers so it can talk to fas and yubikey
• Some quick and dirty way to query pin

Longer term:

• FAS changes to store and set/reset pin
• ADD google auth or OATH to the CGI
• Increase parts thats are covered/where 2 factor is required

All in all some great sessions today. I think we have some lovely plans in fedora infrastructure, ready to dig in and get working in the coming days and weeks.

Thursday (Fudcon Blacksburg 2012 day 0) was a travel day for me. Had to get up at the unreasonable hour of 4:45 to catch the shuttle to the airport, then to my first flight to chicago. That went off fine, but unfortunately my plane from Chicago to Roanoke was delayed quite a while. ;(
On the plus side I got to hang out with 3 other fudcon bound folks and we did some chatting there in the airport. Finally we got a plane and got to Blacksburg, where the magic Spot van picked up about 15 of us and took us to the hotel.
The hallway track was well in progress there, and I ended up staying up later than I had thought I could talking to people I usually only see on IRC. Finally got to bed around midnight after catching up on some emails.

Looking forward to a great fudcon!

Next weekend is Fudcon Blacksberg 2012 and a number of Fedora Infrastructure folks should be there (Including me).

I’ve signed up to give/facilitate 3 workshops/hackfest sessions:

• Fedora Infrastructure: Revamping our staging server setup, then implemeting it. This will be hopefully friday and sunday. We can try and hash out a plan on friday and look at starting implementation or at least timeline for implementing things on sunday.
• Fedora Infrastructure: 2 factor authentication brainstorming. This will hopefully be saturday. I’d like to try and get full 2 factor auth setup for yubikeys and/or a clear plan to do so. Ideally we will setup a framework where we can also optionally use other 2factor setups (google authenticator, etc). I will also have some yubikeys with me to distribute
• Fedora Infrastructure: Getting started/apprentices/Getting involved. This will be a intro session for folks that would like to get involved with infrastructure. We can answer questions, try and find out what would be a good fit for people starting out and work on our apprentice program to try and make it easier for people to join us.

In addition to those, Toshio is going to be running some sessions on Python programming and software releases using our infrastructure as the example. Should be great stuff.

Of course as always in addition to the talks and hackfests, there’s a lively “hallway” track: A chance to meet up with folks and discuss things in a high bandwidth way. I hope to see lots of folks I know, and lots of new faces as well. Happy to discuss any fedora related issues or items there.

If you’re unable to attend this Fudcon, do follow along in #fedora-fudcon on irc.freenode.net. There should be pointers there to whats happening, whats being discussed and allow for remote input. Hopefully we will have irc transcribers in various rooms as well.

Dear LazyWeb,

From time to time, I am required to use third party tools in my job where I may have to copy a bunch of information (separate fields of data) and paste them one at a time into a web form.  Years ago, while working at a small web design firm, I was introduced to a tool for Windows called ClipMate.  At that point in time, ClipMate proved to be a huge time-saver when copying customer data from our Access database and then pasting it into an online account setup form for a third party web-stats tracking service that we used on client websites.  What sets ClipMate apart from other clipboard tools is that it not only stores a list of your clips (either text or images), but it will allow you to select a clip from where you would like to start pasting from, and then continue pasting down the list, one clip at a time.  In the example that I stated previously, can you imagine going to the database, copying the client’s name, then pasting the name in the web form, then going back to the database to copy the next required chunk of information, and so forth…?  Wouldn’t it be nice to be able to go to the database and copy, copy, copy, copy, copy…, then go to the web form and paste, paste, paste, paste….?  ClipMate allows you to do this.

There are a handful of clipboard managers/organizers in the world of open source, but I have yet to find one that will do as I have described above.  It has been some years now since I’ve used ClipMate, but recently, because of some repetitive copying and pasting that I’ve been doing in my job, I purchased a new license and am again happily plodding along with my friend ClipMate.  However, this requires that I am using a Windows machine.  Fortunately in my job, I have two separate production machines – on Linux and one Windows, so I am able to pick and choose the tools that work best for the task that I am trying to accomplish.  Below are some screenshots of ClipMate in action:

I would like to have a Linux-friendly clipboard tool that will do as I have described – pasting successively from the list of clipboard elements.  Seeing as how the clipboard tools that I’ve seen will allow you to select an item from the clipboard list and paste that item, it doesn’t seem like it would be all that difficult to implement a feature that would allow you to successively paste one item at a time from the list.  I’m not sure that my programming skills are sharp enough to accomplish this at the moment, nor do I have the time to invest in trying to figure out how to implement such a feature, so I am hoping that some skilled individual will read my plea and decide to do what Open Sourcers do… grab some fine piece of software that already exists, make it better, and re-release it to the world.

Incidentally, I’ve tried running ClipMate under Crossover and Wine, and although the software did run, it somehow did not seem to be able to connect with my clipboard and so it was useless to me.

Maybe what I am looking for already exists and I just am not aware of it.  Please feel free to comment and let me know if you have found a tool that will work as I have described, or if you have created something that will perform the desired task.  Thank you!

I’ve just updated in epel and installed on fedorahosted.org another trac plugin: PeerReviewPlugin

This is a nifty little plugin that lets you do codereviews. You can set users to have CODEREVIEWDEV (developer who can submit reviews) and CODEREVIEWMGR (can approve reviews). You can enable the plugin in the admin section of your trac instance, and should then see a “Peer Review” button.

Enjoy.

EDITED TO ADD(2012-01-06): We ran into a problem with the plugin that I missed in my testing of it. ;( I’ve disabled it for now… if anyone has ideas on fixing http://trac-hacks.org/ticket/7034 please chime in there.

Thanks to some quick packaging work by Andrea Veri we have some new plugins we have added to trac on fedorahosted.org for your tracking pleasure:

• trac-workflowadmin-plugin: This plugin lets you change the ticket workflow on your project. You can setup what states tickets must pass through and in what order. It even gives you a nice little graph of workflow.
• trac-navadd-plugin: This isn’t too visible yet, but I am going to hopefully use it to add handy ‘register’ and ‘forgot password’ links to all projects. This should help with end user ticket reporting.
• trac-watchlist-plugin: This allows any logged in user to watch tickets or wiki pages without having to CC to the ticket or otherwise distrub it
• trac-tocmacro-plugin: This was enabled in the old fedorahosted, but wasn’t packaged up correctly and has been now. This allows you to use a TOC macro to provide table of contents on wiki pages.

As always, these plugins are available in the EPEL repos (currently in testing) for all to use, improve and share. If you need some specific plugin or macro in fedorahosted, just let us know!

From: Sean Reifschneider

Via Richard who got it from Tarek...

1. What's the coolest Python application, framework or library you have discovered in 2011?

I haven't been using it much lately, but looking back the library I think I was most excited about was bottle a Python micro-framework for building web applications. The thing I like is that it's extremely small, just a single file. For simple things that I might have used a CGI for in the past, bottle is great.

As far as the thing that I use the most on a day-to-day basis, that would probably be cony. Cony is a personal URL-bar command-line, so you can write custom shortcuts to different locations. For example, I have shortcuts like "backup [HOSTNAME]" which searches for the host and takes me to the page on the backup server, or "nagios [CUSTOMER]" which takes me to their nagios page.

2. What new programming technique did you learn in 2011?

Via a recent code retreat (written about previously), I really got into test driven development. Which is great, because our system administration work I like to call "test driven system administration", via Nagios, so it's very comfortable to me. But I hadn't had time to really get comfortable with testing my code until I got this opportunity at code retreat.

3. What's the name of the open source project you contributed the most in 2011? What did you do?

That's a tough one, because I've mostly done a lot of small contributions rather than any large ones other than ones I maintain. So probably the python memcache module or vPostMaster would be at the top. Until, that is, we release the source to our backup system, which I spent a lot of time contributing to this year. Aside: We've finally settled on a name for it: Network Attached Backup.

4. What was the Python blog or website you read the most in 2011?

I don't really follow any Python blogs... Most of my news comes from my friends feeds in facebook or plus. But the number one website I read in 2011 is almost certainly the library documentation. :-)

5. What are the three top things you want to learn in 2012?

I don't really set out goals for what to learn, certainly not a year in advance, because things change too fast. In general I want to continue to improve my business chops.

I'm expecting that in 2012 I'll be able to start programming seriously in Python 3.

6. What are the top software, app or lib you wish someone would write in 2012?

I'd love to have something that did a really good job of bringing together and making searchable all the data that comes at me. Thunderbird can often be pretty good for e-mail, but sometimes it seems like it just misses huge parts of my correspondence for reasons I don't fully understand. But I'd also like web pages I visit in there, photographs, and more...

Thanks Tarek and Richard! So what are your answers?

I just thought I would shoot out a quick post about some changes and enhancements at fedorahosted.org (an excellent place to host your open source project).

We’ve moved fedorahosted to RHEL6 and a newer set of hardware. Everything should be faster and newer. You will probibly most notice the upgrade to trac (from 0.10 to 0.12), but mailman, scms and all other software should also be newer. The backend is now a cluster of two drbd using nodes. We are going to look at decentralizing things much further in the coming year.

Speaking of trac, there’s a ton of new items in the new version along with some new plugins we have enabled:

• The MasterTickets plugin is now installed, so you can have tickets block or be blocked by other tickets, and get a nice graph of the dependency tree.
• Multiple repository support. You can now have your project list and show multiple source repos, using the same or different SCM’s
• A great deal of configuration is available to trac admins on notification emails and permissions.
• Unicode and i18n support is vastly improved
• With a bit of adjustment, the PrivateTickets plugin works fine with the new trac. This allows you to have tickets only some groups can read/see if needed.
• Configurable work flows for tickets. You can set things to go through specific states before being fixed.

Mailing lists (lists.fedorahosted.org) have some small but nice improvements, like properly handling bounce messages from inactive google.com accounts and better utf8 handling.

Our review board instance is still running along, and should be a bit faster now.

We have great plans for 2012 for fedorahosted. If you’re interested in helping us out, take a gander at the Fedora Infrastructure Getting Started page and jump in and join us.

See the fedorahosted.org FAQ for more information about Fedora Hosted and how to host your next open source project there.

I'd like to announce the general availability of a YUI 3 Nightly combo server. Each night, it will check out and create a new build of the YUI 3 source code, and serve it via a combo handler. In order to use it, just point your YUI_config to the new combo loader. Information about it can be found at the main page at YUI Nightly Doc Page.

Why did I do this? There are a few reasons. First, Ubuntu One can't use the Yahoo! CDN because we're using SSL. This means we have to run our own combo loader. The problem with this is that it's a pain to go out and get new YUI and plug it into my dev environment just to get surprised by something that broke or something that changed, etc. With the nightlies, I can regularly just point YUI to this combo loader and let it run.

I'm using this in "production" for a small personal site, but I wouldn't recommend it be used it any production site that actually has uptime requirements.

Comments, questions, and pull requests welcome.

Update: The URL has now been changed to http://yuinightly.com/.

From: Sean Reifschneider

Several of us have been running Gnome 3 lately, and have been happy enough with it. One thing I was thinking was that it would be nice if the date/time bar app showed my google calendar. Then Mike found this project that does just that: gnome-shell-google-calendar.

But, I subscribe to 8 other calendars. So whenever we had a company meeting, it would show a bunch of duplicated events, and it was hard to tell my events from others. So I hacked on it this weekend and have pushed a new version up to github: Sean's fork of gnome-shell-google-calendar.

This version de-duplicates events with the same title that start at the same time, and also displays what calendar the event came from. So far it's been working really well for me, but the upstream author is reporting an error so I'm working with them on that at the moment.

As you may have seen if you are on the fedora announce list, we had an outage the other day of our main build system NFS storage. This meant that no builds could be made and also data could not be downloaded from koji (rpms, build info, etc). I thought I would share here what happened so we can learn from and try and prevent or mitigate this happening again.

First, a bit of background on the setup: We have a storage device that exports raw storage as iSCSI. This is then consumed/used by our main nfs server (nfs01). It’s using the device with lvm2, and has a ext4 filesystem on it. It’s around 12TB in size. This data is then exported to various other machines to use, including builders, kojipkgs squid frontend for packages, koji hubs and release engineering boxes that push updates. We also have a backup nfs server (bnfs01) that has it’s own separate storage with a backup copy of the primary data.

On the morning of December 8th, the connection between the iSCSI backend and nfs01 had a hiccup. It retried current in progress writes, and then decided it could resume ok and kept going. The filesystem had “Errors behavior: Continue” set so it kept going (although no actual fs errors were logged, so that may not matter). Shortly after this, NFS locks started failing and builds were getting I/O errors. A lvm snapshot was made and a fsck run on that snapshot, which completed after around 2 hours. A fsck was then run on the actual volume itself, but that took around 8 hours and showed a great deal more corruption than the snapshot had. In order to get things into a good state, we then did a rsync of the snapshot off to our backup storage (which took around 8 hours), and merged that snapshot back as the master fs on the volume (which took around 30min to complete). Then, a reboot and we were back up ok, but there were some small number of builds that were made after the issue started. We purged them from the koji database and re-ran them with the current/valid/repaired filesystem. After that builders were brought back on-line and queued up builds processed and things were back to normal.

So, some lessons/ideas here, in no particular order:

• 12TB means most anything you decide to do will take a while to finish. On the plus side that gives you lots of time to think about the next step.
• We should change the default on-error behavior to at least ‘read-only’ on that volume. Then errors would at least stop further corruption, and best prevent the need for a lengthy fsck. It’s not entirely clear if the iSCSI errors would have made the fs hit error condition or not however.
• We could do better about more regular backups of this data. A daily snapshot and rsync off of that snapshot to backup storage could save us time in the event of another backup sync being needed. We would also then have the snapshot to go back to if needed
• Down the road some of the cluster fses might be a good thing to investigate and transition to. If we can spread the backend storage around and have enough nodes, the failure of any one might not be as much impact.
• Perhaps we could add monitoring for iscsi errors and note and react to them quicker
• lvm and it’s snapshots and ability to merge a snapshot back in as primary really helped us out here.

Feel free to chime in with other thoughts or ideas. Hopefully it will be quite some time since we have another outage like this one.

...K-I-S-S-I-N-G

The Web and Mobile Ubuntu One team is in Buenos Aires, Argentina (along with most of the rest of the Ubuntu One teams) exploring the boundaries we may experience merging the Web and Native App experiences. I realized very quickly in this exploration project that I needed an end goal. Since we are already using YUI on the web part of Ubuntu One, I figured that I needed to figure out where the limitations of YUI would be there.

Enter PhoneGap.

It became very clear that it'd be much better to use one of these "glue" frameworks than to roll our own (the important reason being that we'd rather do more fun things). One of the first things I noticed is that the default PhoneGap doesn't really "compile" out of the box, which annoys me. Once I linked in the www folder properly, I had a good stub.

Using the YUI Loader in PhoneGap is not impossible, but it's relatively impractical. It stores a whitelist of URLs that you can get to, and even then, javascript is so required on the site that we can't afford to not have javascript. At this point, I realized that all the front-end tweaking/ricing I've been doing for the last year or so gets thrown out the window. I broke down and made giant rollup files for yui.js and yui.css.

As a sidenote, I also took the time to actually explore the YUI App Framework by coding rather than reading about it (which I've done a lot). There is an example Todo app (thus, the reason why I made a todo app). So I essentially cargo-culted that code.

Sometime during this cargo-culting, I realized one very important thing: Debugging PhoneGap apps is a pain in the ass. It really is. It resulted in this late night tweet (which I was at least half serious about). Unless you're a human jslint machine (and if your name is not Doug Crockford, I assert that you are not a human jslint machine), you're bound to make typos and syntax errors and such. You don't really get a decent access to the console1. I tried running the app in a normal desktop browser but I found that events I was expecting either weren't firing or weren't being handled (without a console, I don't know). I eventually went digging into the PhoneGap js that comes with the default template, and realized that I needed to just kill the PhoneGap js when testing in my browser. The browser only takes you so far, but it at least helped me to get the YUI toolchain sorted out.

I then took a bit of time sorting through the touch-specific interface and adapting the App example to that interface. For instance, hover events are useless on mobile. I also found that the "keypress" event for the Enter key in an input wasn't firing with the keyboard "return". Apparently, the iPhone wants the form input wrapped in a form itself, and then it'll show the "Done" button in the on-screen keyboard, which will send the right event (make sure that the form doesn't actually submit).

The end result is MoarTodo2. It's a really ugly app, but it's academic and I just wanted to learn about the toolchain. I can't imagine not using something like PhoneGap (it doesn't have to actually be PhoneGap) to build a mobile web app to be distributed through the various app stores. I expect there will be more to the debugging story, but it's still a learning curve.

I still feel pretty strongly that the Native vs. Web argument will continue, and that they'll still have their various places. I look forward to a day when that argument is settled, but I also look forward to the Year of the Linux Desktop, the rapture, and Santa Claus.

1 I didn't try Weinre because I had switched to using iWebInspector a few weeks ago (when I wasn't using a PhoneGap container), and so didn't have it immediately on my system. I'll probably have to go back and try it out in a PhoneGap context.

2 Xcode creates new projects with Git. I'm not making a political statement here. I just pushed it where it was easiest. Don't hate.

From: Sean Reifschneider

Yesterday Bill Tucker and Matt Rose organized a Code Retreat in Fort Collins. It was "Global Day of Code Retreat", and there were events happening all around the world. The basic idea was that you paired up and for 45 minutes used best practices (as if you had all the time in the world to do it) to work on a programming problem. Then everyone deleted their code, stood up and discussed the previous round. Then you paired up with someone new and started over on the same problem.

It was a great event, extremely rewarding on many levels. Read on for more of my thoughts on the event. (read more)

Recently, Fedora infrastructure requested everyone change their password and upload a new SSH public key. We gave folks almost 2 months to make the change. I’d just like to note here the reactions we had, divided into 4 ‘groups’ of users. Of course some people were between two of these groups or some probably had slightly different reactions, but this is what my reading of the feedback leads me to:

1. Group one sees the announcement, skims it, goes and changes their password, generates and uploads a new ssh key, goes back to what they were doing.
2. Group two sees the announcement, reads it, reads the links off it. Adjusts their firewall, checks their backups, re-enables selinux or applies updates, then changes their password and uploads a new key
3. Group three sees the announcement. Complains that their private ssh key is safe and always has been, and they know all about passwords and ssh and encryption and this change is unneeded.
4. Group four doesn’t even see the announcement. They are no longer involved, too busy to read it, or just don’t care

Group one spends about 5 minutes. The advantage to Fedora Infrastructure isn’t great here, but they do have a new password that meets the guidelines and a new ssh key in case they were careless with the old one. Of course they didn’t learn much, so they could be careless with the new one as well.

Group two are the very people we are trying to reach most, and here’s the most advantage to this plan. These people will learn how to improve their security some, how better to handle their ssh private keys and hopefully prevent compromise on their personal machines. They may spend a good deal of time following the links and learning about best practices, but it’s all time well spent.

Group three are the vocal minority. While they already know best practices and keep their ssh private keys safe, they don’t realize we have any way of telling them apart from group 4 (below). Time spent here is large for both them and Infrastructure folks, but advantage is low because it amounts to “I know I am fine and shouldn’t have to do this” vs “We have no way of knowing that”. There are likely a less vocal subset of these folks that show up in Group one (just do it and grumble and move on).

Group four is another group where there is good advantage for infrastructure. These are folks that are gone, don’t pay attention to their Fedora work, or are too busy to spend a few minutes on it. Packages with these folks as maintainers would be better off being orphaned and reassigned to people who use and care for them. Sysadmin groups with these folks are better off not having someone who they think are involved, but really doesn’t have time to be.

So, in the end I think there is still good advantage to us having done this. I hope the folks in group two are large and learn from our documents and best practices. I hope the people in group three realize that this isn’t just about them, it’s about the community, and I hope pruning the people from group four helps improve the health and activity of our community.

Finally, as a side note, the deadline was last night, but we are still assessing exactly how we want to mark inactive folks who haven’t yet changed their password and uploaded a new key. You still have time to go do it now.

Last night, I came home from a meeting, kissed my wife, and was walking back upstairs when my coat started vibrating. I pulled my phone out to see that I was too late, and wasn't able to answer a phone call from my dad. I called him back, and here's a summary of the beginning of that conversation.

Me: "Hi dad."

Dad: "Hi"

Me: "Did you call? What did you need?"

Dad: "I did. I installed Ubuntu and now I'm trying to figure out how to install the driver for my video card."

Me: "Waaaaaaaaaaitaminute. There's a backstory here that I have to know before we go any further."

My dad is "technical". I remember as a little kid realizing how excited my dad was when he was installing Windows 95 the day it came out. I spent summers and school holidays making money at the company he worked for by being the floppy swapper when installing software. He was also the person who introduced me to the term "freetard". He's always been pretty resistant to free software, and sometimes for good reason (ever tried sending a document back and forth between on OpenOffice environment and a MS Office environment multiple times?). Last year for Christmas, my little brother Matt received a netbook that got infected before Christmas Day was over. I offered to put Ubuntu on it (something he wanted) but my dad was so resistant that he went out and bought a USB CD-ROM drive just so he could make sure Windows stayed on the machine.

For me, Ubuntu has never been about "So easy my mom can use it", but "So simple my dad doesn't have a reason not to use it".1

As the story unfolded, my dad had been exploring the idea of virtual machines for a client, and had installed Ubuntu in a Virtual PC environment on Windows 7 host. Unfortunately, his video would only give him 800x600, when his Windows XP guest would fill his dual-monitor setup easily. He expected this to be a driver issue (Virtual PC apparently doesn't have a driver pack for guests like VirtualBox and VMWare do). I found this article, but as I looked at the instructions, I thought "Holy crap. I wouldn't even do this." It became pretty clear that Microsoft Virtual PC was a pretty hostile environment to anything non-Windows.

At this point, my dad starts asking about the Windows installer he saw for Ubuntu (Wubi). Most of his concern centered around whether or not it was going to screw up his Windows partition (it doesn't). After a bit of talk about Wubi, we ended the phone call.

This morning, I get an email from him that says:

Installed 32-bit Ubuntu on my system last night (Wubi). Booted this morning and found that my USB keyboard/mouse combo is not recognized at Ubuntu login screen. No time to troubleshoot this morning before work and probably won’t spend a great deal of time on it, but wondered if you had any ideas of things I could try this evening.

Sounds like he's got a bluetooth keyboard/mouse combo that is having problems. It's not a happy ending just yet, but it's a giant step in the right direction.

The Chasm. Ubuntu is crossing it.

1 I gave my mom my old laptop, running Ubuntu, and connected to Landscape so I can manage it for her. She's not technical, and Ubuntu satisfies most of her computer needs (the only one that it doesn't satisfy is that she has no genealogy software that's compatible with the other software she uses).

From: Sean Reifschneider

Vim Omnicomplete is extremely useful. We use vim for an internal project and have some cases where I wanted to do a custom Omnicomplete, however I'm not really that comfortable with vim's own scripting language. Looking at the existing completion scripts really wasn't helping much either.

What I really wanted to do was build the "omnifunc" completion function in Python. After a few hours of playing, I was able to come up with exactly that and get my completion code working. Now when I press Control-X Control-O I get a menu of my custom completion items, based on project names. It's wonderful!

So I spent a few more hours cleaning it up and making it into a template that hides all the magic and presents (what I hope is) a dead simple structure that a Python programmer can use to make omnicompletion plugins. I've released this template on github as: vim-omnipy-template

Just a friendly reminder: If you are a Fedora account system account holder, and haven’t changed your password and uploaded a new ssh key since we announced the mandatory change, you best do so NOW. The deadline is 2011-11-30 (only 2 days away).

If you don’t, you may no longer have access to groups you currently do (like packager, or sysadmin or ambassador).

Go take a few minutes, read the announcement and security information linked to it, and change your password and upload a new ssh public key.

If you aren’t a Fedora contributor, the information linked in our announcement is still a great read and may just help you be more secure on your machines.

From: Sean Reifschneider

This weekend I wrote a lot of documentation. I spent around 12 hours on it in total. I knew there was no way I could stand being in LibreOffice for 12 hours, but I also knew that the consumers would prefer a prettier format than just plain text. I've been writing READMEs lately using markdown format, which converts to pretty HTML on github, so I decided to go that direction. Boy did it work well, once I found the magic tool: pandoc! Read on for more details... (read more)

Tonight I decided to hack on a quick little webapp and use the opportunity to explore some tech that I've only gotten to tinker with so far. I wanted to build a small webapp with express, but I hate jade templating (I realize this may make me a javascript heretic). Since I'd rather just write HTML, I decided to use Mustache (plus, I have previous experience with Mustache).

The biggest problem with Node.js in general is that development is too fast. Blog posts get out of date really quick. I found a few blogs that show how to do it, but they're more than a year old, and at that point, it's almost worthless. Then I found stache. The docs there helped out to get me most of the way there. Here's what I did to get a basic express app using mustache.

First, install express and stache, and then create a new express app named myApp:

sudo npm install -g express stache
express myApp

Now change into the myApp directory. Edit app.js.

var stache = require('stache');

...import stache and then configure the app to use stache (you'll replace the existing jade config)...

app.set('view engine', 'mustache');
app.register('.mustache', stache);

In this case, I'm using a .mustache extension for my templates, and then passing stache to it as the handler for mustache templates.

In your views/ folder, you should see two files that the express command created by default: layout.jade and index.jade. We want to replace these with mustache stubs similar to these files. Let's start with layout.mustache.

<html>
    <head>
        <title>{{title}}</title>
    </head>
    <body>
        {{{yield}}}
    </body>
</html>

In stache, {{{yield}}} seems to be a special keyword for using the subtemplate. Make sure you remember all three suspenders around it, or you'll spend 20 minutes scratching your head and wondering why the HTML is being escaped (a behavior I hadn't ever used in mustache).

Now in index.mustache, we add the following:

<h1>{{title}}</h1>
<p>Welcome to {{title}}</p>

At this point, we've got a similar layout and template to the jade templates. The last thing left to do is change the data structure of the context we pass (since mustache wants the template context in a different format). Crack open routes/index.js and change the res.render call to look like this:

res.render('index', {
    locals: {
        title: 'Express'
    }
});

Now fire up the app with node app.js and hit /. Does everything work? Hooray! If not, uh, sorry. I'd be interested to know what didn't work, so I can fix the post.

In fact, since Node.js posts tend to get out of date, if this one does, please let me know and I'll update it accordingly.

From: Sean Reifschneider

One problem that production system administrators have is running systems with older Python versions installed as /usr/bin/python. Worse, you probably have some systems with a newer Python and some with an older Python. For example, in our environment we're managing some systems with Python 2.1 and some with Python 2.6...

And you can't just upgrade /usr/bin/python, because that will likely break many other things on the system, mean that packages that provide modules either have to be rebuilt or are no longer reachable by the Python interpreter, etc...

We've been struggling with this for one of our system maintenance packages, where there is newer Python available on the system, but just doing "#!/usr/bin/env python" will pick up a very old version.

The "ineedpy2" module will either try to find a newer Python version and re-run the currently-running program with that, or you can specify a minimum minor Python 2 version that you need, and only if you are running something older than that will it swap in a newer one. For example, you may have both Python 2.6 and Python 2.4 on a system, but 2.4 is sufficient, so you don't need to do the extra work, but if you are running under 2.3 it will.

For example, you can say "ineedpy2.rerunonlatest()" to get the latest available Python on the system, or "ineedpy2.requireminor(4)" to only re-run the program if you are on 2.3 or older.

I've released this module on github at https://github.com/linsomniac/ineedpy2 and will put it in pypi soon.

From: Sean Reifschneider

A few weeks ago I had a 1U system that had a RAID card, but we needed to add more network interfaces to. I had the idea of switching it to software RAID, freeing up the only slot for the network card, and it went amazingly smoothly.

The bulk of the system was installed on an LVM, with just a small /boot partition. So what I did was add two drives to the system, and partition them with a small partition for /boot, and the remainder for LVM. These partitions were set up with software RAID-1.

I then added the LVM partition to the existing LVM, and did a "pvmove".

"pvmove" is a great tool that tells LVM to move the blocks from one device to another. It achieves this by first creating a mirror of a block from the source device on the destination, waiting for that mirror to sync up, and then breaking the mirror by removing the source block. After an hour or two, with the system still operating normally, the source RAID array just contained /boot.

Finally, I copied the /boot partition over to the software RAID, rebuilt the initrd to include the software RAID tools, re-ran grub-install on the two new drives.

So with fairly little muss and fuss, and little down-time, I had migrated from a hardware RAID array to software, on 2 new drivers. LVM is a very wonderful thing!

From: Sean Reifschneider

As part of system automation work, the other day we were talking about changing the postfix configurations. Mention was made of using sed, but I brought out "postconf -e", a tool that postfix provides to allow modifying the configuration. Particularly useful for more complicated entries which can span multiple lines and a simple sed script wouldn't do the right thing.

You run it as: "postconf -e name=value"

This will modify the configuration to change the setting "name" so that is assigned the value "value". Of course, proper quoting is necessary if you are using special characters.

I also frequently use "postconf" to have it tell me what the current configuration is (because sometimes looking at the file it isn't obvious) and "postconf -d" to tell me what the default configuration values are.

Definitely useful tools to have in the tool-box.

A few days late, sure...

Once again, I pity the fool who looks at my notes expecting them to be coherent. They make sense to me, and that's why I took the notes. Here's the highlights:

Andrew Wooldridge - YUI Hidden Gems

I didn't expect to take many notes here, since I'd taken my fair share of dives into the YUI source code (I generally no longer care to read the online docs), but Andrew pointed out a few things I had glanced over and missed the value of. Among them, he mentioned:

• Y.Frame - this is a wrapper for using the IFRAME element. It creates a YUI instance inside the IFRAME, so there's sandboxing, etc. but without the complications that come from IFRAME communication. There are quite a few places that we could use this in Ubuntu One. In particular, I think we could use it for the multi-file upload that we've been wanting to do for a long time.

• Y.DOM.inViewPortRegion - The previous day, I went looking into the ImageLoader source to see how it worked, since the NFL talk mentioned a WidgetFold that only rendered the widget when it came into the viewport, and I was curious how it did it. Even though I had found this hidden gem the day before, it's worth noting its value in all sorts of cases. What I'd like to do is make a synthetic event for Node such that we can hook eventHandlers into "enterViewPortRegion" and "exitViewPortRegion" or something similar.

• Y.on('hover') - How did I miss this? It's a synthetic event that YUI provides, since mouseOver and mouseOut can be a bit of a fun house of edge cases. Quite useful.

• Y.on('clickoutside') - I use this in my little phazr library to close on Overlay or Dialog. It's fantastic.

• Y.Later - A wrapper for setTimeout and setInterval. I made specific notes to play with this in some of the U1 things that I'm working on right now.

Eric ForRealYo - The App Framework

This talk Melted. My. Brain. A common theme that I heard repeatedly was that people tend to implement things in YUI, and right when they're finished, the YUI team announces the exact same thing in YUI. I did this recently with the App Framework.

The App Framework in YUI 3.4 leaves just a bit to be desired, but Eric's talk made it clear in his talk that all those desires will be available in YUI 3.5 (with a promised pr release in December). Eric had mentioned earlier in the week that he kind of looked at the iOS UI framework API when designing the App Framework, and I can see that connection. It's got a pretty good disconnect between data and presentation, which can often be kind of painful if you don't give it some thought. A nice API like the one the YUI team has designed makes it easy to design things correctly without putting too much thought into it.

One of the things I'm looking forward to the most in YUI 3.5 is the addition of support for Handlebars templating. This templating will be directly connected into Y.View, but won't be compulsory, so you can use any template system you'd like, and aren't forced in to Handlerbars.

As it stands now, YUI 3.4 has some of the App Framework stuff, and in particular, it has Model and ModelList, which have events that can be hooked into. To me, that's the best way to start using the App framework. Create widgets that handle events in your Model and ModelList instances. At the very least, that means that I have a clear upgrade path from my app framework to YUI's (so I don't have to maintain code, I can just file a bug and complain... :)

After all of this, it became clear that I've been neglecting the YUI Gallery, both in contributing to it and using it. I need to remedy that.

Day 1 of YUIConf 2011 is over. I didn't keep copious notes that are coherent to anyone but myself. Here is a congealed version of my notes (hopefully they aren't too hard to follow).

Dav Glass - Keynote

Dav said something that seemed to ring very true to me about the "Graded Browser Scale". Instead of talking about browser support, he started referring to it as a list of platforms to test on. If everything works on that list of platforms, it should work on everything. This is usually the way that I think about testing my own projects, and probably the way we should start thinking about Ubuntu One (granted that we don't see too much IE 6/7 on Ubuntu One currently).

The YUI team is starting to think more about time based releases. Having worked for Canonical for so long, I love the "cadence" that time-based releases bring. I hope this means new YUI features are available in a faster manner, but mostly I'm looking forward to the promise of being more open with what features are being worked on. It seems that we at Ubuntu One aren't the only ones implementing things that the YUI team end up releasing right after we finish writing it.

Luke Smith - From one, many; From many, one

One of the hardest and most overwhelming things about YUI is figuring out how to implement features. Should I use a Plugin, a Widget, or an Extension. Along with that, if you're used to class-based inheritance, inheritance (or the oddities thereof) of javascript is a little awkward. In general, for what I'm building, I don't need to worry about how the constructors link together, so I usually just build off of Y.Base and everything is fine.

One really important thing I had previously personally asserted and Luke confirmed to me is this: Unless you have a valid reason, use Y.Base.create.

Jeff Craig - Demystifying Loader

This was easily the best talk I went to today. The YUI Loader is a fantastic little piece of YUI, but it's also a black hole for documentation and isn't something most people grok. At Ubuntu One, we use convoy for combo loading YUI, and our configuration is a bit complicated. I've known about some of the things we could do to make it less complicated, but this was pretty beneficial for me. Particularly, the Loader can conditionally load modules, which should help in cases where we use modules to "monkey-patch" other modules in certain cases coughIOcough.

Ryan Cannon - There is no off-season.

Ryan talked about implementing YUI on NFL.com. This is very similar to a talk I proposed for YUIConf this year as well. Essentially, I like the stories of "This is how we got our site all working in YUI". They wrote a really neat sounding widget they called WidgetFold that only rendered the widget once the user scrolled down to see the widget. I can think of all kinds of uses for that kind of widget.

The highlight of Ryan's talk was "Are you writing scripts or building applications?" He used it in the context of picking the right tool for the job. I often bring up this theme when people ask "Why not just use jQuery?", but the way Ryan phrased it is pretty excellent.

I bypassed taking notes for the last two talks of the day. After dinner, Douglas Crockford spoke. My notes would make absolutely no sense, in that they are all over the place. The talk was about jslint (which Doug wrote). As a summarizing quote (although it doesn't nearly do the talk justice), I noted "use of good style can help reduce errors".

Afterwards, someone pointed me to the github issues page for jslint. Doug's responses are pure comedy. Next time I'm having a bad day, instead of going to laugh at bad Craigslist personals, I'm going to laugh at jslint issues on Github.

After a few years of being a happy Ubuntu user, I was getting tired of their too much consumer-oriented focus. I mean, I am a geek and I don't care much about the latest and greatest Unity, UbuntuOne, Apple-like styling like why-don't-we-put-the-windows-buttons-on-the-left and the likes. Worst, I was annoyed by this stuff. Don't take me wrong, I still recommend Ubuntu to family and friends (most notably, my father in law is a good example). But myself? I wanted something more suitable to my geek soul. So, I tried Debian (Wheezy, current in testing to be precise).




You know what? It's great, works out of the box as Ubuntu (on a super recent hardware which has some weird driver requirements), doesn't have the consumer-friendly the-geek-in-me-thinks-it's-crap feeling of Ubuntu. There are a couple of minor things to tweak, such as it doesn't install vim by default (like Ubuntu, see here, some of it applies to Debian too). The only Debian-specific problems are the gnome-display-properties (I still hate the GNOME policy of using a different name on the window) keeps forgetting that I have two monitors. And it doesn't suggest to install the stuff you don't have by default. But nothing too bad that I wouldn't figure out myself quick. If you just one to do a single thing, the most useful one is to include the following aliases (like in Ubuntu):

alias rm='rm -i'
alias cp='cp -i'
alias mv='mv -i'
alias sudo="sudo "

Why I did the latter is left as an exercise to the reader :-)

Bottom line, I plan to run Debian for a long time.

Looking at a bug report today, I saw some pretty poor “bugside manners” on the part of a few people (including the upstream developer). There’s tons of examples out there of good bug report interactions and poor ones. I’d like to urge everyone to take a minute and think before posting that next bug comment.

Good bugside:

• Ask for facts. Program output, versions, exact behaviors
• Provide info if asked for it by others
• Try and see the issue from the reporters view
• If you find yourself unable to say something nice or move the bug along, perhaps you should refrain from saying anything at all

Bad bugside:

• Don’t pile on unrelated bugs. Open a new bug for a new issue.
• Avoid talking about “big picture”/design/philosophy. Those should go to mailing lists or the like, bug trackers are to track and fix bugs
• If you don’t have things to add about the specific bug at hand, don’t post

I’m sure there’s other good/bad rules. Anyone run accross any other common good or bad bugside manners they want to share?

Your blade centers are pretty nifty… however:

a) Isn’t there any way the remote console could just be vnc directly instead of requiring a java applet? Because the java applet has various issues and there are some very nice direct vnc clients available. It would be less hassle for you even.

and

b) Could you pretty please have servers pop up a list of boot options/items right away, even if it takes 10minutes to finish booting to any of them? Having to spend 10minutes staring at a java applet waiting for the 10 seconds I can press “1″ to get to the SMS menu is sure a drag on my time management. Especially when it doesn’t take the keypress and I have to just do it again.

Thanks!

Last night, Skeleton Jump got published to the App Store. Download it!

I've been working on this game for about a month, having had no previous iPhone development experience (outside of tinkering, really). I've been terrified of putting this out there, knowing about some heinous bugs that still exist in the game (but I can't find the cause for). It doesn't look like many people have problems with them, so I've been afraid for no reason.

Hooray for releasing code!

I've been thinking about writing this post for about a month now. I should preface it by saying that I've found myself internally conflicted between software freedom and making a living. This is a fact that I'm reminded of every time I point out to a member of the Ubuntu community that I've moved from working on Launchpad to working on Ubuntu One. Let me be straight about this: I love Ubuntu, and I love software freedom. I also love fistfuls of cash. Some days, this internal conflict is maddening.

I have loved games since I was young enough to remember. My interest in technology came from learning to repair my NES console when I was 5. Last month, I met a fantastic artist named Jess Smart Smiley. He and I conspired to make a Halloween themed game in our spare time.

I don't have much of a desire to create games professionally, so I'm more interested in designing and creating games classified as "indie". Having blogged about the potential ecosystem for indie games before, I already had a head start on what our first target would be.

The first thing we thought about is existing and mature development libraries. The most popular mobile game framework is cocos2d. It was originally a framework for iOS, but also has an Android port. I also considered Ansca Mobile's Corona framework, but its high cost for both platforms (iOS and Android) made me eliminate it. The Android port of cocos2d had it's last commit Nov 15, 2010.

After looking at software, we started looking at the hardware. I have never owned an iPhone, but I've owned every one of Google's official Android phones. My wife has also had Android phones for a number of years now. She's not a gadget person, but she recently fell in love with an Android tablet (the Asus Transformer) and so we have a number of Android devices around the house. The only iOS device we had was my iPad. As I looked at the form factors and various hardware for Android, I thought "Oh man, this is going to be a QA nightmare." I tried out a number of Android devices and they each had their own quirks/lag/bugs. With the exception of a weird home screen bug in my Android devices, I never had that "lag" that everyone seems to be complaining about on my Nexus phones, but my wife's first Android phone was a huge piece of junk with a plastic touchscreen and Android 1.6. There are basically 4 different hardware configurations for iOS, and only 4 (currently).

We decided that our first game would target iOS.

After we decided on iOS, I had lunch and/or !coffee with a few local mobile game developers. All but one had experience with both Android and iOS. They all had the same problem with the Android market: people decompiling/recompiling their apps and putting them on the Android Market for free. It's basically an arms race at that point, where you spend your days sending takedown notices to various Android stores instead of developing new stuff. The developers who were still committed to Android said that the only way they've made money is by using ads. I don't like that, since your mobile's screen is small enough already without ads taking up part of the screen (or dealing with artificial load times while you show ads).

We'll probably move into Android in the future, especially with its market share growing like it is. However, for the our first game, I feel confident iOS was the best platform to deploy to.

If you're interested in our game, become a fan of Skeleton Jump on Facebook!

UPDATE: Lucio points out on Twitter that cocos2d is actually originally a python implementation. I knew this, but didn't see much connection between the two.

This morning, we announced that we are requiring Fedora contributors to change the passwords and upload new ssh public keys by the end of next month. There’s no breach or cause for alarm, we just thought it would be a good time with all the high profile hacking happening out there for everyone to go look at their security practices and create new keys and passwords (see the announcement for full list).

Please do go and change your password and create a new ssh key at your convenience (but before 2011-11-30).

I’m sorry that the ssh key requirement has caused stress for some contributors, but realize we are not singling anyone out here, there’s good reasons to ask for this change now when it’s not urgent or triggered by outside events. Just a few of them:

• Allow you to revisit your security process and policies and read about best practices
• Allow you to see how to make changes and what machines and places you need to make them in the event you were making a more hurried change
• Allow you to setup a separate ssh key for Fedora matters. This separates out some risk, at the cost of another passphrase and possibly hitting ssh server limits (most allow you to try only 6 keys).

While many of the more savvy Fedora contributors already know these things, and have good practices, we hope that everyone will learn something from this or at least not let it inconvenience them for too long.

One thing I really like about Google+ is the ability to limit your posts to a "Circle" of people. Facebook recently added a similar feature. I decided to try it out and share a post with all my roller derby friends (including my wife). So I start exploring the UI and hit this:

I had no idea what to put in the "Hide this from" text box. Maybe "everybody else"? Turns out, you can ignore it, and it'll be limited to just that single list. I assume that box is so that you can add a list and then exclude someone. Please to hide that away, or even take away the feature entirely.

From: Sean Reifschneider

I recently had a kernel bug that caused a process to hang in D (disc wait) status. I decided to do a reboot, but thought I'd try upgrading the kernel before I did that. The kernel upgrade also got stuck in "D" while trying to do a depmod, so I finally did a reboot, and had to hard power down.

During the reboot, it would get to where it would say "Begin: Running /scripts/init-bottom", and hang there. I searched around and didn't find much, so I figured I'd document the solution here.

I booted into rescue mode (off the initial media, I wasn't able to boot with "init=/bin/sh"), and then got a shell in the root file-system. I did a "dpkg --configure -a", and then rebooted and that solved the problem.

I found this issue by trying to run "apt-get update" in the rescue environment and it helpfully told me that it had been interrupted and I needed to run the above command. :-)

From: Sean Reifschneider

Last month I had wanted to try to get some real monitoring of a bunch of my personal systems. I spent weeks dabbling with setting up a serious monitoring system (tried Zabbix, Zenoss, nagios, and Opsview), but I was making little if any progress with those. Largely that was related to running a RHEL 6 derivative and running into compatibility problems...

After spending around a day of effort on them, I decided to roll my own. My needs were modest, I wanted to be able to run some nagios check scripts and get an e-mail if checks failed continually for 15 minutes. But not get e-mails every 15 minutes if they were down, and only alert if they failed several times.

Read after the fold if you're interested in my solution... (read more)

Android's Back button (whether in hardware or software) is braindead. I can understand the technical reason for its behavior. It's supposed to take you to the previous Intent...

...unless the current app has overridden the Back button.

Here's an example: Yesterday I attended BLDR.UX11. When I got into Boulder, I pulled over and pulled up my email with the ticket confirmation in it (using my Nexus S phone, not an Android phone with crapware on it). From there, I opened the conference web site. I then found an address for the venue and clicked on it, opting to open it in Maps. Once there, I had Maps give me directions to the venue, and then hit the Navigate button for turn-by-turn navigation. When I got to the Boulder Events Center, I grabbed my phone and hit back, which took me to the Maps app (from Navigation). I hit back again and it went to the BLDR.UX11 page in the browser. I hit back again. WTF? Now I'm looking at an article I read on my phone while waiting in the doctor's office a week ago. At this point, I repeatedly mashed the back button out of frustration until I was at the Home Screen (I realize I can use the Home button-I blame muscle memory).

While at BLDR.UX11, I met lots of folks who feel that having a single unified experience across mobile devices is a myth, and almost all of them cited the Back button as the reason. Some people go as far as to create their own back button in the app, which just makes the matter worse: now you have TWO back buttons.