• Testing NetBSD's share/mk

    For a long time, a pet peeve of mine has been the lack of tests for the build infrastructure files of NetBSD: i.e. those bsd.*.mk files that live under /usr/share/mk/ and on which the whole source tree depends.One could argue that writing tests for these files is not strictly necessary because the successful build of NetBSD is the real final test of whether the files work or not. That's partly true, but unfortunately is not the whole story:Other projects depend on these build files and thus rely on their behavior. This can either be by just relying on NetBSD having these files installed, or by using the separate mk-files package (available e.g. in Fedora). The functionality of the build infrastructure should not regress, or it would break these third-party projects. (Think about these files as having a public API outside of NetBSD.)Even if the build of NetBSD is the real test of the functionality of the build infrastructure, the build infrastructure supports dozens of options and tweaks to change its behavior. One would need to rebuild NetBSD tens, if not hundreds of times, each with a different combination of build options, to ensure the files work.Lastly, when a new functionality is added to the build infrastructure, it is because some component of the source tree needs such functionality. It may latter happen that this component disappears or stops needing such functionality. The functionality becomes unused by the source tree, and thus can regress unexpectedly (breaking third-party packages or introducing bugs, as mentioned earlier).With this in mind, it's clear that we should have some tests for the share/mk files. Unfortunately, that's easier said than done: the files in share/mk are extremely complex and expose hundreds, if not thousands, of different behaviors each with its own subtleties. Adding tests for these is hard. The fact that this code that was never designed to be unit-tested doesn't help either.Regardless, I have just submitted some "placeholder" tests to the tree. The major point of these new test programs is to lower the barrier of entry to writing tests for share/mk and, therefore, maybe get other people to write some tests. To predicate with the example, I have populated these skeleton test programs with a couple of test cases each, although as you will see they are very trivial tests.And, to conclude, why have I done this now? Well: I'm working on integrating Kyua into the source tree and, to make this happen, I need to do a couple of changes to the build infrastructure files. The changes are tricky, so I want to write tests to have some assurance that my modifications work and that, specially, they do not regress over time. Having the placeholder test programs in place makes this much easier, and the real functionality changes easier to review. [Continue reading]

  • Introducing shtk

    Have you ever wanted to have a collection of ready-to-use modules for shell scripts? I have, particularly because I keep reimplementing the same functions over and over and over and over again whenever I write non-trivial shell scripts, and I'm tired of doing so.That's why I have just abstracted all the common code in the aforementioned tools and put it into a new package called the "Shell Toolkit", or shtk for short. Yeah, this name sounds very pretentious but, really, I don't intend this to be anything big. The only thing I want to do is simplify my life when implementing shell scripts, and hope that other people might find the modules useful. So far, I have taken the generic (and common!) code from sysbuild and sysupgrade, reconciled a few tiny divergences, and moved it into this new shtk package.In reality, writing something like shtk is sin-borderline. I really should not be using shell scripting for the kind of tools I am implementing (they deserve better data structures and better error checking than what shell provides, for example). However, shell scripting is incredible convenient to get reasonably-good implementations of such tools with minimal effort, and is the only scripting language available in NetBSD's base system. (Yes, yes, there is Lua, but my limited knowledge of Lua would not let me write these tools in any decent manner nor in any reasonable time.)So, what's in shtk? There are a few functions to deal with command lines (error/warning reporting and such things), some trivial stuff to deal with lists, a bunch of code to interact with cvs and, what I like the most, a module to implement configuration files with some kind of key/value validation.At the moment, shtk can only be found in pkgsrc under pkgsrc/devel/shtk and I don't currently have any plans to make it more widely available. If there are enough people interested in that with real needs, I could reconsider, but the maintenance effort would be non-trivial.To showcase the features of shtk, I have updated the sysbuild and sysupgrade packages to depend on this new toolkit while at the same time dropping all this duplicate supporting code. It's a good thing that I wrote exhaustive tests for all possible code paths, because the migration from the built-in modules to shtk was riddled with subtleties that would have impacted end users otherwise.Now... time to really consider taking the task of rewriting pkg_comp in a more maintainable style so that I can add the features I have wished for for many years (like OS X support). [Continue reading]

  • Introducing sysupgrade for NetBSD

    Over the last two weeks, you might have had fun rolling your own NetBSD binary releases with sysbuild. But what fun is that if you have no trivial way of upgrading your existing NetBSD installation to a newer version?Upgrading NetBSD to a newer version from distribution sets generally looks like the following;Fetch new distribution sets (or roll your own).Upgrade the kernel.Unpack the distribution sets over the root directory, without fat-fingering the command and unpacking etc.tgz along the way.Use etcupdate to merge new changes to configuration files.Use postinstall to validate the upgraded system.Simple? Yes. Easy? No. The above procedure is obscure to anyone new to NetBSD. (Actually, if you tell anybody that the way to upgrade your machine is by unpacking tarballs over / will stare at you thinking you are kidding. It's 2012.) "Jokes" aside, what is worse is that the procedure is quite monotonous and, therefore, it is very easy for the administrator to make a trivial mistake along the way and screw up a running system. (Been there, done that... multiple times.)Machines are made to automate trivial and repetitive tasks like the above, and they are actually very good at that. Over the years, I have performed NetBSD updates manually and later written crappy, unreliable scripts to do the upgrades for me. These scripts have never been reusable and they haven't dealt with error conditions gracefully. Furthermore, because these scripts live in my home directory, I have to remember to carry them around every time I set up a new NetBSD box.It was about time I sat down and rewrote my custom scripts into something more "decent". Something with documentation, with a configuration file, and with tests.So today, and for all the reason above, I am introducing sysupgrade.sysupgrade is a script that automates (trivializes) the whole process of upgrading an existing NetBSD installation to a newer release, be it the currently-running system or a non-live system. sysupgrade does so by following the steps outlined above and is coordinated by a configuration file. You can find the tool in pkgsrc/sysutils/sysupgrade, next to sysbuild. The bundled sysupgrade(8) and sysupgrade.conf(5) manual pages, and the default sysupgrade.conf configuration file should get you started and hopefully answer most of your questions.For the impatient, the following command would upgrade your machine to the specified version target:$ sysupgrade auto     ftp://ftp.NetBSD.org/pub/NetBSD/NetBSD-<X.Y.Z>/$(uname -m)At the moment, please consider sysupgrade to be experimental. It works well for me on my various machines (running both NetBSD 6.0 BETA and -current), and I have been using the upgrade procedure outlined above for years without issues. However, as with any shiny-new software, be careful. If you use NetBSD on a virtual machine, take a snapshot before running this tool; I don't think your machine is going to blow up, but better safe than sorry!Enjoy, and feedback very welcome!PS: There is lots of room for improvement. The TODO file in the package directory includes some of the ideas I'd like to work on later. [Continue reading]