• Kyua: Weekly status report

    Unfortunately, no activity this week other than some brainstorming on the database design.Why? My shipment container from Dublin arrived and I spent most of the weekend organizing stuff, setting up my little NetBSD box and attending a friend's wedding! [Continue reading]

  • VirtualBox and port forwarding (and Fusion)

    I have been a VMware Fusion user for a long time and I think I am about to be a VirtualBox convert. Let me explain you my story.With the release of the Windows 8 Developer Preview, I felt like giving the system a try. Not because I particularly like or care about Windows, but just because I am genuinely curious about operating systems and wanted to check out their new Metro interface. So I went ahead, downloaded the ISO image, created a new virtual machine under Fusion 3, started the virtual machine, and saw Fusion crash while the Windows installer was booting.Oh well, an emulation problem. I tried to fiddle with the virtual machine settings a bit, selecting different CPU virtualization settings and different hardware devices. No luck. After that, I resorted to Google to look for a solution to the problem and found that other people were experiencing the same issue. But, at the same time, I also found that VMware had just released Fusion 4 and that this new release works with Windows 8 just fine. Ah, cool! Or not... $50 for the upgrade. For my use cases (which amount to booting a few BSD and Linux virtual machines in console mode), it is hard to justify the upgrade — specially when it would just be to fulfill my curiosity and then delete the virtual machine 10 minutes later.But wait! There is that thing called VirtualBox. I had used this product a while ago and did not enjoy doing so because its interface was utterly (really) confusing. It was nowhere near the polish of Fusion or Parallels. However, to my surprise, most of my major itches have been fixed and the new interface of VirtualBox is bearable. And it boots the Windows 8 preview just fine!Right. So after installing Windows 8, playing with it for 10 minutes, and later deleting it, I decided to look around and see what else VirtualBox has to offer. It has many settings. Yes, I know, Fusion has many settings too, but most are hidden from the UI and can only be accessed by modifying the vmx text file. In the case of VirtualBox they are easily accessible.I kept looking around... and there it was, the "Port Forwarding" option staring at me under the Network settings.One of the features I've been wishing for a long, long, long time is the ability to easily configure a NATed (not bridged) virtual machine in my laptop, assign a host name to it from the graphical interface and then be able to SSH into the virtual machine from the laptop. As it turns out, this is doable in Fusion but it requires some nasty tinkering: one has to edit a dhcpd configuration file by hand to add a new stanza for the virtual machine (/Library/Application Support/VMware Fusion/vmnet8/dhcpd.conf). This involves manually copy/pasting the MAC address of the virtual machine into that file (which you must first extract from the vmx file), assigning it an IP in the corresponding subnet, and sticking the IP plus host name in the /etc/hosts file. This gets boring pretty quickly.VirtualBox may or may not support this kind of setting, but it has the aforementioned "Port Forwarding" option which is interesting on its own. As you can imagine, this sets up forwarding of a port from the host system to a port of the virtual machine. Creating an entry for SSH is as easy as clicking a couple of buttons, choosing a local port on the host system (e.g. 2022) and forwarding it to port 22 of the guest system. Voila. You can now access your virtual machine locally and remotely without having to go through any of the host name nor DHCP madness I experienced before:$ ssh -p 2022 localhostYou can later create an alias in your ~/.ssh/config file to make accessing the virtual machine a breeze:Host myvmHostname localhostPort 2022And later:$ ssh myvmIt couldn't be easier. This trivial feature is a killer for the kind of environment I am interested in. And I also know that VirtualBox provides some other interesting features, like headless mode, that are not (easily) available with Fusion.So yes, I think I am a convert. [Continue reading]

  • Kyua: Weekly status report (db designdoc edition!)

    Moved the code of utils::lua to a new project, Lutok.Attempted to integrate a copy of Lutok into the Kyua source code to simplify installing Kyua. I have been playing with Subversion externals and Autoconf/Automake hacks to make this happen, but unfortunately haven't got a pleasant solution yet.Modified Lutok to not expose the Lua C API at all from header files and cleaned up the Kyua code to cope with the changes.Been chewing through the first chapters of the "Using SQLite" book to refresh my SQL skills. And, most importantly, wrote a preliminary design document for the database store of Kyua and the reporting features. Comments certainly welcome! Be aware that this is how atf-report will be replaced, so once this is done we should be able to finally kill atf-run and atf-report altogether :-) [Continue reading]

  • Name your C++ auto-cleaners

    As you may already know, RAII is a very powerful and popular pattern in the C++ language. With RAII, you can wrap non-stack-managed resources into a stack-managed object such that, when the stack-managed object goes out of scope, it releases the corresponding non-stack-managed object. Smart pointers are just one example of this technique, but so are IO streams too.Before getting into the point of the article, bear with me for a second while I explain what the  stack_cleaner object of Lutok is. The "stack cleaner" takes a reference to a Lua state and records the height of the Lua stack on creation. When the object is destroyed (which happens when the declaring function exits), the stack is returned to its previous height thus ensuring it is clean. It is always a good idea for a function to prevent side-effects by leaving its outside world as it was — and, like it or not, the Lua state is part of the outside world because it is an input/output parameter to many functions.Let's consider a piece of code without using the stack cleaner:voidmy_function(lutok::state& state, const int foo){    state.push_integer(foo);    ... do something else in the state ...    const int bar = state.to_integer();    if (bar != 3) {        state.pop(1);         throw std::runtime_error("Invalid data!");    }    state.pop(1);}Note that we have had to call state.pop(1) from "all" exit points of the function to ensure that the stack is left unmodified upon return of my_function. Also note that "all exit points" may not be accurate: in a language that supports exceptions, any statement may potentially raise an exception so to be really safe we should do:voidmy_function(lutok::state& state, const int foo){    state.push_integer(foo);    try {         ... do something else in the state ...        const int bar = state.to_integer();        if (bar != 3            throw std::runtime_error("Invalid data!");    } catch (...) {        state.pop(1);        throw;    }    state.pop(1);}... which gets old very quickly. Writing this kind of code is error-prone and boring.With an "auto-cleaner" object such as the stack_cleaner, we can simplify our code like this:voidmy_function(lutok::state& state, const int foo){    lutok::stack_cleaner cleaner(state);    state.push_integer(foo);    ... do something else in the state ...    const int bar = state.to_integer();    if (bar != 3)         throw std::runtime_error("Invalid data!"); }And we leave the boring task of determining when to actually call state.pop(1) to the compiler and the runtime environment. In this particular case, no matter how the my_function terminates, we ensure that the Lua stack will be left as the same size as it was before.But, as I said earlier, all this was just an introduction to the idea that made me write this post.When you declare an auto-cleaner object of any kind, be sure to give it a name. It has happened to me a few times already that I have written the following construct:lutok::stack_cleaner(state);... which is syntactically correct, harmless and "looks good" if you don't look closely. The compiler will chew along just fine because, even though we are declaring an anonymous object, its constructor and destructor may be doing who-knows-what, so their code must be called and thus the "unused variable" warning cannot really be raised.However this does not give us the desired behavior. The cleaner object will be constructed and destructed in the same statement without having a chance to wrap any of the following code, because its scope is just the statement in which it was defined. In other words, the cleaner will have absolutely no effect on the rest of the function and thus will be useless.So, moral of the story: always give a name to your auto-cleaner objects so that their scope is correctly defined and their destructor is run when you actually expect:lutok::stack_cleaner ANY_NAME_HERE(state); [Continue reading]

  • Introducing Lutok: A lightweight C++ API for Lua

    It has finally happened. Lutok is the result of what was promised in the "Splitting utils::lua from Kyua" web post.Quoting the project web page:Lutok provides thin C++ wrappers around the Lua C API to ease the interaction between C++ and Lua. These wrappers make intensive use of RAII to prevent resource leakage, expose C++-friendly data types, report errors by means of exceptions and ensure that the Lua stack is always left untouched in the face of errors. The library also provides a small subset of miscellaneous utility functions built on top of the wrappers.Lutok focuses on providing a clean and safe C++ interface; the drawback is that it is not suitable for performance-critical environments. In order to implement error-safe C++ wrappers on top of a Lua C binary library, Lutok adds several layers or abstraction and error checking that go against the original spirit of the Lua C API and thus degrade performance.Lutok was originally developed within Kyua but was later split into its own project to make it available to general developers.Coming up with a name for this project was quite an odyssey, and is what has delayed is release more than I wanted. My original candidate was "luawrap" which, although not very original, was to-the-point and easy to understand. Unfortunately, that name did not clear with the legal department and I had to propose several other names, some of which were not acceptable either. Eventually, I settled with "Lutok", which comes from "LUa TOolKit".At this point, the source tree of Lutok provides pretty much the same code as the utils::lua module of Kyua. While it may be enough to get you started, I'm pretty sure you will lack some functions in the state class. If that is the case, don't hesitate to file a bug report to let me know what is missing.In case you missed the link above, the project page is here: Lutok in Google Code. [Continue reading]

  • Using va_copy to safely pass ap arguments around

    Update (2014-12-19): The advice provided in this blog post is questionable and, in fact, probably incorrect. The bug described below must have happened for some unrelated reason (like, maybe, reuse of ap), but at this point (three years later!) I do not really remember what was going on here nor have much interest in retrying. [Continue reading]

  • Kyua: Weekly status report

    My plan for this week was to release utils::lua as a separate module. Unfortunately, this has not been possible because I've been fighting with legal to clear the name for the project. I don't have an approved name yet, so this will have to wait a bit more :-(On another order of things, I have started writing a design document for the database that will collect test case results and other information. Will share it as soon as it is readable and more or less complete. [Continue reading]

  • Diversions in Autoconf (actually, in M4sugar)

    Have you ever wondered how Autoconf reorganizes certain parts of your script regardless of the order in which you invoke the macros in your configure.ac script? For example, how come you can define --with-* and --enable-* flags anywhere in your script and these are all magically moved to the option-processing section of the final shell script? After all, Autoconf is just a collection of M4 macros, and a macro preprocessor's only work is to expand macros in the input with predefined output texts. Isn't it?Enter M4sugar's diversions. Diversions are a mechanism that allows M4 macros to output code to different text blocks, which are later concatenated in a specific order to form the final script.Let's consider an example (based on a few M4 macros to detect the ATF bindings from your own configure scripts). Suppose you want to define a macro FROB_ARG to provide a --with-frob argument whose argument must be either "yes" or "no". Also suppose you want to have another macro FROB_CHECK to detect whether libfrob exists. Lastly, you want the user to be able to use these two independently: when FROB_CHECK is used without invoking FROB_ARG first, you want it to unconditionally look for the library; otherwise, if FROB_ARG has been used, you want to honor its value.We could define these macros as follows:AC_DEFUN([FROB_ARG], [    AC_ARG_WITH([frob],                [AS_HELP_STRING([--with-frob=yes|no], [enable frob])],                [with_frob=${withval}, [with_frob=yes]) ])AC_DEFUN([FROB_CHECK], [    m4_divert_text([DEFAULTS], [with_frob=yes])    if test "${with_frob}" = yes; then        ... code to search for libfrob ...     elif test "${with_frob}" = no; then        :  # Nothing to do.     else        AC_MSG_ERROR([--with-frob must be yes or not])     fi])Note the m4_divert_text call above: this macro invocation tells M4sugar to store the given text (with_frob=yes) in the DEFAULTS diversion. When the script is later generated, this text will appear at the beginning of the script before the command-line options are processed, completely separated from the shell logic that consumes this value later on.With this we ensure that the with_frob shell variable is always defined regardless of the call to the FROB_ARG macro. If this macro is called, with_frob will be defined during the processing of the options and will override the value of the variable defined in the DEFAULTS section. However, if the macro has not been called, the variable will keep its default value for the duration of the script.Of course, this example is fictitious and could be simplified in other ways. But, as you can see in the referred change and in the Autoconf code itself, diversions are extensively used for trickier purposes. In fact, Autoconf uses diversions to topologically sort macro dependencies in your script and output them in a specific order to satisfy cross-dependencies.Isn't that cool?  I can't cease to be amazed, but I also don't dare to look at how this works internally for my own sanity... [Continue reading]

  • Introducing 'Dirt and the City'

    A friend of mine recently started a blog called Dirt and the City and, because he has invited me to contribute to it, I feel like posting a reference here ;-)Dirt and the City is a blog that intends to show you the "ugly" (literally) side of NYC. The blog is named after a famous TV show filmed in this same city; I haven't watched the show myself, but I bet that there is no dirt to be seen in it. If you are wondering what the city really looks like, this blog is your place.But, before visiting the blog, be sure to take it as it is meant to be: just a fun collection of photos! If you ever want to visit NYC, don't let the contents of the blog change your plans. You will definitely enjoy your visit regardless.Without further ado, and just in case you missed the link above, click here to visit Dirt and the City. [Continue reading]

  • Kyua: Weekly status report

    Slow week.  We had some team-related events at work and I have friends over from Dublin, so it has been hard to find some time to do work on Kyua.  Regardless, here comes the weekly report: Split utils::lua into its own package, per some user's request.  I'm still setting up the separate project and have to do lots of cleanup on the code, so nothing is available yet.Started experimenting on the long promised "tests results database".  So far, I have started writing a small utils::sqlite3 C++, RAII-based wrapper for SQLite 3 and a SQL schema for the database.  My rusty SQL skills don't help :-PHowever, all this work is local so there have been no commits to the repository this week; sorry! [Continue reading]

  • Splitting utils::lua from Kyua

    If you remember a post from January titled C++ interface to Lua for Kyua (wow, time flies), the Kyua codebase includes a small library to wrap the native Lua C library into a more natural C++ interface. You can take a look at the current code as of r129.Quoting the previous post:The utils::lua library provides thin C++ wrappers around the Lua C API to ease the interaction between C++ and Lua. These wrappers make intensive use of RAII to prevent resource leakage, expose C++-friendly data types, report errors by means of exceptions and ensure that the Lua stack is always left untouched in the face of errors. The library also provides a place (the operations module) to add miscellaneous utility functions built on top of the wrappers.While the RAII wrappers and other C++-specific constructions are a very nice thing to have, this library has to jump through a lot of hoops to interact with binary Lua versions built for C. This makes utils::lua not usable for performance-critical environments. Things would be way easier if utils::lua linked to a Lua binary built for C++, but unfortunately that is not viable in most, if not all, systems with binary packaging systems (read: most Linux distributions, BSD systems, etc.).That said, I've had requests from a bunch of people to provide utils::lua separately from Kyua regardless of any performance shortcomings it may have, and this is what I have started doing this weekend. So far, I already have a pretty clumsy standalone package (I'll keep the name to myself for now ;-) that provides this library on its own with the traditional Automake, Autoconf and Libtool support. Once this is a bit better quality, and once I modify Kyua to link against this external library and assess that things work fine, I'll make the decision on how to publish this (but most likely it should be a separate project in Google Code).Splitting the code doesn't come with its own issues though: maintaining a separate package will involve more work and hopefully/supposedly, dealing with quite a few feature requests to add missing functionality! Also, it means that utils::lua cannot use any of the other Kyua libraries (utils::sanity for example), so I lose a bit of consistency across the Kyua codebase. I am also not sure about how to share non-library code (in particular, the m4 macros for Autoconf) across the two packages.So, my question is: are you interested in utils::lua being shipped separately? :-)  Do you have any cool use cases for it that you can share here?Thanks! [Continue reading]