• Forget about test(1)'s == operator

    Some implementations of test(1), in an attempt to be smart, provide non-standard operators such as ==. Please forget about those: they make your scripts non-portable and a pain to use in other systems. Why? Because, due to the way the shell works, failures in calls to test(1) will often just result in an error message (which may not be seen due to other output) and the script will happily continue running even if it missed to perform some important operation.So... just use the standard equality operators:= for string equality comparison.-eq for numeric equality comparison.Note that whenever I refer to test(1), I'm also talking about the [ ... ] construction in conditionals.Also, please note that this also affects configure scripts, and the problem in these appears much more commonly than in other scripts! [Continue reading]

  • Always define an else clause for portability #ifdefs

    If you use #ifdef conditionals in your code to check for portability features, be sure to always define a catch-all else clause that actually does something, even if this something is to error out.Consider the following code snippet, quoted from gamin's tests/testing.c file:if (arg != NULL) {#ifdef HAVE_SETENV setenv("GAM_CLIENT_ID", arg, 1);#elif HAVE_PUTENV char *client_id = malloc (strlen (arg) + sizeof "GAM_CLIENT_ID="); if (client_id) { strcpy (client_id, "GAM_CLIENT_ID="); strcat (client_id, arg); putenv (client_id); }#endif /* HAVE_SETENV */}ret = FAMOpen(&(testState.fc));The FAMOpen method queries the GAM_CLIENT_ID environment variable to set up the connections parameters to the FAM server. If the variable is not defined, the connection will still work, even though it will use some default internal value. In the test code above, the variable is explicitly set to let the tests use a separate server instance.Now, did you notice that we have to conditional branches? One for setenv and one for putenv? It seems reasonable to assume that one or the other must be present on any Unix system. Unfortunately, this is flawed:What happens if the code forgets to include config.h?What happens if the configure script fails to detect both setenv and putenv? This is not that uncommon, given how some configure scripts are written.What happens if neither setenv nor putenv are available?The answer to the three questions is: in the above code snippet, the code builds just fine but will misbehave at run time: neither HAVE_SETENV nor HAVE_PUTENV are defined, so the code will not be able to define the required environment variable. However, FAMOpen will later be called and it will not behave as expected because the variable has not been set.Note that this code snippet is just an example. I have seen many more instances of this exact same problem with worse consequences than the above. Read: they were not part of the test code, but just part of the regular code path.So how do you implement the above in a saner way? You have two alternatives:Add an #else clause that contains a fallback implementation. In the case above, we could, for example, prefer to use setenv if present because it has a nicer interface, and fall back to putenv if not found.This has a disadvantage though: if you forget to include config.h or the configure script cannot correctly detect one of the possible implementations (even when present), you will always use the fallback implementation.Keep each possible implementation correctly protected by a conditional, but add a #else clause that raises an error at build time. This will make sure that you never forget to define at least one of the portability macros for any reason. This is the preferred approach.Following the second suggestion above, the code would get the following structure: #if defined(HAVE_SETENV)setenv(...);#elif defined(HAVE_PUTENV)putenv(...);#else# error "Don't know how to set environment variables."#endifWith this code, we can be sure that the code will not build if none of the possible implementations are selected. We can later proceed to investigate why that happened. [Continue reading]

  • Where does Gnome use file monitoring?

    One of the readers of my post yesterday, wonders where Gnome uses the file monitoring APIs. Well, the answer is: everywhere.Here are some examples:Nautilus monitors all open folders so that it can update their contents whenever the underlying file store changes. Say you are viewing the Documents folder and you save a new Document from within OpenOffice into that folder. You definitely want Nautilus to show it immediately, without having to manually hit Refresh from the menu.The trash applet monitors the trash folders and updates its icon from empty to full whenever one of these folders ceases to be non-empty.The panel monitors the applications directory to notice when new applications get installed. This allows it to update the Applications menu immediately as soon as a new program gets installed into the system.The GTK file open/save dialogs monitor the directory they are viewing for the same reason as Nautilus. (Actually, I'm unsure about this point. My Linux installation is an old Ubuntu 8.04 LTS that does not have GIO, so I can't verify in recent ones. However, this makes perfect sense and if not implemented, it should be!)The background switcher control panel monitors the folders containing images to be able to show new installed backgrounds. (I'm not sure about this either. It doesn't happen in my Linux installation, but it also makes sense.)Media players such as Rhythmbox and Banshee allow you to point them to a folder containing music and have an option to automatically add music to the library as soon as it pops up in such folder.Potentially, any document editor, picture viewer, etc. monitors the documents opened in them so that these applications can notice external notifications to those files. This is useful to prevent overwriting a file with an out-dated in-memory copy. For example: you are taking some notes with GEdit. On a separate terminal window, you quickly edit the notes file with Vim to add a new note. When you go back to GEdit, you want the editor to tell you that the file has changed out of its control and offer you a choice: e.g. reload or ignore?And many, many more other situations that I'm surely missing...As you can see, it is fairly important to get the file monitoring subsystem working flawlessly. Otherwise, all these tiny details don't work and the end user experience is undermined. [Continue reading]

  • New gio-fam package

    As briefly outlined in the previous post, new versions of Glib provide GIO, a library that intends to be a low-level file system API on top of the POSIX interface. This library provides an interface to asynchronously wait for file system change notifications including the creation, deletion and modification of files.The monitoring functionality in GIO is modular: it is backed by different loadable plugins that implement OS-specific functionality. In particular, GIO uses an inotify module in Linux and a FAM module everywhere else.Up until now, the devel/glib2 package in pkgsrc provided a build-time option to specify whether to build the GIO FAM plugin or not. Given that this plugin is built as a shared object that is loaded dynamically at run-time, having a build-time option for this is clearly wrong: it gives no choice to those relying on binary packages (e.g. end/new users). Furthermore, it adds a dependency on the ugly-FAM at the very bottom of the huge Gnome dependency chain. (As already stated, FAM is outdated and hard to set up.)So, based on this, I've just removed all FAM support from devel/glib2 altogether and packaged its loadable module as sysutils/gio-fam.Now waiting for a clean rebuild of the Gnome packages to see if the desktop now works on my machine by avoiding FAM/Gamin. [Continue reading]

  • File system monitoring, Gnome and NetBSD

    A few days ago, I decided to start using NetBSD, as well as Gnome on NetBSD once again, mostly because the lack of their use makes my skills feel rusty in many different areas. While NetBSD has surprised me in a good way (I am running it on a Macbook Pro and things like wireless and DRI work), Gnome has not. There are tons of broken things that prevent a smooth user experience.One of these broken things is the monitoring of changes in the file system. Actually, this has never worked 100%. But what is this and why does it matter, you ask? Well, file system monitoring is an internal component of the Gnome infrastructure that allows the desktop to receive notifications when files or directories change. This way, if, say, you are viewing the Downloads folder in Nautilus and you start downloading a file from Epiphany into that folder, Nautilus will realize the new file and show it immediately without requiring a manual refresh.How to monitor the file system depends on the operating system. There are basically two approaches: polling and asynchronous notifications. Polling is suboptimal because the notifications are usually delayed. Asynchronous notifications are tied to the operating system: Linux provides inotify, NetBSD provides kqueue and other systems provide their own APIs.In the past, Gnome monitored the file system by a combination of FAM, a system-level service that provides an API to file system monitoring, and GNOME VFS, a high-level layer that hides the interaction with FAM. This approach was good in spirit (client/server separation) but didn't work well:FAM is abandoned.Does not support kqueue out of the box.FAM runs as root.FAM is too hard to set up: it requires rpcbind, an addition to /etc/services, a sysctl tweak, and the configuration of a system-level daemon.To solve some of these problems, a drop-in replacement for FAM was started. Gamin, as it is known, still does not fix everything:Gamin is abandoned.Supports kqueue out of the box, but does not work very well.Actually, Gamin itself does not work. Running the tests provided by the distfile in a modern Linux system results in several test failures.Anyway. Did you notice the abandoned pattern above? This is important: in the new world order, Gnome does not use FAM any more.The new structure to monitor files is: the low-level glib library provides the gio module, which has some file system monitoring APIs. The GVFS module provides higher level abstractions to file system management, and relies on gio for file system monitoring. There is no more GNOME VFS any more.The key point is: gio uses inotify directly; no abstraction layers in between. FAM support is still there for platforms without inotify, but as it is not used in Linux any more, it rots. Linux developers will never experience what it is to have a system that needs to use FAM to get this functionality to work.At last, let's look at the status of all this in NetBSD:The FAM package was patched to support kqueue. Although this kinda works, it is not perfect. Also, as mentioned above, FAM is, I'd say, the package with the hardest installation procedure of the whole Gnome platform.The Gamin packages are nicer than the FAM package regarding their configuration. However, when using Gamin instead of FAM, all sorts of bugs appear in Gnome (it actually gets stuck during startup for me). The breakage of the unit tests does not provide any confidence, and the fact that Gamin is abandoned, the idea of fixing it doesn't make me thrive.The glib2 package depends on FAM. This is ugly; really ugly. I had to shout WTF when I saw this, seriously.Seeing the direction gio/gvfs take, it is obvious that things can only get worse in the future.If time permits, I'm planning to work on improving this whole situation. Ideas include:Splitting the FAM gio module out of the glib2 package. Ideally, this would happen upstream.Implement a gio backend for kqueue.Check if the core packages still using gnome-vfs have a more recent version that uses gvfs instead and, if so, update them.Can't promise you anything other than, if I get to work on it, I will keep you posted! [Continue reading]