NODOWS

Etckeeper

2009-12-18T21:21:49Z

I decided to install etckeeper. It went ahead with an initialization after install on my debian machine, which somewhat surprised me, but not a big deal.

Looks like its a fairly standard git repository, but the README says that it has special support to handle changes caused by upgrades and installations - which I think includes hooks for pre and post installations. Sounds to me like it automatically tracks changes. Cool!

How does this effect NODOWS? I'm not quite sure yet, but I think the first step will be to move the NODOWS codebase to git.

Embedded Systems, Git, Etckeeper and more

2009-12-17T07:37:07Z

There has got to be something cool that can be done with embedded systems, git and etckeeper.

I've been thinking about configuration management for some time, and on a totally unrelated topic - bookkeeping - I might have some cool ideas in the works.

It may be that reading and writing are completely different paradigms.

NODOWS Web Interface

2009-12-14T20:17:14Z

Based on the wonderful experience I've had using yui-app-theme with Regdel, I'm planning to use it with NODOWS as well.

In other news, I'm thinking of using Ruby to power NODOWS in general, and switching from Subversion to Git for code revision system.

Now We're Getting Somewhere

2009-11-16T04:35:59Z

It took some serious reworking of the mindeb.xml and the use of an old busybox static binary, but I'm happy to report that NODOWS can now again build an operable (although its very rough at the moment) mindeb image!

Todo:

Autobuild busybox with insmod support (why doesn't the debian busybox-static include insmod??)
Learn about cross-compiling

General notes:

Kernel 2.6.30 uses ide-gd_mod.ko instead of ide-disk.ko
Squashfs is now in the mainline kernel tree
Aufs2 has a ways to go before it gets mainlined, but patches work at the moment

Where I'm Stuck

2009-11-15T17:28:50Z

Its difficult to stop working on a project for 9 months in mid-stride and then pick it up again and remember where I left off.

Right now I'm trying to get mindeb to build and I'm stuck trying to get grub to install properly. For some reason, it looks like the last few builds I successfully made back in February of 2009 depended upon a copy of a disk image I backed up before removing the backup target from the mindeb.xml build file. Doh!

I think that I need to:

Turn off squash
chroot-setup
rsync-temp (restoring this back into mindeb.xml)
cleanup
Turn on squash
chroot-setup
Run qemu / kvm to install grub
finalize

Hopefully that is all that's needed to get mindeb image builds working again.

UPDATE: All that might not be necessary. I might be able to just copy the grub files from the squash image, before it is squashed, that is! I do not want to require two different manual paths for squash and non-squash builds!

UPDATE 2: I still haven't figured out what I want to do about the backup image, but I've also come to realize that for mindeb I ended up taking a different approach when I last left development on NODOWS: the use of debootstrap and make-kpkg (from kernel-package). The target names are "go-go-ve" and "go-no-ve". Several packages have changed / evolved from lenny to squeeze. I've updated those and am continuing to try out the builds...

Finally Getting Back to NODOWS Work

2009-11-15T00:13:58Z

Today I finally found an iota of time to resume work on NODOWS. Unfortunately I spent a bunch of time realizing that gnu-fdisk does not play well with command line scripting.

Now I'm focusing on just getting back up to speed, testing out mindeb builds, and continuing to move more development notes from the Trac wiki to Ikiwiki.

UPDATE: I've made it through the phing build process for mindeb, but so far I've been unable to get an image that will boot with kvm or qemu. I definitely forgot to do the grub-setup, but I just tried it and it didn't work, so I must have forgotten something else. In the process, I recall that chroot-setup is an important task!

New Thoughts and Notes on NODOWS

2009-09-29T09:51:36Z

I haven't done much work on NODOWS lately, but I continue to think about its evolving design.

A couple of thoughts:

I'd like to be able to easily configure a NODOWS box remotely, using a GUI powered by another machine. I'd like to be able to transfer the data over a serial connection, to avoid networking mistakes. XML-RPC might be fine for some simpler configuration details.
NODOWS should focus on performing a simple goal, such as running web applications which conform to a particular design, for example.
NODOWS will not be at the "bleeding edge" of core OS software, but will focus on operating system stability. It will also include up-to-date application software, such as NGINX, libfcgi, and interpreted language processors (ruby, python, etc.)
I want to add NetBSD to the list of supported platforms

Serious Flux

2009-03-01T05:48:43Z

Note to interested parties: NODOWS is going through a lot of flux lately, and I'm probably not going to get to normalize things for a little while as I'll be working on a few different projects. I don't want to curb your enthusiasm, but not might not be the best time to dig into NODOWS.

T2 SDE

2009-02-28T15:37:58Z

Its amazing how trying new things can lead to unanticipated and wonderful discoveries!

Simply by investigating PureDarwin a little bit, I just found out about the T2 SDE project - which appears to be a quite mature project with similar goals to NODOWS (though I think much grander and wider).

qDecoder and qhttpd

2009-02-23T00:00:49Z

In researching cgi/fcgi libraries for use with NODOWS, I came across qDecoder. This project is sincerely a hidden gem. I'd never heard of it before, but I wish I had. Its simple and straightforward, well thought-out and well documented.

So what is it? Its a C/C++ cgi / fcgi library. It includes a lot of functionality, and most of it would be very useful to almost any project. There were two functions that I needed that weren't included in qEntry.c, so I added them, and will submit them back to the project for their review. Another thing that makes this project a winner: the code is licensed under a BSD license. Its strange, but so many of the C cgi libraries and code I found had no license at all!

Because of its efficiency, simplicity, and its capabilities, I think that qDecoder is a perfect companion to NGINX, libxml2, and libxslt.

qhttpd
The qDecoder project also includes qhttpd, a web server with Lua hooks. I haven't explored qhttpd as much as qdecoder, mainly because I'm so pleased with NGINX, but I am interested in how it hooks up with Lua. In fact, for the time being, I'm holding off on integrating qDecoder into NODOWS and using Lua instead. Nothing against qDecoder - I'm basing the decision on convenience. Most of the Lua libraries I'm interested in are available in the stable debian packages, so that's the decision maker.

Once the NODOWS design becomes more specific, I may return to qDecoder to implement more of its main components. In the meantime, I definitely recommend checking it out. You might be very impressed!

Lua

2009-02-19T02:19:18Z

So lua appears to be a rocking good choice for NODOWS. Its tiny, fast, and has a lot of cool packages and libraries available for it, so the choice is an easy one. There will be a couple of different setups - flat XML storage for data, cdb for big key value pair databases, and mysql/sqlite/postgres for more complex data. Since NODOWS is meant for low-power devices, its likely that only sqlite will be supported locally. MySQL and Postgres would be hosted elsewhere and accessed via TCP.

NGINX is also turning out to be a really terrific solution for NODOWS. The xslt module is amazing, and paired with lua via FastCGI, its a great combination.

cgihtml

2009-02-15T17:11:59Z

Now I'm emailing the author of cgihtml asking for a clearer license than "e-mail ware".

Hi Eugene,

I found your cgihtml library and like it very much. I'm thinking of usingit for an open source project I'm working on called NODOWS which is atoolkit for building embedded appliances. I'm trying to keep the weboperator's interface as lightweight as possible, using xslt for rendering,and some sort of C library for parsing POST requests and handling uploads.

I notice that your cgihtml code is available as e-mail ware. Would it bepossible for you to release it under a more specific license like the GPLor BSD license?

Collect.c / collect.cgi

2009-02-10T23:50:59Z

I'm hoping that the author of collect.c will release the code under the BSD license for use with NODOWS, so I've written him an email:

Hello,

I am interested in using collect.c in a project I'm working on called NODOWS. Its an open source tool set for building embedded projects like pfSense, m0n0wall, and Askozia.

http://www.nodows.com/blog/

The file I'm looking to use is:

http://www.cs.tut.fi/~jkorpela/forms/collect.c

It would be excellent if you would consider licensing this code under a permissive license like the BSD license.

Its pretty much exactly what I need, though I've slightly altered it to allow passing the filename to write a part of the form data. With jQuery and a modified json2xml.js script, form data is serialized into xml and then collect.cgi is used to write it to a file, which can then be parsed with xslt.

Let me know your thoughts, I appreciate you sharing the code even for educational purposes!

Grrrrr

2009-02-09T23:34:35Z

I spent a few hours last night figuring out how to get linux kernel 2.6.18 compiled with these three patches: aufs, squashfs, and openvz. I had previously gotten these to work with linux kernel 2.6.26, but then I discovered there is a bug (not sure where) preventing VZ containers from being NFS clients.

And today the USB wireless adapters that I ordered arrived.

Unfortunately, I don't believe that 2.6.18 has support for the wireless adapters. Doh!!

I guess my next step will be to find if I can use an iSCSI device as an storage area for a VZ container. That should work fine, but I need to check it anyway.

Alternatively, I've compiled 2.6.28.4 with aufs, squashfs, and the vserver patches. It compiles but I haven't tried it out yet.

Back to Unionfs for Custom Kernels?

2009-02-06T04:28:53Z

Aufs has been good so far, but unfortunately when I customize the debian kernel, I'm then not able to use the pre-compiled kernel modules. Yes, that includes both squashfs and aufs, but thankfully the squashfs patches are super easy to apply.

The aufs patching process is beyond my comprehension at the moment. It uses something called splice which I'm not familiar with, yet!

NO WAIT! I found a Makefile which automagically creates a patch, it works. :-) Its a little tricky though:

tar -xjf linux-2.6.26.tar.bz2cd linux-2.6.26make menuconfigmake include/linux/version.h include/linux/utsrelease.hcd ../aufsmake patch_cleanmake KDIR=/path/to/linux/source patchcd ../linux-source-dir/patch -p1 < ../aufs/patches/aufs.patch