Adventures as Me

For two years now my testing platform has consisted of half a dozen physical machines (running a mixture of CentOS Linux and FreeBSD) and two powerful servers running VMware server. During that time I've had a couple instances to introduce Xen into the mix, but never on a permanent basis, usually due to time constraints. With two people my department now, I've had the opportunity to setup a more permanent Xen server. The Host itself is modest: AMD 3800+, 3 GiB RAM and A few hundred GB of Drive space.

The VMware server experience has soured me on the OS-in-a-loop-back-file method. Yes it can support LVM partitions, but only with some trickery, which I don't want to do. Xen, however, needs no trickery.

With the help of LVM and images from jailme.org, I've been working out a nice routine which I will eventually automate. The process itself is rather simple:

1. Create a new LVM partition for the OS ( e.g. lvcreate --size 20G -l OsName GroupName )
2. Create a new LVM partition for swap ( e.g. lvcreate --size 1G -l OsSwap GroupName )
3. Format the OS partition ( e.g. mkfs.ext3 -L OsPart /dev/GroupName/OsName )
4. Format the Swap partition ( e.g. mkswap /dev/GroupName/OsSwap )
5. Mount the Jailme.org image ( e.g. mount -o loop image.img /mnt )
6. Mount the OS Partition ( e.g. mount /dev/GroupName/OsName /xen )
7. Use rsync to copy the image contents to the OS Partition ( e.g. rsync -aP /mnt/* /xen )
8. Make changes to some configuration files on the OS Partition
9. Unmount the image and OS Partition
10. Customize the Xen configuration file
11. Start the image

My requirements only necessitate having a few setups like the above. My goal in automating the process is not just to remove the tedium, but to improve the speed. With a rough estimate, I think the above could be automated to occur in less than 10 minutes (accounting for load on the system).

Once that is automated, my next goal is to tackle taking 'snapshots' of the Xen guest. I'm still undecided as to the exact method, although I am currently leaning toward creating a tarball of the partition.

After a very long hiatus, I'm working on Loquacity development again. This past year has taught me something: don't shoot for the stars. Having a road map is very useful, but it must be realistic according to the resources at hand.

• cleaning the current code
• fix known bugs
• removing code/features that don't work
• make the installer work near flawless
• release

Along with the above, I need to remove people from the project that never did anything. This is more a management type thing, only on Berlios.

To start, I've fixed a nasty bug in the Comment admin interface. The SQL statement to retrieve the comments was very broken; a junior level mistake that caused a match of LIKE "%". Of course, that won't work.

Deleting comments is much faster now, as some redundant queries were removed.

In the interests of improving the speed of Loquacity, I've begin keeping a list of Database tables that need indexes added. These changes will be implemented in the Upgrader and base Database schema.

One of the problems inherent to using an Apache Module in a shared web hosting environment is the fact Apache modules gain all the permissions of Apache. Hence, using something like PHP as an Apache module can create serious security vulnerabilities. Since the PHP scripts run in the Apache context, users can easily cause problems (e.g. peeking in other users' directories, overwritting other peoples files, etc). The easy way to solve this issue is to run PHP as CGI only. This means a user's scripts will run in the context of the user's permissions, rather than Apache's.

Running as a CGI presents other problems. For example, the majority of 3rdparty PHP Applications assume one is running PHP as an Apache module. To use a script via CGI, one has to use something called a she-bang. The first line in the script is a special line that points to the binary that actually executes the script. Thus, if the PHP CGI binary is /usr/bin/php, then the first line in the script must be:

Another (mainly superficial) problem is often CGI scripts must be placed in a cgi-bin directory. This means the URI to your blog won't be www.example.com or blog.example.com but www.example.com/cgi-bin/blog.php or blog.example.com/cgi-bin/ Not exactly pretty. There are ways around that particular problem. There are still bigger problems, even running PHP as a CGI binary: how to lock down the user? You are essentially giving the user a lot of system access (the same is true when running Perl, bash, Ruby and other Scripting languages), how can we prevent a user from stomping all over the system? Even as CGI, the scripts are still executed as the Apache user.

Two common ways are PHPSuExec(*) and SuPHP. Both provide means of changing the UID/GID of the executing process to that of the end user, and many, many other safeguards. However, they both fail on one common problem with PHP: by default, PHP allows users to place their own php.ini file in their Document Root, which will be used to the exclusion of the system php.ini Thus if you have disabled various functions (e.g. dl, system, passthru, etc), a user can easily override it by dropping a blank php.ini file in his Document Root (typically ~/public_html)

There are several ways to disable user php.ini files.

• Configure php with --with-config-file-path=/usr/local/lib --with-config-file-scan-dir=/usr/local/lib
• ( Won't work with PHP 4.4.x) Install the Zend Optimizer
• ( Won't work with PHP 4.4.x) Create a wrapper script

As noted, only the first option works for all modern versions of PHP. The directory path provided should match where your system php.ini file is located. Once compiled with these options, PHP is essentially "jailed" into using a php.ini file where configure stipulated. One odd thing to note with this, phpinfo() will lie. Try the following in a user account:

• In your user Document Root directory, create an empty file named php.ini\
• In a separate file, place <?php phpinfo(); ?>
• Access the phpinfo file via your web browser
• Note that phpinfo() reports it is using the php.ini file located in your Document Root

An easy way to verify which is being used is to disable a function in the system php.ini file (such as ini_get). In the user php.ini, override that (disable_functions=). Create a short script that tries to use the disabled function. You'll receive an error stating the function is disabled for security purposes.

What about the other methods? In PHP 5.2.x, installing the Zend Optimizer forces PHP to use the one stipulated in the Zend configuration, thus overriding the user php.ini file. And the wrapper script?

Locate your PHP CGI binary (we'll use /usr/bin/php as an example) and rename it to php-cgi (thus /usr/bin/php-cgi). In the same directory, create a file named php. Give it the same permissions as php-cgi and insert the following contents:

#!/bin/sh
/usr/bin/php-cgi -c /usr/local/lib/php.ini

The -c parameter instructs PHP to use the configuration file stipulated and ignore all others. Astute readers/users will notice this doesn't filter out the possiblility of a user passing -c/path/to/user/php.ini If you are concerned with that, a script similar to the following will work (as contents of /usr/bin/php):

#!/usr/bin/perl --

use strict;
use warnings;
die '-c not allowed' if grep { $_ } map { $_ eq '-c' } @ARGV;
exec( qw(/usr/bin/php-cgi -c /usr/local/lib/php.ini), @ARGV );

Now, if a user attemps to pass -c/path/to/user/php.ini to the script, it will die. If done via a web-accessible script, Apache will return a 500 error and the die message will be in the error_log The use of Perl is just an example, you could do something in BASH, Python, C or whatever. Unfortunately, PHP 4.4.x ignores -c when compiled as a CGI binary, hence this trick only works on PHP 5.

Back in 1998 I took my first job in "the industry": an EDI Clerk/IT Assistant. Being completly self-taught, from the corner or nowhere, I was extremely happy to have the job. It offered potential to learn about the IT field and would give me real-world experience, not only in solving problems, but dealing with users in a manufacturing environment.

Soon after being hired I was introduced to "Sears day." This referred to every Wednesday when the Company shipped the orders to Sears, and only Sears. At that time, and still today, Sears has some of the worst Shipping requirements of the large retailers. It was nothing for them to generate 300+ 2 line orders, with a total value of ~USD$50 per order. To process the orders, from receiving them to closing the door on the truck, was very taxing. True, part of the processing was like ever other order, but there were aspects of the Sears orders that were unique.

For example, the Bill of Lading required a manifest, which was merely a list the cross-indexed each order with the Distribution Center (DC) that would receive the order. The stack of orders needed verified with the Manifest to be certain each was routed properly. This was done by comparing the order number and DC number as printed on the order, with the same information printed on the Manifest. The comparison was needed because the manifest was keyed to the Bill of Lading number. The Bill of Lading number was not known at time of order, but rather generated (read: invented by the Shipping Supervisor using a consistent pattern) when scheduling the pickup with the Carrier.

That was not the only comparison, two additional reports were generated, each needing data matched with the order, all to ensure correctness. To make matters worse, the entire cross-referencing was performed manually. It would take four people, myself, the shipping clerk, the IT Manager and the Assistant Plant Manager, the full ten hour work day, at minimum, to perform the entire process. Something needed done, the entire process was error prone, highly repetitive and consumed too many resources.

Taking stock of the situation, I considered the tools available. My workstation used DOS 6 and Windows 3.11. Our ERP (Enterprise Resource Planning) system was accessed via a Terminal application. Joy of joys, the Terminal application, wIntegrate could be scripted. The script file was a simple text file containing the raw text commands one would actually perform when using the ERP System. All I needed to do was find a way to take the order numbers and generate the necessary script.

The problem was there were no development tools available, save two: DOS Batch and MS Word. Being young, green really, I chose the lesser of two evils: MS Word. On reason I chose it was for the eroneous view 'If someone else needs to use this program, they will be more comfortable using MS Word than typing a command on the commandline.' A noble thought in a sense, but ultimately wrong

1. A list of order numbers, newline separated (\r\n on DOS/Windows)
2. The Bill of Lading number
3. The Carrier SCAC
4. Quantity shipped on the order (this was simple as all orders had the same quantity)

the "program" generated a script which processed every order and generated the necessary reports. With that automated, the cross-referencing could happen sooner since the majority of resource consumption was in the data-entry.

After performing a few test runs, I was ready for the next Sears Day. Wednesday came, I took the stack of orders and went to my workstation. Earlier in the week I had generated the text file with the order numbers. Using a variation of my VBA Program, I had created a script that automated the generation of the shipping labels. Its successful execution told me my Program was capable of creating wIntegrator capable scripts. As everyone bustled around beginning their work day, my work station was busy feeding the script to wIntegrator.

Within 10 minutes every order was processed and the reports were generated. Printing all the reports took another hour, cross-referencing, another hour (you can really tell now what took the bulk of time: order-entry). By first break (9:30), the entire process was done. Around the same time my boss came rushing into my office. She had just remembered it was Sears Day. One can easily image the surprise she experienced when I told her everything was done. After a few minutes of checking everything herself, she left.

We continued using that system the rest of the shipping season. The following year they replaced the old Unix-based ERP with a newer system, one that was far more data-intensive and not scriptable. Plus different routing requirements caused further disruption to the previous flow rendering my continued automation worthless.

Joel presents a very fine analysis of the situation: should highly visible bloggers accept gifts?

His conclusion? No, because such gifts amount to a bribe. With this I agree. Afterall, the same metric is often applied against other people in 'positions' of trust. Public Officials, for example must follow explicit, detailed guidlines when accepting gifts, especially gifts that are intended to sway the opinion of the receiver, or those the receiver has dealings with.

However, one scenario makes me wonder? What if the Microsoft laptop arrived on Mr. Spolsky's doorstep unannounced, without accompanying instructions. To me at least, this is a different dynamic, a gift without strings, if you will. Under such circumstances, I could see acceptance as being ethically valid. Of course, who could expect a Public Relations to do something like that?

A new feature available in Loquacity alpha2 is automatic backups. The data in your database, along with your configuration file, will be archived:

1. During software upgrades
2. Periodically

That should provide plenty of safeguards for a person's data, as long as the backup is downloaded.

Later I want to extend the backup management to allow 3rd parties to provide plugins that allow a person to backup other items. For example, if a 3rd party creates a photo album plugin, part of the plugin could be a module that causes the backup to also include the photo albums.

My workplace recently bequeathed a G5 Mac to me, my first "real" Apple computer. With it came two mice:

• A mighty mouse
• A wireless mighty mouse

After using the Mighty Mouse for a couple months, I can honestly say this is the worst mouse design I've ever used. Oh, don't get me wrong, aesthetically, the mice are very appealing. My Microsoft and Logitech mice cannot compete with the smooth curves and the simple lines of the Mighty Mouse. The Mighty Mouse is somewhat comfortable, albeit a little on the small side However, the Mighty Mouse is a very poor performer. What do I mean by this?

The Mighty Mouse is not designed to be functional. Consider the buttons. It has a total of four buttons:

1. Left Mouse Button (LMB)
2. Center Mouse Button (CMB)
3. Right Mouse Button (RMB)
4. Side "Squeezable" Button (SSB)

The first three require precise placement of the fingers inorder to work properly, except for the left mouse button (note: I'm using the mouse right-handed). More often then not, when I attempt a RMB Click, the mouse produces a LMB Click. Only after several attempts do I obtain a RMB Click. This also happens, although with less frequency, when I attempt a CMB Click. It's always the LMB Click the mouse generates. It happens with both mice. The SSBs are merely a joke. Attempting to use them is an exercise in finger contortions that usually result in generating LMB or RMB Clicks before a SSB Click.

After more than two months of this, I'm returning to my Logitech mouse, which never gave me such issues. Please Apple, make your hardware functional as well as aesthetic.

Wow. I just experienced a major bug with Loquacity and it cost me a post. Apparently the Edit Post function is seriously broken. It destroyed all the record data in the database.

--edited November 6, 2006 06:10 PM

Just a quick note that this blog is now using the pre-release version of Loquacity-alpha2. The only items needed accomplished to release -alpha2 are:

1. Timezone handling
2. OpenID client
3. TypeKey

Items 2 and 3 will appear as plugins while item 1 will be integral to Loquacity.

During the past few weeks, I've had opportunity to install more than my fair share of GNU/Linux distributions. Indeed, likely I installed more distributions the last three months than the prior 10+ years of using GNU/Linux. The distributions installed are:

• cAos i386, x86_64
• CentOS 3 i386, x86_64
• CentOS 4 i386, x86_64
• Fedora Core 1,2,3,4,5 i386, x86_64 (for 3,4,5 only)
• Debian 3.1
• Debian 4-testing
• Mandrake <=2005
• Mandriva 2006 i386, x86_64
• WhiteBox 3 i386
• Trustix 2 i386
• Trustix 3 i386, x86_64
• RedHat 7.3
• RedHat 9
• RHEL 2.1, 3, 4 i386
• RHEL 3, 4 x86_64
• SuSE 10 i386, x86_64
• SLES 9, 10 i386, x86_64

Also, I've installed:

• FreeBSd 4.11
• FreeBSD 5.4 i386, x86_64
• FreeBSD 6, 6.1 i386, x86_64

Each I installed at minimum once, some two or more times. The goal for each install was a workable server, with minimal software, absolutely no Vendor packaged server applications (Apache, PHP, Exim, etc). Developer tools were installed. No GUI, graphical stuff like X.org was installed. Each had the same partition/slice arrangement:

sda -----
        +--tmp sda1
        |
        +-swap sda2
        |
        +-- sda3

These servers are for testing purposes only, thus the somewhat strange requirements. Following are observations on some of the installers:

CentOS/Fedora/RHEL

Easy to use. Nothing hidden. Great for installing whether using the graphical or text mode installer. Allows near full control over everything installed, including disabling the Firewall and SELinux, which is great if you have your own configuration/preference for such things.

The update tool was almost fully configured at installation end. GPG keys needed imported and anything handled by the Legacy project needed reconfigured.

Mandrake/Mandriva

Likes to hide as much as possible. The Text mode installer is abysmal. When partitioning a disk, when finished tehre was no obvious means of progressing to the next step. If you don't have a floppy drive (or handy floppy), don't try to save the partition information as the installer blindy tries to write to the floppy and throws continual errors requiring a restart of the system.

Their security stuff (msec) is impossible to disable during install. You have to do it after the first boot (msec 0). The Corporate Server does not include the ability to install Developer Tools during installation (the Developer Tools package only installs LSB), requiring a manual install after first boot). The Graphical installer tells you little information about partitions. The partition creation tool uses a slider to change the partition size. This is extremely inaccurate and slow.

The update tool, urpmi, requires configuration after first boot in order to download updates.

Debian

Allows control over nearly every aspect of the installation. I'm slightly biased because Debian is my preferred distribution. The installer is easy to use and keeps getting better and more flexible. One annoyance, the installer requires a user be created. It will not progress unless that criteria is met.

The update tool was fully configured after install, ready to use.

cAos

Very easy to install, despite their unique installer: sidekick. No complaints here. Adequate control over everything during installation. The update system was ready to use by the end of install.

FreeBSD

The install is very easy to use, but very fragile. The slightest thing, such as inability to access an FTP site, placed the installer in a broken state, requiring a reboot and restart of the process. It seems the newer version of FreeBSD (5.5, 6.0, 6.1) have this problem. Other than that, one has full control over the process, enabling and disabling what one wishes.

The package management system sometimes needs configuration after install. Ports often need upgraded.

WhiteBox 3

The oddest of the odd. It is based on RedHat, so should work the same as CentOS, Fedora Core, etc. But it has issues. I counted a minimum of 10 times the installer had me switch between CD 1 and CD 2, for a minimal install. I stopped counting after 10.

After those notes, I leave you with this: the Mandriva installer should be scrapped. It is terrible. It's one thing to hide functionality to make the process smooth, its another for the functionality to not exist (why can't I disable msec during install? why is urmpi not configured? No, configuring to use the CDs is not adequate).SuSE is not better in that regard (aside from SLES) as you is configured to use the CDs. Bad, very bad.

And that is my opinion, after umpteen installs.