Floodgap's Power MachTen Hacking Page: Base Recommended Software and Porting Packages to Mach Ten

This is a primer and step-by-step to getting your system to what I consider a "contemporary minimum level" of functionality and security. It also includes some notes on how you might port future packages and software to this environment. By the time we are done, you will have a new(er) perl, updated zlib, OpenSSL and OpenSSH, and lynx. After that, you might want to move on to rolling a graphical browser, but these steps are prerequisites for that too.

I assume that you are handy with a Unixy command line. If you are not, a lot of this will go over your head. If you're just looking for suggestions or user tips, look at the mad skillz'n'at page. If you have your own step-by-steps, you can send them to me at ckaiser@floodgap.com for inclusion in a future edition of this document.

General notes about porting and compiling (and some bad news)

YOU SHOULD READ THESE PARAGRAPHS VERY CAREFULLY BEFORE YOU BEGIN!

Let's do the bad news first: you are stuck with gcc 2.8.1, which is what comes with MachTen out of the box. You cannot update it any further; gcc 3 and 4 do not compile at all. Even after I hacked their build environments to be compatible, they fail make bootstrap. This means that many current packages which assume everyone has gcc 3 or better (fltk2 and gtk+, I'm looking at yooooouuu) cannot be built, and by extension anything that needs them as a dependency (i.e., just about any recent toolkit-based X app). You can still build a lot with gcc 2, as we'll demonstrate, but there is a limit.

Here is some more bad news: relatively recent GNU configure/autoconf suites will cause a kernel crash when attempting to determine the maximum length of a command line. This appears to be due to running out of stack space, but because of the way stack space is runtime-allocated (more about that in a moment), the simplest solution is merely to skip the test. If grep maximum.length config* shows that your package has this poison test, you need to patch it. Find lt_cv_sys_max_cmd_len, and then find the default *) section. Immediately before it, add

machten*)
	lt_cv_sys_max_cmd_len=8192 # probably underestimate
	;;
and throw an exit into the default *) section just in case you do this wrong. You can then safely run ./configure.

Speaking of stack space, very complex applications require more stack space than MachTen allocates them by default (because they're probably over-using automatic space instead of malloc()), and annoyingly, this stack space allocation is static for a given executable. Although the default 36K stack seems rather stingy, most of the time this works fine; however, when you start getting lockups in an application you can't code around, expanding stack space is the next option. Unfortunately, this tends to be a tedious cycle of trial and error to find an allocation that lets the executable run without whacking the kernel or sucking RAM and there's no easy way to do it short of trying settings (and rebooting a lot) until it sticks. I suggest throwing in a few megabytes of stack and then cutting it down until you think your luck is running out, or it actually does run out. To build a custom stack space setting into an application, pass ld an -Xlstack=bytes option (or put it into your Makefile's LDFLAGS). You can run an already built executable with a custom stack size with the setstackspace tool (setstackspace bytes file), or watch its total allocation on the console by setting the STACKCHK environment variable to 1. For an example of an application requiring beaucoup stack space, see Mosaic-CK.

Although MachTen 4.1.4 does support dynamic linking, it does not seem to be fully operational. Unless you have a good reason, you should build executables statically linked. We will be doing that here.

Software that assumes // is the same as / will fail on MachTen, because they are different (for example, // is the HFS root). If you get spurious "file not found" errors, this might be the cause; it's arguable who is the buggy one, the package that generates such paths or MachTen for not accepting them. I am chagrined to admit that some of my own software blows up because of this and am currently patching their configure libraries.

MachTen seems to have incomplete support for the set[re][gu]id() family of functions. This is a minefield because sometimes the attempt will fail silently, meaning you might think you have dropped your privileges and you haven't. You may be able to coerce certain applications into compensating for this (OpenSSH) but most of the time you can't (Perl), meaning you should always run services as unprivileged users unless you absolutely can't avoid it, as you should be doing on any system.

Finally, you would be well-advised to back up your MachTen folder before doing a big compile. As I said on the main page, things that thrash memory may cause kernel farts and compilers really thrash memory.

inetd or rc.local? (or cron?)

Most people are used to compiling daemons and slapping them in their local rc files. Great idea: you build superfunctiond and write it into /hfs/etc/rc.local, reboot, and find that MachTen locks up immediately. For safety, I recommend not running startup daemons within rc.local -- although clearly Tenon's own daemons operate in this environment, none of the ones you will build here will, and I don't know why (if you know better, please tell me).

This leaves you with two options for apps/services you want running as soon as you start up. If you can run it from within inetd, you should. For whatever reason, this seems to work just fine.

If you can't, then you should put the apps in root's .login, log in after MachTen boots, and log out. Don't put them in cron because as soon as they try to run from it, they will also lock up the system until you log in from the console. So you might as well just run them from .login; it won't interfere with su from another account. Frankly, you might as well not run cron at all.

To enable inetd, check Enable Incoming Connections in the Networking portion of the MachTen control panel, but that should be the only thing you check -- don't let it run any other services! Also make sure you comment out everything in /etc/inetd.conf until you have installed what you want, and in /etc/rc.conf you should only be running inetd, macmntd and syslogd. That brings us to:

Things you'd be surprised to find, and things you need to replace

I was delighted to discover that it comes with tcsh, and since all you hooligan uncircumcised bash-loving masses won't use it, you can fend for yourselves. (Okay, okay. It also comes with bash out of the box too. Are you happy now?) There is also gunzip and gzip.

Snickers and shell whizzing contests aside, you will need to replace a number of libraries and executables. We'll be replacing zlib and updating perl, and replacing the SSH1-only ssh and sshd. However, many of the services in inetd, not to mention its named and sendmail, are vulnerable to a virtual cavalcade of remote exploits. If you want to use MachTen as a mail server, for example, you will need to compile a new postfix or sendmail; if you want to use MachTen for your DNS, you'll need a new BIND package. These are very large programs to link and I won't get into them here. Until you find a need for that, simply don't run these services and make sure /etc/inetd.conf and /etc/rc.conf are running the bare minimum.

Creating /opt

For the following steps, you're gonna need a bigger boat. Although I'll be paving over some of the old and busted stuff, the new hotness will need a home larger than /usr allows, let alone space to unpack and compile the packages, so we'll make an /opt.

You might want to chown -R /opt to yourself, otherwise you'll be doing the rest of this as root (not advised). You can chown everything back to the system when you're done.

This is probably a good time to put /opt/bin in your path, too.

Getting packages to yourself

Download them onto a computer with a working ftpd. Until we get lynx running, I'd advise you just ftp them from that computer into /opt/src. Unpack them there; MachTen comes with gunzip.

Perl

Perl is a prerequisite for a number of the below tools, and is a needed part of GNU configure, so we're going to start with that. Out of the box, MachTen comes with 5.00501 which should be updated; you should upgrade to at least 5.00504, which we will do here and can substitute directly for 5.005_01.

If you want to build a later Perl, 5.6.2 is known to work, but is the last officially supported version as the fixes for MachTen were removed from the Perl core starting with 5.8.0.

It is possible to successfully compile 5.10.0, but it fails so many base tests that I strongly recommend not using it. I plan to work a little more on that later. If you want to try that, use machten_2 for the hints, do not allow dynamic loading, and do not use Perl's built-in malloc() (use MachTen's). Also, make this change to pp_sys.c line 5151:

#   if defined(HAS_GETSPNAM) && !defined(_AIX) && !defined(__MACHTEN__)
and this change to ext/Time/Piece/Piece.xs starting with line 244:
#if defined(WIN32) || (defined(__QNX__) && defined(__WATCOMC__)) || defined (__MACHTEN__) /* No strptime on Win32 or QNX4 */
/* #define strncasecmp(x,y,n) strnicmp(x,y,n) */
Then run make test and realize you wasted all that time.

Back to Perl 5.005_04, which we will simply install over 5.005_01. Use the machten hints when asked by the Configure script, and read the production notes carefully! The MachTen vfork() is okay and should be used.

sh Configure
make
make test # a couple nonessential tests will fail, see configure's notes
su
make install

Choose your source of entropy

There is no /dev/*random in MachTen (and for reasons I will discuss presently, you should not use the built-in RNGs for OpenSSH or OpenSSL), so you will need an entropy source, and you can choose either egd or prngd. Neither is a great source of random numbers because both rely on system tools as a cheap source of random data and MachTen doesn't have the typical tools to accomplish this, but the alternative is locking up your machine, so we'll go with this. The advantage of egd is that it is in pure Perl, and installs/runs without changes, and gives you Digest::SHA for free, but prngd is faster. I personally prefer prngd but egd will work just fine and I have used both.

egd (0.9)

perl Makefile.PL
make
make test
su
make install
make -f Makefile.aperl inst_perl MAP_TARGET=perl # this replaces your Perl!!
egd.pl /var/run/egd-pool

OR: prngd (0.9.29)

This needs some adjustment. In config.h, add this section before everything else:
#define HAVE_GETRUSAGE
#define HAVE_DAEMON
#define O_RSYNC         0
#define O_SYNC          0
#define SOCKLEN_T       int
#define PATH_TMP        "/tmp"
#define PATH_VAR_TMP    "/var/tmp"
#define PATH_PASSWD     "/etc/passwd"
#define PATH_WTMP       "/var/log/wtmp"    
#define PATH_UTMP       "/var/log/wtmp"
#define PATH_SYSLOG     "/var/log/messages"
#define PATH_MAILLOG    "/var/log/messages"
/* some of this is me being lazy but works well enough */
In the Makefile, uncomment CC=gcc and add
CFLAGS=-O3
Create prngd.conf (add your own?). I left the netstat in even though there will be no TCP state shown when using OpenTransport tunneling because at least you can sample Unix domain sockets that way.
# Format is: "program-name args" path rate
# The "rate" represents the number of bits of usable entropy per
# byte of command output. Be conservative.

"ls -ali /tmp" /bin/ls 0.01
"ls -ali /var/tmp" /bin/ls 0.01
"ls -ali /var/log" /bin/ls 0.02
"netstat -an" /usr/bin/netstat 0.05
"ps auxww" /bin/ps 0.03
"w" /usr/bin/w 0.01
"who" /usr/bin/who 0.01
"last -100" /usr/bin/last 0.01
"df" /bin/df 0.01
Then,
make
su
cp prngd /usr/sbin
cp prngd.conf /etc
# seed the rng at this point, optional
prngd /var/run/egd-pool
While that runs and makes stuff up, install ...

zlib (1.2.3)

zlib is a dependency for a number of things and the version that comes with MachTen is lamentably old and has some security issues. To prevent it from being incorporated into our new builds, we're simply going to pave over the old library. First, back up /usr/include/zlib.h, /usr/include/zconf.h and /usr/lib/libz.a if desired, then:
./configure
make
make test
su
make install prefix=/usr

OpenSSL (0.9.8j)

Make sure your entropy daemon is running! This doesn't install to /opt; I believe crypto should be part of the base operating system, so this will go in /usr. First,
perl Configure gcc
In crypto/des/speed.c (line 92) and apps/speed.c (125) insert,
#define TIMES
#undef TIMEB
and to apps/Makefile, add -ldl to EX_LIBS. Then,
make
make test
If you get warnings about the RNG, your entropy daemon is not functioning. FIX IT NOW. Once it is,
su
make install

SOCKS5 (optional, 5-v1.0r11)

This is up to you, but if you live or work around heterogeneous environments and networks with proxies, you should probably build SOCKS at this point.
./configure --prefix=/opt --with-static
make
An ld warning at the end is normal. I don't recommend make install since it may bork your built-in clients; copy what you need manually to different file names. Leave the libraries in /opt.

OpenSSH (5.2p1)

STOP! SHUT DOWN AND BACK UP YOUR MachTen FOLDER NOW! This may crash your kernel if you do this wrong!

First, as root, mkdir /var/empty and chown it to yourself. In compat.h, add to the end:

typedef u_int16_t in_port_t;
Then,
./configure --prefix=/opt --with-prngd-socket
You will get warnings about no SO_PEERCRED. This is okay, but IF YOU SEE THAT IT IS CONFIGURED TO USE ITS BUILTIN RNG, STOP NOW AND FIX! THE BUILT-IN RNG LOCKS UP MACHTEN!

Next, add these lines to the end of config.h (they are not detected correctly). Fortunately, these changes seem to properly work around MachTen's incomplete uid function family support.

#define BROKEN_READV_COMPARISON 1
#define SETEUID_BREAKS_SETUID 1
#define BROKEN_SETREUID 1
Then, run make (lots of harmless inane warnings about /* within comments). Once the build is successful, SHUTDOWN, BACK UP and RESTART NOW! MAKE SURE YOUR ENTROPY DAEMON IS RUNNING WHEN YOU RESTART MACHTEN! You can either start it up manually, or put it into root's .login (neither one will run from inetd for obvious reasons).
su
chmod ugo+w /dev/tty
make install
This builds all your keys and installs the new daemons. Part of the keys involves, naturally, starting up the RNG. If it locks up your system, the internal RNG got triggered. Hope it didn't crap its pants too bad.

Once this is done, try out

/opt/bin/ssh
You must use the full path right now to avoid running the original old and busted ssh. If you keep getting Host key errors with the ssh client, tty perms are wrong (see above).

Assuming that works, try out sshd: configure /opt/etc/sshd_config to taste, then start (as root)

/opt/sbin/sshd
and connect from another system. Ah, proper flush! Much better than telnet! You can configure sshd to run under inetd with the -i option, but you must make sure that the entropy daemon is running first (see cryptocrap below for the way I do it). sshd generates lots of complaints about the primes in /opt/etc/moduli. I don't know why. Tips appreciated.

If you decide to enable X11Forwarding in sshd_config, you may need to X11UseLocalhost no as well. Try it both ways and see. (I had to.)

Once this is done, disable the old ssh, either by deleting it out of /usr/bin and/or taking the execute bits off it.

cryptocrap

Here is one way to make sure everything starts up in the correct order.
% more /etc/cryptocrap
#!/bin/csh -f

if (! -e /var/run/egd-pool) then
        echo 'seeding rng'
        /usr/sbin/prngd /var/run/egd-pool && sleep 5
endif
if ($?SSH_CLIENT) then
        # we must already have ssh running
else
        /opt/sbin/sshd
endif
Then slap /etc/cryptocrap into your appropriate .login. Remember, root may then log off and they will stay running.

Finally,

Lynx (2.8.6)

Finally, we install Lynx. Yes, the text browser hounds will chime in with their various favourites such as w3m and links, but the biggest advantage of Lynx is it is highly portable even in non-Unixy environments. Since we have SSL now, we will build it for SSL. If you don't need or didn't build the SOCKS5 libraries, remove the --with-socks5=... option:
./configure --prefix=/opt --with-ssl --with-socks5=/opt/src/socks5-v1.0r11
You will get some harmless permission warnings at the end. Then,
make
su # optional
make install
make install-help

Congratulations

You are now approaching a contemporary distribution in functionality and built-in tools. You can pretty much stop here, or try your hand at a graphical browser (which includes building libjpeg and libpng), or build some of the options below (and/or look at some of the stuff I couldn't get to work).

GNU make (3.81) (optional)

The built-in GNU make seems perfectly decent, but if you want it or need it, the current 3.81 will build without changes. You might as well replace the built-in make; use ./configure --prefix=/usr to do that.

unzip (5.52) (optional)

Although gzip comes with MachTen and works fine most of the time, sooner or later you're going to get a zip archive you need to break apart. First, modify the generic Unix Makefile:
cp unix/Makefile .
Edit this new Makefile and change prefix= to /opt. Then,
make generic
make test
(you may get a timezone warning which may or may not be relevant to you)
su # optional
make install

A webserver: HTTPi (1.6.1) (optional)

Yours truly maintains HTTPi, a pure-Perl portable webserver. This page is in fact being served by HTTPi to you right now; I eat my own dog food.

MachTen comes with a very old Apache and frankly I don't use Apache. HTTPi will work just fine on MachTen with the following caveats:

To start the configuration process, run
perl configure
and answer the questions. If you run as root, it will autoconfigure your /etc/inetd.conf and (if necessary) /etc/services for you. Ignore the warning about NIS unless you've really got your system set up to use it. I use /opt/htdocs as my document root. After it finishes, find inetd and kill -HUP it, and you should find your webserver answers immediately. Performance is pedestrian because it runs from inetd, but it is quite reliable.


Things that didn't work at all

Running out of things to do? Well, take a crack at some of my failures.

tcl/tk

Both tcl 8.4.19 and 8.5.6 seemed to build just fine (8.5.6 built out of the box; 8.4.19 requires that you take -fno-strict-aliasing and -DHAVE_STRTOLL and -DHAVE_STRTOULL out of the Makefile after running ./configure). However, both of them fail multiple tests, and 8.5.6 just sits there on chanio.test (it does not crash MachTen, but the process just hangs). I didn't even get to tk.

busybox

Where to begin? It's so Linux-centric it hardly builds on anything else, let alone an oddball like this one.

So, you want to try that web browser now, or back to the main page?


Cameron Kaiser