mLib Overview mLib is a collection of bits of code I've found handy in various programs. Once upon a time, almost all of the code was nearly- strictly conforming C. Since then, a big pile of Unix-specific code's been added, with a particularly strong networking emphasis. The Unix-specific code is targetted at sensible POSIX-conforming systems, and should port easily. My programming style is probably what you'd describe as `unconventional'. I'm skeptical about object-orientation, and not a huge fan of over-hiding information of any kind. The structure and interface of the library reflects these views. On the other hand, I like documentation quite a lot. Documentation There is now a (hopefully fairly good) set of manual pages for mLib. The manual isn't installed by default since it takes a while to install and it's not a very good idea when it's part of a larger package. To install the manual pages, say make install-man (after everything else is built, obviously). There's also documentation in the header files. The header file comments were, in general, written at the same time as the code. This has the disadvantage that they focus at a fairly low level, and use terminology relating to the implementation rather than the interface. The header file comments can be handy for quick browsing. However, the manual pages are considered the authoritative source of information about the programming interface. If you have to look at the source code, it usually means that my documentation or interface design is wrong. Quick tour mLib doesn't have much of a `structure' as such. It's more a collection of useful things than a coherent whole. Even so, it's possible to detect a vague layering of the library's components. The underpinnings of the system are the exception structure, and the memory allocation routines. * `exc.h' declares some macros which do a reasonable (though not perfect) job of providing exception handling facilities for C. * `alloc.h' declares some thin veneers over `malloc' and friends which raise exceptions for out-of-memory conditions, so you don't have to bother trapping these in main code. Above this are the memory tracking system, the suballocator, and a couple of useful data structures. * `track.h' declares the symbols required for a (very) simple memory allocation tracker. I wrote it because I had a memory leak in a program. It tags allocated memory blocks with information about who allocated them, and keeps track of the memory allocated so far. Most of the time, you don't bother compiling this in, and you don't need to care about it at all. [This may be withdrawn in a later release. Too much code in mLib doesn't support it properly, and it's not being maintained or documented very well.] * `sub.h' provides an allocation mechanism for small, known-size blocks. It fetches big chunks from an underlying allocator and divvies them up into small ones. This reduces the overhead of calling the underlying allocator, and (because the small blocks are linked together in lists by their size) means that allocation and freeing are very quick. It also reduces the amount of memory used by a small amount. * `sym.h' provides a symbol table manager. It uses an extensible hashing scheme (with 32-bit CRC as the hash function), and permits arbitrary data blocks in both the keys and values. It seems fairly quick. * `hash.h' provides the underpinnings of the `sym' hashtable. It's a completely generic hashtable skeleton. It provides the basics like rehashing when the table is full and so on. It needs a fair bit of work to turn into a usable data structure, though, which is what `sym' is for. * `dstr.h' provides a dynamically sized string type. In the rather paltry tests I've run, it seemed faster than libstdc++'s string type, but I shouldn't read too much into that if I were you. The representation is exposed, in case you want to start fiddling with it. Just make sure that the string looks sane before you start expecting any of the functions or macros to work. * `darray.h' implements dynamically growing arrays which can be extended at both ends, a bit like Perl's. It replaces the old `dynarray.h' which wasn't very good. However, `darray's arrays are not sparse. Maybe a good sparse array module will be added later. At the same conceptual level, there's some code for handling multiplexed I/O. Although the core is named `sel', and it uses `select' internally, it could fairly easily be changed to use `poll' instead. * `tv.h' declares some useful arithmetic operations on the `struct timeval' type used by the `select' system call. * `sel.h' declares a collection of types and routines for handling `select' calls in a nice way. There's nothing particularly exciting in itself, but it's a good base on which to build other things. * `lbuf.h' accepts arbitrary chunks of data and then passes completed text lines on to a function. This is handy when you're trying to read text from a socket, but don't want to block while the end of the line is on its way. (In particular, that'd leave you vulnerable to a trivial denial- of-service attack.) * `selbuf.h' implements an `lbuf' tied to a read selector. Whenever completed lines are read from a particular source, they're passed to a handler function. * `conn.h' handles non-blocking `connect'. It starts a connect, and adds itself to the select system. When the connect completes, you get given the file descriptor. * `ident.h' is an RFC931 (identd) client. * `bres.h' is a background name resolver. It keeps a pool of resolver processes to answer queries. * `sig.h' traps signals and dispatches them through the event-driven `sel' system. Then there's a bunch of other stuff. * `base32.h' does Base32 encoding and decoding. This is a mad thing one needs for sha1 URNs. * `base64.h' does Base64 encoding and decoding. * `bits.h' provides some portable bit manipulation macros. * `crc32.h' declares the 32-bit CRC used by `sym'. * `env.h' provides some routines for handling environment variables in a hashtable. * `fdflags.h' encapsulates some traditional little dances with fcntl when playing with nonblocking files. * `hex.h' does hex encoding and decoding. * `lock.h' does fcntl-style locking with a timeout. * `quis.h' works out the program name from the value of `argv[0]' and puts it in a variable for everything else to find. * `report.h' reports fatal and nonfatal errors in the standard Unixy way. * `str.h' provides some occasionally useful string- manipulation toys. * `trace.h' provides a simple tracing facility, which can be turned on and off both at compile- and run-time. * `testrig.h' is a generic test rig skeleton. It reads test vector files with a slightly configurable syntax, passes the arguments to a caller-defined test function, and reports the results. It's particularly handy with cryptographic algorithms, I find. * `unihash.h' provides universal hashing. This is useful in hash tables for preventing uneven loading even in the presence of a malicious person choosing the hash keys. * `url.h' does url-encoding, which armours mostly-textual name/value pairs so they contain no whitespace characters. -- [mdw] Local variables: mode: text End: