GMail

!GMail

Back in the A5000 days, whilst I was at University, I was finding that having email was really useful. Whilst in the labs I could send and receive email and so provide some comments to people on what I was doing, problems I'd found with things, and other such random bits. And talk about University work as well - I did do some of that. But when I worked at home I didn't have that opportunity. No permanent connection meant that I couldn't receive mail - I could send mail though. The mail clients at the time were pretty large though, and generally required an Internet stack to be loaded in order to even run. The result was that you couldn't test stuff and be running the mail client easily, or if you could it meant that any problems that meant a reboot were more time consuming because you had to load up the mail client afterwards to get back to where you could send a mail (usually to report 'nope, that crashed badly!')

So I wrote !GMail - a small command line client just for sending mail. Initially it was quite simple, it let you provide an email address, subject line, and then read lines of text until you gave a line which was just a dot ('.'). Pretty much like you would if you used one of the simple command line mail clients on the Unix systems I used at University. It would store the email in a file on the disc so that a transport could deliver it.

Initially it supported !POP, but as I had a number of different mail clients and systems which I'd obtained whilst learning about Internet things, I added other transports. In the end it supported !POP, !KA9Q, !PopStar, !Termite (Doggysoft's suite), !InetSuite (ANT's suite), !FreeSMTP, and !Mail (Acorn's mail software). It also supported the batch mail formats used by Arcade, Digibank and Arctic BBSs. Each was just a slightly different way of storing the mail in a queue so that it would be sent when you next connected to the Internet.

I even wrote a proxy for the FILE functions that allowed them to be used for connection to a socket. This meant that with the proxy (a simple header that directed the calls at my own functions) in place you could either write to a file, or with some very slight changes to open a socket instead of a file you could communicate with a remote TCP server. !GMail used this to insert the SMTP prologue and footer around the mail stores when the 'direct' method of storage was configured, and communicated directly with the mail server. It may not have been the most efficient way to send emails, but it worked quite nicely if you needed to send a mail and you no longer had a disc to write to.

Things like getting the Message-Id correct were relatively easy (albeit I still see email from some clients that don't seem to be generating correct Ids from modern software!), but I also added support for quoted printable body text and headers (which are different formats, and the needs to know which character set you're using which was interesting from a testing perspective).

It handled !Marcel and Pine format address books, so you could quickly rattle off emails to people without remembering their full email addresses, and it could run as multiple 'users'. The users support was really just a way of switching configuration. As the configuration was entirely system variable based, you could change to another configuration by changing the variable prefix. Each 'user' could have different real user names, email address, address book and signature to be appended automatically. I used the feature for a few different roles to try to make the distinction between mail for particular tasks.

Initially, !GMail only supported UUEncoded email, and you could attach multiple files using UUEncode from the command line quite easily. Mainly that was because the mail clients at the time didn't support the newfangled MIME formats. Wanting to try to be ahead of the game (I wasn't, because !Marcel did support MIME - and even supported nested MIME attachments which I've never seen any other client handle properly), I added support for MIME attachments when I added the quoted-printable body support. The two have to go together, really - the body and the attachments are just parts of a multi-part MIME message.

Attachments would be encoded in the most appropriate way for their content. They could be attached as text, Base64 or quoted-printable. These would be determined automatically, but you could override them at the command line. RISC OS types would be attached by looking up the relevant type in the MimeMap module (which I wrote an implementation of, partly so that it could be used with !GMail and other applications), or through a hard coded table of the common types, or using the registered 'application/riscos' content type (which most RISC OS clients eventually supported, which is nice).

Amusingly, it also supported the 'message/external-body' content type for 'attachments' that were URLs. This was an interesting part of the specification that stated that the actual body of the message could be obtained by some other method, and was later extended to allow the access-type to be specified as 'url'. So if you gave !GMail a URL to attach, it would generate one of those.

The command line could be passed a whole range of options to attach files, override the sender, recipients, subject, etc. Eventually I even added a switch that would just let you enter the raw message, headers and all. In this way you could just pipe a message to it which contained all the details you wanted in the final mail and it would be stored in the relevant quote for the transport (together with any headers that you'd missed, like the 'Message-Id'). That made it quite useful for just redirecting messages.

As well as sending the mail, it could be configured to keep a 'File Carbon Copy' of the message, so that you always had a record of what you had sent, just in case it came back to bite you later. I still have almost all of the mails that were sent with it, because of that - I can find all those things that I said in the past that make me cringe.

The 'editor' for !GMail was line based input, so you had to get your mail right when you wrote it (or edit the mail queue before it was sent). When writing longer emails (because it was very useful for messages that needed thought, as well as the shorter messages) it needed something more clever. Initially I was writing my messages in an editor and then pasting them into the mail I was composing, but this was not really a good solution. So, I wrote a small library to allow it to call out to the External Edit protocol (a way of using the configured editor to perform the editing, before returning the content - I've mentioned it in a few of the rambles).

The external edit library was just written in !JFPatch and linked directly with the mail executable at first. Eventually, though it became tool in its own right- ee - and was converted into a proper library that the main mail tool linked against.

I'd finally given up trying to keep it under 32K, and it had become about 50K. The final features I added were to allow you to specify a directory to be attached. If you did that it would collect all the contents of that directory into a 'store' (uncompressed) Zip archive and attach that file.

!GMail is the primary reason that I felt very comfortable sending 'quick' messages to people. If I found a problem with a piece of software, even if I wasn't connected to the Internet, I could just rattle off an email to the author, stick in the debug information that I collected there and then, and any details about what I was doing. These days, I use Linux, and I've got pine always running to send emails, although I don't tend to send quick mails as much these days.

I imagine that a few of my friends became a little frustrated by the number of mails that I tended to churn out - especially as they would all turn up at the same time, once I'd transferred the queued mail to a system that could send the emails.

I think (I can't remember, and the !Help file doesn't say explicitly) that that tool was included as part of the ANT Internet Suite. I remember chatting to Nick Smith about it quite a bit, and they registered the tool on my behalf.

The tool was configured entirely by system variables, and whilst this was documented, it wasn't as simple as some other tools, so I also wrote a front end. The front end application could be used to run the mail tool in a TaskWindow, or to configure the various system variables (and could control the different 'users', or roles, that the mail clients could use).

As one of my earliest (useful) C applications, !GMail is a little embarrassing, and I had started to rewrite the whole thing in a more modular way, doing things properly rather than the somewhat messy methods that I'd used previously. I'd learnt that having tests for each little bit worked well - the original had tests for the UUEncode and decode, the quoted-printable and Base64 code. The rewrite had tests for all the bits that I'd written, but I soon tired of the project and moved on to other things. I'd come back to it now and then and write a little more, but what was the point? !GMail worked incredibly well and had no real functional problems that meant it should be rewritten.

Forecast

!Forecast

One evening, whilst chatting on IRC and looking at some webcams, I decided to write a little application that could regularly reload a JPEG image from a website and display it in a window. The idea was to show web cams in an window in the background so that you didn't have to have a browser running. Not a particularly hard process.

The reason for using JPEGs was really because they were easy to render using SpriteExtend, and most web cams were JPEGs. However, some were progressive JPEGs which SpriteExtend couldn't handle at that time (back in 1997). To handle those, I added a automatic conversion using cjpeg and djpeg. I'm not entirely sure why I didn't use jpegtran, except that maybe I didn't know it existed back then.

You could configure the reload period, and the list of camera URLs that it supported was generated from a configurable hotlist which was quite nice. Originally the application wasn't intended for looking at general webcams, but more so that I could download a satellite image for the weather - hence the name. That's also why it's possible to stick little weather symbols over the downloaded image - there are clouds and suns and rains and lightning. And you can write on the image as well, if you want.

I never got around to writing the 'save image' functions though - which is sad as that is pretty simple once we've done the rest of it. It was quite neat though - a few people submitted suggestions and lists of more cameras around the world, so the hotlist became quite interesting.

HTMLTidy

Of the many little things that I ported to RISC OS randomly, HTMLTidy was one of the more interesting (of the small ones anyhow!). David Raggett's tool would parse and reformat HTML so that it was more readable and would check for many problems that might cause compatibility issues between browsers. The changes were almost entirely to make it easier to use on RISC OS - unlike quite a few ports, the source was pretty flexible and required very little specialisation.

Because there was so little specialisation necessary, it was easier to add features. First there was the Throwback support, which is always handy in a processing tool. Any warnings or errors that were raised for lines of the source would make a Throwback window appear with direct links to those lines. I think the later versions changed the way in which the reports were done, so my changes wouldn't have been quite so easy to make, but at the time it was very useful. Alex Macfarlane-Smith did a later port, although I think he found he couldn't reuse my changes.

The other change that I made was partly for fun - if the filename was prefixed by 'url:', it could be read through the URLFetcher interfaces to fetch the page live from the network. Rather than changing lots of bits of code, I replaced all the file I/O code so that any calls to them went through a small library that read from either file or URLs - any 'fopen' calls would decide which type of file access was required and all the subsequent operations would go through the right one for that type. It wasn't the most impressive bits of code, but it did the job - which was partly for amusement anyhow .

MyRC

!MyRC

There were quite a few IRC clients for RISC OS - for some reason it seemed to have a lot of people writing their own particular variants which were a little different. As I've mentioned, people mostly moved away from !IRClient, and I'd been using !JFTerm for some time. !ixRC looked like it was going to be top dog for a while, and certainly looked very polished. Thomas Olsson's !LIRC was the client to use for some time - probably still is, I'd expect. Apparently there was !Grapevine, too, but that was very late on and I never saw it. There were quite a few more, but I can't remember them all. I know Chris Johns wrote a couple, and clearly I had to write one more. !MyRC was ... the one day multi-channel windowed IRC client, and had a users pane, mIRC colour support and input history, too. Primarily I wanted to create a multi-channel IRC client, because people had complained that they wouldn't use !JFTerm because it only supported a single channel - a design feature when I wrote it.

It was updated a little after the one day, mostly to fix small bugs, but it did what I wanted after a day. I thought it was pretty neat at that, really. Whilst it was there to show that you could have a simple client with most of the features you come to expect in very little time, it was also intended to show that it was easy to do this kind of thing when you want to. Anything you can write in a day can't be that hard can it ?

As a quick experiment, I added a 'bubbles' library so that I could see what it looked like. I thought it would be interesting to have proper speech and thought bubbles, but it wasn't that impressive really. However, it didn't really look as good as I had hoped. I had wanted to do some extra testing with it and to see how it would work if it alternated the sources so that some speech came from the right as the conversation progressed. I really wanted to try out something like the Microsoft Comic Chat client, but it wasn't easy to get the rendering to work in a useful way.

I've only got one picture of the bubbles experiment, and it doesn't really show how it looked. It was quite neat, but wasn't sufficiently useful to be kept in the IRC client.

RSS

!RSS

I had begun to use RSS - Rich Site Summary - feeds from a few web sites to track their updates. I think it was probably something like David Chess' web log that encouraged me to investigate them. I'd added RSS feed to my own diary, to make it possible for others to track my own entries, but it was only after I'd created the ParseXML module that it became relatively easy to create the !RSS application.

The application itself was built as a set of libraries, in a far more modular manner than many of the applications I'd written previously. Initially there were basic 'Bubbles' which were rows of text that could be rendered within a rounded rectangle, which I later used in !MyRC. This library was used by a 'BubblesList' which was a collection of the Bubbles, and could handle resizing, detecting which Bubble was clicked on, and rendering the list as a whole efficiently. A 'URL' library provided an interface to the URLFetcher modules and could download content progressively.

An RSS library provided the means to parse RSS files, using the ParseXML to perform the actual XML processing. The ParseXML module made a lot of the work significantly easier, as it produced a simpler structure that we could process into the Bubbles. The Hotlist library was taken from the !Forecast application, as a list of URLs and names is a shared concept for both the applications.

The image referenced by the RSS data would be downloaded (in parallel with any other fetches) and rendered using the ImageFileRender modules. The later versions of the application also included OCS parsing. OCS files ('Open Content Syndication') provided lists of RSS feeds which could be processed by RSS readers to provide a collection of feeds which could then be imported or viewed. It doesn't seem to be in very wide use these days, with OPML being more commonly used for collections of feeds (as well as other data).

In theory, the !RSS application would have been trivial to extend to include Atom, which I began to use more in later years, as it would have merely meant replacing the RSS parser library with one that understood Atom. Atom itself was very modular, and reasonably simple to parse. The whole application was written pretty quickly, and although it was built up from simple pieces, there are still some quite poor bits of code in there. Error handling in particular is very bad.

A lot of the goal of the application was to produce an RSS reader that worked, just to show what you could do. It didn't do many of the things I wanted, like keeping track of feed details, highlighting new entries and providing pretty pop up notifications. In theory, some of the Database work I did later would have helped there, but I never came back to the code - except for one small experiment.

At the same time as I had played with the !MyRC client using Bubbles, I used the same implementation in the !RSS application to provide speech bubbles. It was pretty simple, as the Bubbles were already quite nicely separated into a library with this in mind. The results, though, really weren't as nice as I'd have liked. I think it's partly the speech hook doesn't look right, and partly the fact that it's just not a very good idea in the first place .

JFinger

!JFinger

One of the small tools I wrote was my Finger client, !JFinger. It connected to a machine, requested information about a user, and then displayed what it received back. The Finger protocol isn't really useful these days. Back then it was used to give an idea of what a user was doing by returning the contents of their '.plan' file.

My own diary was inspired by the finger information which John Carmack used to have, which gave some idea about what he was doing. At that time, I was working on Doom and it seemed appropriate. The !JFinger application was one of the simplest of the EasySocket Wimp applications, and used my DispLib to provide the text display. That made it nicer because selections were all present, all the handling for updating the screen with anti-aliased fonts was there and I didn't have to think too much about it.

!JFinger was - I think - the only one of my applications that I ever added as part of the RISC OS Select components. Mainly because the earlier applications were sufficient, but also because my applications weren't generally up to the required standard.

FTPUpdate

!FTPUpdate

I needed a way to upload my website to the provider, as I'd previously just used the freely available solutions, which were mostly desktop based. The problem with that was that it couldn't be easily triggered to upload the changes in a batch. I wanted to make changes and then have just those bits uploaded. Rather than having to work out what had changed manually, I wanted to have the changes worked out automatically.

The application was actually built up from simple bits. There was an FTP library I wrote which contained the state machine for connecting and communicating with the server. The basic connection, transfer of files to and from the server, renaming, deleting, directory creation and cataloguing were implemented in the library, and could be used by any client. It was pretty simple and quite easy (I thought) to work with.

Alongside the FTP library there was a URL handler that broke down the requests so that they could be used to work out the parameter for the connection. Above them, the main tool handled the scanning of the directory, recording the details of the files in it and uploading (or removing) files from the server.

The tool had two ways of checking for differences between the local and the remote copies - either using the time stamp and size, or by checking the CRC of the content. It could be configured to use either through the command line. The tool itself could be used independently, but I generally used it through the front end application.

The front end could show all the sites that were managed by the tool and allow them to be updated independently. The front end itself used the Filer library that I had previously written, so it was all pretty easy to control, and the tools were invoked through the TaskWindow and DispLib library that I'd written as well. It's all a simple matter of linking the bits together .

The tool was still in use up to the point at which I stopped using RISC OS - I moved all my sites to being stored on Linux and started using a 'sitecopy' tool instead. It does a very similar job, although probably a little better!