JFAddress

!JFAddress

I had written a collection of utilities to handle FidoNet mail and echoes. They were all grouped together into an application called !ReadMail. It would control the de-batch, process, thread and replies to messages. It worked with !Zap (but would also work with other editors) to handle the replies to messages. !ReadMail was one of my first applications which managed a reasonable amount of data (for that time).

Maybe that's not quite true. Whilst !ReadMail was expected to handle data that was larger than memory, my !Shades game on the BBC (later ported to RISC OS), also had a dataset that was larger than memory, and the indices that !ReadMail created are actually based on some of the things that I learnt from working with the Shades data. Of course, even on my 4MB A5000, I didn't have quite the problems with storing the data that the BBC had with the Shades source data. It contained descriptions of the hundreds of locations, and those files could not be held in memory; the editor that managed the description linkages - written in WordWise+ script - would switch the 5 description files in and out of memory as it needed. Fortunately I did not need to go to those lengths with !ReadMail.

After I'd been working with !ReadMail for a while, I decided that there was a problem with how it - and other mail clients - worked. I needed an address book so that I would be able to find the addresses of people I was in contact with. For a lot of addresses, I used !Almanac for that - I'd taken a free demo version of the application and manually hacked in the necessary code to re-enable the features that had been disabled. That process taught me both the way that applications could be crippled by authors to create a demo version, and how some of those ways were insufficient to a casual hacker. I never wanted to do anything dodgy with my hacked version - it was just nice to have an application that worked, and immensely satisfying to know that I made it work like that. Plus !Almanac was a really great idea.

Although !Almanac had an address book, it wasn't exportable, and I wasn't proficient enough at that time to be able to add facilities to make it do so. More importantly, though, my requirements were slightly different. What I saw as the hole to be filled was for an application which was a good address book, which could provide lots of different user information, and which could be called upon by other applications to provide that data for them. Most other mail clients at the time - be they for FidoNet or Internet use - either didn't have an address book or the one they did have was integrated.

!JFAddress showing my old home, FidoNet and Internet addresses.
Don't try me at those addresses - none of them are valid anymore. So, I wrote !JFAddress. It wasn't particularly complicated, although for me it was a bigger step, being both an application and a tool for other applications to communicate with. It was also the first time I decided I needed to handle memory management better. In the past I'd used large arrays in BASIC, with fixed lengths. That's fine for many things, but it doesn't work if you want an application whose operation can vary. I wrote a heap management library so that I could handle variable memory allocations in BASIC - it's really mostly a wrapper around SWI OS_Heap with support for automatically extending the application slot as the heap increases.

However, from the beginning I seemed to be aware of the wastage problem caused by heap fragmentation and wanted to avoid it. I'm certain that I didn't characterise it like that at the time - I probably just felt that having a heap that has unused space in it was Just Wrong and must be avoided. I would have also have been influenced at the time by other existing 'shifting heap managers' and I would have heard of - but not understood - the way in which allocations managed their heap using the 'Flex' memory manager.

The initial version of the Heap library (and all other versions as well, I believe) had support for movable heap blocks, which it incorrectly calls 'garbage collection'. If you called a function in the heap library to reduce its space (PROCheap_garbage), it would call back to the application to tell it that a heap block was moving, so that the application could update its pointers for the new location.

!JFAddress used this and this meant that its memory usage was very good - not that it actually used a huge amount anyhow. But the processing time when each block was moved was not great. I believe I used these features of the heap manager only a few more times, before I abandoned trying to manage the fragmentation. It wasn't so much that I didn't care that the heap was fragmenting, but that the way in which I was trying to handle the relocations made it impractical to use in any larger program.

The address book protocol itself seems to have been far simpler than I'd have liked. There was a message to locate an entry, which would open the window showing the entry, and would request a 'use' action button which generated a message back to say that the address entry had been selected. There was a message to open a menu to chose a name, and it would return a message in the same way when one had been selected. There were a couple of forms of address that could be returned, but I only ever used the FidoNet type addresses - regular mail addresses were probably too long to return through a Wimp message. Internet mail addresses were still new to me, and only accessible to me through the Arcade BBS email gateway.

It was a little disappointing seeing what !JFAddress did. It looks like a dated RISC OS 3 application - not quite at 'my first application' level, but it was close. However, the important things about it were that it gave some very good experience of memory management in BASIC, and allowed me to think about inter-application protocols. Unlike the Pinboard Server system, where the protocol had been hacked badly and was - effectively - just an thin veneer on to SWI calls, this protocol was performing more semantic actions.

Oh! Here's a small surprise... apparently the External Edit support was added to it shortly after I wrote it. The 'Notes' button beside each entry would invoke an external edit of the text content of that note, which would be stored in the address book file. The notes themselves could be (pretty much) any size, so long as you could load them into memory - !JFAddress loaded the entire address book into memory when it started.

The application still suffers from one of the criticisms I had of the other built in address books, in that it only stored the information that I had cared about at that time. Extensibility wasn't something I had really thought about. If I were writing this about 8 years ago or so, I'd have used vCard as the base format for address storage, with any extensions for features that were otherwise necessary. I don't know what the current recommended interchange format is for addresses. I briefly looked at SyncML in the past, but I don't remember much about it at all.

ReadMail

!ReadMail

The !ReadMail application itself wasn't all that great. I remember being very pleased with it at the time, but I look back at it and it's incredibly tedious to use and badly thought out. Whilst it had a lot of the things that I brought forward from the Shades data handling (as I mentioned), it was not the best at managing the data. Because the indexing was very simple, and I had not really used any sort of sensible algorithms, it didn't really perform very well.

Lots of the processing was through temporary files, and different programs would be invoked to do small parts of the processing, from debatching the incoming mail from Arcade, indexing it, and expiring it, to generating new views on the old data. Because at the time I hadn't got to grips with the External Edit protocol, emails would actually just be saved to a file and the editor invoked on them. When you wanted to send the message you'd composed, it would send the keystrokes necessary to save the file to a known location and close the window.

On the other hand, I learnt a lot about interacting with other applications and the ways that were bad! I think that was also the first time I had implemented user redrawn windows, so that is probably where a lot of the issues I avoided with DispLib later on were discovered .

FormEdExt

!FormEdExt

Back in the early 1990s (really ? yeah, it was that long ago) Acorn released !FormEd, for editing templates. It was rudimentary but it got the job done. There were a lot of modifications that people did to this and distributed at the time. Some were pretty good, and others not so much. Many addressed the bugs in the original version before starting to add features that their authors felt were missing.

In looking back on the template tools, I can see 5 variants of the application in my 'Templates' directory, and that doesn't include the other versions of the modifications that are hiding away in sub-directories. My own variation, !FormEdExt, was created from a necessity. When I started out, I wanted to use one of the Wimp support modules that made things easier. The support modules generally did make things easier, but they introduced their own set of problems.

The Interface module was the first I came across, because a few applications I had been taking apart used it. The module would provide automatic support for some operations, and render icons differently. You could use a special validation string on the icons to make them 3D, which wasn't the general look of the desktop back in the RISC OS 2 days. However, to do this, it needed to be called as part of your SWI Wimp_Poll handling - back in RISC OS 2 there was no FilterManager to help with this. The first change, for !FormEdExt was to add these calls to the poll loop so that you could see the Interface module effects when you used them. Not especially clever, but it did a solved the problem of seeing what you were creating.

I didn't continue with using that extension; it wasn't really right for some of the things I was doing. I'm not sure why, because it was a while back, but instead I looked at WimpExtension from Doggysoft. They'd produced a lot of clever things and knew how to produce decent documentation and useful tools ("If you're wondering why Doggysoft doesn't seem to be producing RISC OS software any more, I think you'll find it's mainly because the government ordered us to be broken up, as we had a dangerous monopoly on Just Being Really Great. Oh yes.")

WimpExtension also needed support in the poll loop as well, so it was pretty easy to add, alongside the code that called the Interface module.

Understanding things a little more, I began to extend the application further, adding control for all the icon and window flags (not just those that might be 'useful'), as that helped when testing things that weren't expected. A much more useful 'Create' menu was added, which allowed a number of stock icons to be created, such as buttons, writable icons and groups. This made the applications that were built with it look more standardised, and less messy. Icon heights and widths tended to be more similar because they came from base icons.

I added some alignment options so that it was easier to see where icons were placed. I, tentatively, introduced a 'linking' validation string, so that pairs of icons (or more) could be grouped together logically. They were only used by the editor, but they meant that when you moved the icons, they all moved as a group. This made it a world easier to place a collection of radio or option icons within a group box, as they could all move together.

The linking wasn't very good, though. It essentially said 'the top-left corner of this icon moves with the configurable corner of another icon'. It made sense at the time, but did become very fiddly to work with. Especially if the icons that were referred to no longer existed, or had been modified to have different rules. You could make some very confusing rules and move random icons if the code went wrong - and it did go wrong if you deleted or renumbered them. I had tried to make it understand the deletes and renumbering, but I never really got a good hold on what I was trying to achieve. If I'd written down what I wanted and how I expected it to work, it would have gone a lot more smoothly. As it was, I was constantly fighting a poor design that I didn't even understand myself. Anyhow, when the linking worked, it was useful - otherwise it was just an annoyance.

There had been some discussion - I forget whether it was on FidoNet or Usenet, but somewhere that I frequented - about the lack of integrated development environments. This was a theme that continued later, through some of the later !ResEd work (and for the same reasons), but was at the time quite obvious. I wanted to try to make it easier for people to work with templates in an integrated environment. Although it wasn't simple to build in more featureful integration, the External Edit protocol was a great way to allow one application to provide services for another.

External Edit was a reasonably simple protocol which built on the regular data transfer protocols to allow files to be edited by other applications on their behalf. This meant that a client (like an IDE) didn't have to worry about how the user liked to edit their files, but could instead just ask the files to be edited by a suitable editor. The 'integrated' part of the Integrated Development Environment depended on how smoothly the editing was performed. The protocol allowed for limited control over the file under edit, which wouldn't always be suitable for every filetype - it was quite focused on editing text files. But it went a little way towards making things feel more integrated.

I'd added the External Edit code based quite heavily on the !JFShared library I'd written previously for other things, and it worked pretty well. You could edit your template file and have to returned back to you when it was complete. Whilst !FormEdExt might not have been the most pretty of editors, it had features that I'm almost certain nobody else had used. On the other hand, nobody actually wrote anything to use that feature so... bit of a waste of time in that regard. Still, it was a learning experience, and might have been useful. 'If you build it, they will come' doesn't always apply!

Editing a Template file and then returning the edited file was quite simple with the EE tool - just press 'R' to return the edited data.
(Animated version (62K))

After I started with RISCOS Ltd, I updated !FormEdExt for the features which had been added - new window flags, and true colour icon and window furniture selection. The latter was fun because it meant that the application needed to communicate with ColourPicker, which I'd used before and had created a library to manage. Working the library into the old application was quite fun.

Later still, I added the mouse scroll flags as well, so that it was easy to set the flags indicating that mouse scroll was to be used.

JFShared

!JFShared

I've mentioned !JFShared a little, but not actually said much about it specifically. The resource was modelled around that of the !CCShared resources supplied with Computer Concepts applications. The idea was that my BASIC libraries could be contained in !JFShared and would not need to be distributed with the application. If you had a !JFShared installed, you probably wouldn't require anything else to run an application.

It was a reasonable idea, but flawed in that people didn't want to have to obtain multiple resources in order to run an application. It helped me because I just had to say 'you need !JFShared to run this', pushing the effort on to users to locate and install an appropriate version. Not really a very good idea. I produced an installer for the !JFShared, so that it could be installed either inside !Boot.Resources or inside the !System resource. This made it a little easier on users but didn't negate the root problem of having to do more than just get the application and install it. I even produced an 'AOL' version of the installer, which parodied the AOL CDs which came with overblown installers - mine came with large graphics and sound samples that told you what it was doing.

I began to distribute the BASIC applications compressed with Cy Booker's excellent !BC ('BasCompress'), and retained the uncompressed code alongside the compressed version. This meant that a) the release archives were larger, and b) you still got the original source. The former was an unfortunate side effect but not one that I worried about much.

The latter was very important to me, though. I'd learnt to program by reading the uncompressed sources of freeware applications, and the example code supplied in Acorn User magazine. One of the best ways to learn how to do something is to see an example of it - together with having the right documentation. By supplying the source to my programs, despite their being compressed, I felt that I was continuing in that vein. The other part - that of having the right documentation - I tried to address by continuing Guttorm Vik's excellent work on the !StrongHelp manuals, editing, updating and managing the main OS manuals for a few years.

As a quick aside here, !BC had some excellent documentation supplied with it, on par with many commercial tools at the time. Many applications (my own included) could have done with that level of effort in documenting the tools.

!JFShared itself was a collection of BASIC libraries, a skeleton application, reference !StrongHelp files, and compression tools to run Wimp applications. Initially the 'WimpLib' was a library I began to put together out of what I had learnt from the other applications. Tools like !StickyBD, and the numerous desktop 'sillies' were incredibly useful to me back then. I extended many little things for my own amusement, and so that they were useful to the family. Little tools like !WhichFont (which I have a vague memory of being by Rosemary Miskin - it is odd the things that you remember) were extended for special features, and I learnt more about the Operating System.

There was, back in those not quite forgotten days, a little red school book which I used to make notes in - my 'JFPRM' - which included details of the APIs for the Wimp, OS, Draw, ColourTrans and other calls. These and little diagrams of windows and their relationships, and other bits of handy information that I needed to write useful programs, all ended up in the little red book. Eventually all of them made their way in to the !StrongHelp manuals, and the red book wasn't needed any more. But that was where the documentation started.

Beyond my WimpLib, I needed a memory management library, so wrote a Heap library, which could also be linked with my applications. As mentioned earlier, it wasn't the best, but it did a very good job of making it easier to write applications with dynamic memory requirements - but also forced you to use direct memory access for structures. Not that this is a huge problem, but it is a quite different way of working to using plain variables.

The External Edit library provided a nice simple interface so that I could create clients and editors using the protocol. It might not have been the most flexible of systems, but it was easy enough to integrate into my applications. Adding a way to edit external documents became simple. With the flexibility, though, I needed a means to list things in menus dynamically.

Up till a point, I'd used simple string delimited menu definitions - which was terrible for internationalisation, but then I didn't think too much about it back then. A menu could be created with something like:

ibmenu%=FNcreatemenu(Appname$+"|Info>infobox%|Quit")

Which would create a menu with the application's name as the title, an 'Info' entry which used the infobox% as a sub-menu, and a 'Quit' entry. These were simple to parse, but if your menu contained multiple items, created on the fly, it was impractical to build them in this way. The DynMenuLib created dynamic menus, which used the heap library to maintain their allocations.

The dynamic menu library was meant to provide a single dynamic menu, but you could preserve the menu context and start a second dynamic menu without the first being destroyed. It wasn't the best system but it worked out quite well. Applications like !JFTerm used it extensively to construct most of its menus for the different styles of connections. The Doom applications used it to provide menus of hot list connections, WAD file details, and other little things.

I added a Socket library so that I could create Internet applications more easily. The library was very useful in being a general place to put useful little bits that I learnt about the Internet module, but it became clear that it needed to be restricted to just the operations that it needed. Although it had support for things like SOCKS 5, very few applications would ever need to use that. Some specialised bits of the Socket library got moved out directly into the applications that used them - it was very rare to need dedicated functions to manipulate the out-of-band data, so they came out.

A Pane library was created to manage panes attached to windows, mostly out of the original code that had been in !Imagen so that I could reuse it in other applications. Little quick applications like !MyRC used the Pane library so that they could attach lists of users and the input box to the main window.

DispLib was one of the more interesting libraries which grew significantly over time. Originally it was created to provide the text display for !JFTerm, and so its goal was to be fast at redrawing from the bottom upwards. It gained the ability to handle colours, anti-aliased fonts, selections of text, and copying to the clipboard. As well as these it provided menus which could be used to configure fonts and the selection.

This made building applications that needed a textual output significantly easier to write. The tools that came with !Doom+ and used TaskWindows (and !DrawPlug, and !SVG and many others) all used this library. In fact, the use of the display library together with a TaskWindow control was so common that the TaskWindow control was made into a library itself and used in a number of applications.

!MP3Encode Applications like !MP3Encode were really just tying together these libraries in a useful way. !MP3Encode had a number of different configurations, which it presented as options in a dynamic menu. When you dropped a set of .wav files on the IconBar icon, it would invoke the TaskWindow display library to run an MP3 encoder on the file, and display the output.

There were also libraries in !JFShared for handling communication with the Econet module, and for running programs as CGIs. Mostly the CGIs were for use under my !ArcWeb protocol handler 'LTP' which I had created solely so that I could run CGIs. Later David Thomas released the !NetPlex HTTP server which supported CGI programs, so my library was updated for it.

The library was abandoned because I no longer ran many things under CGIs on RISC OS - it's not exactly the best environment for it .

There were other libraries that were used in a few places, but never made it into !JFShared because they were not as widely useful. A ColourPicker library made it simple to add colour selection dialogue boxes to applications. There was a Plug In library that helped me to write the !DrawPlug browser plug in and later the !SVG viewer plug in. There was a simple styling library that allowed styles to be nested over one another and applied, similar to how CSS works, but not quite as flexible.

A hugely useful Filer library was added to handle most of the places where I needed to present blocks of data. Many of the WimpCTCP tools, a few !Doom tools and my !FTPUpdate front end all used the Filer library. It many be the most widely used of my libraries that wasn't a part of !JFShared.

There was also a URL library which handled URL decomposition and fetching through the URLFetcher modules. I forget where this got used, to be honest. I'm sure there must have been a few applications which needed it - maybe !Forecast and others like it.

!JFShared and the libraries that went with it were a great learning experience for me, and I hope they provided a useful resource for others to learn from. I won't say that the methods it used were always the best, but it did provide a lot of good insight into how you could do things.