Writing the rambles

There are a lot of things that I have missed out from these rambles. Some old, some more modern, and some utterly pointless but fun. It is probably not sensible to enumerate everything because that would be very dull... well, more dull than the rambles already are. But, there is one more bit of work I did which I should at least mention - if only because its taken about a year to complete.

These rambles took longer than I expected, and despite wanting to include everything they will ultimately fall short. But they are important to me, not only to put some of these things behind me, but also to justify what was a quite painful few years - but ones that I cared about.

The idea for the rambles came partly from a short chat I had with Matthias Scheler at work about some of the network things that we had worked on in the past. I thought it might make an interesting diary entry, much as I had done in the past when talking about the resilient nature of the network stack. When I came to write about it, though, I found that it meandered a little, and there were other topics I wanted to talk about. I had said in the past that if anyone wanted to invite me to talk about technical areas of RISC OS, I would be interested in doing so - nobody ever had, but since I know a lot about 'stuff', I had always wanted to try to pass that on to people.

So I began writing some more words about different areas. After about 4500 words on the the Internet stack (which became most of the 'Internet support modules' ramble), I wrote some bits about the graphics system. I easily managed 7000 without even touching on anything other than Draw and DrawFile. I realised that it wouldn't work as diary entries - it needed more structure, especially if I ever wanted to find things in the future. The diary is really meant to be just a diary, not something better thought out, as the rambles have become.

I asked Chris Williams if he could look over what I had written and see if it was worth continuing. He encouraged me (whether just to be kind or because he thought it had merit, I am not sure, but I am glad he did!) and I began to put together a template website to publish on. As I did not want to let the rambles get accidentally published, the site was (and is) set up separate from my main website, and gets published to my local server, rather than the live server.

I knew early on that I wanted to publish the rambles on a website so that everyone could see them, but I didn't want to just dump a whole lot of text in one go. That is a way that guarantees that nobody will read things. Instead, I thought that publishing daily, in manageable chunks, would work quite well. Restricting publication to just the working week would mean that it could be read by anyone at night (or during the day at work, of course), with a weekend to catch up. Or maybe the week's rambles could be read at the weekend if anyone wanted.

I am not sure what people will really do, and that is all assuming that anyone reads what I have written. When I started writing, I hoped that it would get a reasonable readership. I am certain that there will be sections that some people won't care about, and will skip, and that some people just won't stick around because I am talking a lot about me. But I tried to make it interesting - there are a couple of styles for the rambles. Whilst a lot of the rambles are very 'I did this, that and the other', in other places I have gone into more detail, particularly where the content is harder if you do not have the background.

One of my reviewers hasn't used RISC OS at all, and provided a lot of very useful comments from that perspective. I cannot explain every little nuance of the system, but it is a matter of trying to make it all accessible to readers. Whilst I am not writing for readers, I don't want to put people off too much. Balancing that style has been interesting itself.

Splitting the rambles to one per day meant that they would go on for a while. At first, I was expecting to maybe get out about a month of text, but it turned out there was a lot more to say than that. There are 76 'days' of text here now, which is about 3 and a half months of actual days.

Initially, the rambles were processed much as the diary was, but I wanted some more control over things. Pictures inline with the text, and able to be aligned to the side were introduced when I started writing about Doom. That was also the first time that I pulled out RPCEmu to try to capture images of what I was talking about. It is hard to have lots of words without something to break up the flow - especially when I ramble off topic.

Together with 'Wink', I could capture little videos of the screen as I did things. Some videos were a bit jerky, but mostly they seem to have turned out alright. The actual content was saved out as PNGs and then converted under Linux from a sequence of numbered PNGs into a GIF with 'Image Magick' and then optimised using 'gifsicle'. Many of the PNGs that I have produced for other images were reduced with 'pngcrush'.

Individual screen shots were taken on RISC OS, though, using !Paint to save the windows that I was interested in directly to PNG. Where images already existed as sprites, they were converted on RISC OS through the Image Convert stack, using 'ImgConvert'. A few things - especially the older applications - needed little tweaks to work. Many of the very old things weren't even safe for use on RISC OS 3.5, so needed 'fixing'.

One of the things about using RPCEmu was that I found bugs in it. When I had been working with QEmu for the 32bit work, the lack of domain support had become an issue. Domains do appear to be supported in RPCEmu, but the memory manager as a whole doesn't correctly interpret the controls on accesses to pages. The intentional changes to prevent writing to some areas of memory are ignored by RPCEmu, which means that some of the protection that was added to later versions of the OS are ineffective.

It was quite interesting to step through the code and be able to report the problems back to the authors though - although reading through a third implementation of the ARM MMU decoder (the prior two being the ARMulator and QEmu implementations) was harder, this being long after I had done any ARM code.

I have pulled out lots of little things from my archives of old things from when I was at University, or things at Stratford. The archives that I created from my old floppy discs, before I threw them all out, were great for finding little gems - and things I had completely forgotten about. It reassures me that my hoarding of old things that I 'might need' is actually worthwhile. Just this once, anyhow.

Much of the old BBC software is now lost, as it was never transferred from 5½ inch discs. It might be sitting in a box somewhere still - I do not have the means to restore it. But also, these rambles were meant to be based on RISC OS things. I drifted a little into some of the BBC programs early on, as it showed a little of the background from which I had come.

Anyhow, having got images into the content, I began to need section breaks to split up those rambles that had no pictures. I only use two levels of sections in the rambles - 'components' and 'sections' - which are shown in the main index, and provide a small contents page on every page. The contents list at the top of each ramble was a relatively late addition, because I think that anyone reading will want to be able to see the kind of things that the ramble talks about.

From the outset, I wanted every ramble to have a short heading, and a subtitle which was a mildly humorous filler. The subtitle follows the chapter headings from "Winnie-the-Pooh" in opening with '... in which'. The style is fun, and it fits my love of Pooh bear. Thinking about it now, the only other way I could have gone with the subtitles might have been to use an apt lyric from songs, but that never occurred to me back then.

Once I had decided to write the rambles, I had put together a brief set of notes on the topics I would like to cover. The final rambles follows those notes quite well, although the order differs, and there are a few things missing (and many, many more added).

And once I had got about 10 rambles in varying degrees of completion I found that I needed to mark each with some notes about where I was up to and what I still needed to to. These were 'fixme' marks, which tagged text that showed areas that were still missing. Some rambles merely contained '<fixme>Write stuff</fixme>' for much of the time!

Around the same time as these marks, I started to work out what would make a good size for the rambles. Along with looking at publishers guidelines, I began to look at online news publications for article length for different considerations. Whilst I was producing something different, I wanted to get a feel for how much text people would read. During development each article has a little graph beside it which shows how many words the ramble contains, which is green up to a set number of words and subsequently red.

I selected an initial word count of 3000 words as a target for each ramble, and the graph would show if I went over that (and if I went over it twice, there would be two red highlighted regions). This was impossible to keep to. At first I began to split rambles which went over the limit, but 3000 isn't actually a lot when you are writing about some topics. So I increased this to 3500, and it stayed there for quite a while.

Eventually I settled on 4000 words. There are some rambles that fall well short of this, because there is less to write about, even if you join a few topics together. There are some that go well over that goal as well. But it is a reasonable target. I tracked the number of words in each ramble, through a 'munin' plug-in that I wrote, taking the statistics from the index generation that was run each time the pages were regenerated.

As I mentioned, I looked at news articles to see how they were structured - how The Register wrote its articles was of interest (and The Iconbar and Drobe to an extent), but also other publications which were in different areas. I began to look at readability statistics, mostly using the 'style' tool to collect different statistics.

My text varies quite wildly in its complexity, and has changed over the time that I have been writing. My own statistics, like word count, and number of contractions or pronouns, are also interesting - and they are also tracked. I have not made as much use of these as I might like, but they have been interesting to see.

The graph of 'fixme' marks makes an interesting 'burn down' chart, although it is a little false as many sections were added without every appearing on the list. The word count graph, on the other hand, marked off the milestones as I hit them. By the time I had begun collecting statistics, I already had 100,000 words (around November 2011). 200,000 in March 2012 was a good feeling, but it had become very hard to write more by that point.

In mid November I add an automatic spell check to the pages. Words that were not know in the dictionary were highlighted in the ramble in red (and when you float over the word, suggestions are offered). I spent about 2 weeks reducing the number of 'typos' from 1,800 to 0, either by correcting spellings, rewording sentences, or adding exceptions to the dictionary. The exception dictionary now has over a thousand words which are my 'special' way of writing, or words that just aren't known.

Staying on top of the spelling mistakes made me feel a lot more confident about writing things that didn't look stupid. There are still grammatical mistakes, I am sure, and typos which make real words cannot be spotted by this, but I have a number of friends who have been reading the rambles to give me review comments.

Because it was hard to actually get comments from people which I could relate to the text, I added a feedback mechanism on every paragraph. Within the development site, each paragraph has a little 'edit' pen beside each paragraph. When clicked, it offers a comment box so that they can provide any notes on that paragraph. Although their notes do not appear inline with the text, I do get an email telling me the context and their comment.

This made it a lot easier to get feedback on the sections that might not be clear, or were just plain wrong. I had toyed with the idea of retaining the feature for the live version, but wasn't sure I wanted to do that, as it could be quite demoralising. Even if I am not including a per paragraph feedback, there is still a single article feedback button at the bottom of each ramble.

However, I have at least prepared for people providing some feedback. I have got an 'edit' element which is intended to be used to indicate corrections and commentary added after publication. I have not had to use it, as nobody has provided any substantive comment that needed any changes during the publication. I decided I still wasn't going to include user comments with the article, unless they turn out to be useful at part of the edits.

As with the diary, I don't want to be held responsible for other people's comments - nor their censorship (which I wouldn't want to do, but there are times when I just wouldn't want the content on my site) - and if people want to say things about my site, they can use their own forums to do so. My rambles, my rules, and I feel comfortable with that.

I also toyed with the idea of including Google Plus and Facebook links on the rambles, but I wasn't sure I want to endorse them. On the other hand, they do provided some nice indication of people liking things without making me feel too bad. Eventually I decided that it probably wouldn't hurt to include them. I haven't really looked at whether people have 'liked' the pages much, although I have tracked the numbers of readers as the site was published.

At the same time as I played with the 'likes' feature, I learnt about the schema.org and Facebook markup which would give more information to sites. My main site, and the rambles, now use these so that if you link to the site you should find that the link is richer than it would be without. Little 'bread crumbs' were added to every page to help Google present the pages in a more structured way - and I can use them to present a proper list of sections at the top of the page as well.

The main index page went through a couple of structural changes since I started, but I've tried to retain a similar style to that of the main diary, as the site should follow a similar pattern in order to be coherent, and not just a collection of random pages. I think I have managed to achieve that, although it

An example of how I, and the people who reviewed the text, saw the development version of the rambles:

  • RISC OS Rambles
  • Future development directions
    ... in which we look to what would have been if we could have been doing it.
    Video systemDiagnosticsAlerter protocolFiler
    Day 62
    Tue, 25 Sep 2012
    Updated: Tue, 5 Jun 2012 23:06:48
    FIXMEs: 10
    ... in which we look to what would have been if we could have been doing it.

    55 paragraphs, 5107 words, 0 pictures, 77 contractions, 74 pronounsReadability: Ari: 13.1, Coleman-liau: 11.7, Flesch: 57.5, Fog: 14.7, Kincaid: 11.5, Lix: 47.2, Lix-school-year: 8, Smog-grading: 12.3

There is a lot of information that has been collected, much of which I do not use, but it is vaguely interesting in itself. Of course, seeing the word count go up also reminds me that I should edit better. I just find it hard to cut things out.

Towards the end of the publication, I updated the main index page to include a list of the pictures that are present in the rambles. I don't know if anyone would be interested in them, but pictures always make a good starting point for reading something, especially if some of the topic is dull - they break the monotony.

Chris Williams had suggested early on (and others have subsequently) that I write a book, and that might be interesting. Publications have become a lot easier now. Initially, I didn't want an electronic book, because the content is already electronic in HTML and presentable as it stands, and I am happy with the way that it is presented. However, since then I decided to try out converting the rambles for the Kindle. It has turned out really quite well - and I used the Kindle to review a lot of the text. Eventually I will make the full rambles available for the Kindle.

If I go down the route of making a real physical book, it seems that, going by the wordage, it will be a number of volumes. I don't expect anyone other than myself to actually want a copy, but I like the idea of having a physical copy of my work in my hands.

The reviewers have given me lots of feedback on little things, some big things and a good amount of encouragement just by reading things. I'll mention a few names of people who have really given their time to help out in reviewing the rambles - Chris Williams, James Coupe, David Thomas, Alex Macfarlane Smith, and Christopher Bazley.

If you have ever wanted to know what a third of a million words looks like, you now know. And if you've read even a fraction of them, I thank you for your interest! Although I am no longer actively doing things, these rambles have been an interesting outlet for a few frustrations.