The GIF conversion module was annoying. Not for itself, but because of the Unisys LZW patent. The GIF decoding code had been supplied by ANT from their Internet suite, so had been around for quite some time. However, we couldn't use it because that would be in violation of the patent, due to the LZW algorithm being used to decode the image.
Although the ConvertGIF module had been created to use the GIF decoding code and would have worked fine, it wasn't usable. We contacted Unisys to check whether we would be required to pay a license fee for the use in a decoding module. After responding to their questionnaire, they informed us that a license fee would be payable, and that it would be a figure for anything below so many units, and an additional amount per unit above that - I don't remember the figures, but in any case it was more than we could even consider.
Consequently the conversion module was pulled from the ROM. Briefly it was replaced with a module that just produced a single image saying 'GIF disabled'. Whilst it made the lack of conversion facilities obvious, it wasn't very useful when you tried to just render an image - normally you would expect the inability to convert a file to report an error. So even the dummy module was removed. I set an alarm for the expiry of the UK patent and waited a year or so.
Once the patent had expired, the module became available again, and we could decode the GIFs.
The conversion itself actually produced a collection of sprites in a single file, corresponding to the frames of the image. The sprites could be plotted in sequence to produce the desired animation. This facility wasn't actually used anywhere, but it was available. It also meant that the resulting image that was produced could actually be 24 bit colour, as each of the frames had its own palette and could build up the image. I'm almost certain I never came across any GIF that ever did this, but it was nice to know that the conversion retained this ability. It didn't retain the ability to read the frame rate, though. At least, not so far as I remember. It is a long time ago, so I'm not certain.
The ConvertBMP module was one of the early modules that started out as an ImageFileRender module. I think it was after I had implemented the interface for the module that I decided that I needed to create the ImageFileConvert module to take care of all the common things that I was doing in the converter. Initially it was only able to convert BMP files to sprites, and those facilities were improved as files were found which supported different features.
BMP files come in a number of different varieties, which are distinguished, mostly, by the size of the header they have. Not only that, but some BMP files come from OS/2, which has a slightly different format to the ones generated under Windows. That said, you don't tend to come across as many of them. There are different types of compression that can be applied to the image data, and the module supports raw data, bit field data (where the image has been quantised to a number of bits per pixel), and run length encoding (for some depths).
BMP images can apparently support bit planes (where each bit, or component, is in a different data array), but I never came across any and so it isn't implemented. Where non-true-colour data is being compressed, the image can be paletted with up to 256 colours, and the module does support that.
I had got a module that I was pretty happy with any gave it to the usual group of testers to have a play with - and they pretty quickly reported that there were some images that couldn't be rendered. Apparently they reported that the image files had an invalid header. I had missed (or the specs I used did not say) that a negative height actually means that the row order is reversed. When the height is positive, the image is ordered bottom to top, and when the height is negative, the image is ordered top to bottom. Not a complicated change, but a subtlety I had missed.
As one of the earlier modules to be written, it was the first time I had to deal (properly) with the validation of the input data. The number of BMP files around meant that it was likely that there would be bad data in some which we should validate, and reject if unsuitable. Like JPEGs, they needed to be treated with care so that buffer overruns and the like were not introduced. I'm pretty sure that the code was quite careful about that.
I used a file buffering library for all the data operations, which was actually reading from (or writing to) a limited region of memory. The data size was fixed when the conversions functions were called and subsequently calls could be performed through the library. Because all the conversion functions provided input and output buffers, it was simple to limit the operation of the file buffers so that we never ran off the end, and would always know the limits of the data we were going to write (for the output) even if we never actually wrote the data. I think that the library was heavily based on the original BMP code that I had used as the basis for the converter, but far more paranoid.
One of the fun things about the bit field data was that the BMP could
declare some very strange bit formats for the data - the components could
even be interlaced, eg with a bit pattern
RGBRGBRGB... and the
specification said that it was valid. Windows XP didn't support it when I
tried creating such files, and I wasn't inclined to make the code that
complex if I didn't need to.
However, the module does support any contiguous bit pattern. Many of which Windows didn't support in tests, but were at least valid. So if you had 24 bits of red, then 4 bits of green and finally 4 bits of blue, it would work just fine. Well, you'd lose the bottom 16 bits of red, but you would get a sprite out of it.
I later discovered that there were other formats that the BMP could contain, such as JPEG data, but I intentionally never implemented that. Until I found an actual file in the wild that had JPEG data in, it wasn't worth trying to guess at how it might have been structured.
Once the BMP conversions were working properly, I moved on to other modules and there were things from the BMP conversion that became common. Validating the memory by using a restricted set of file operations became the way that all the modules worked. It made the code a little more long winded, but that's always true when you validate your data.
Later, I came back to the ConvertBMP module after I'd worked on the ConvertWMF modules. One of the object types that WMF files contained was a bitmap object, which could be represented as a 'device independent bitmap' or 'DIB'. These were almost identical to BMP files, with a few small differences in the header format. To support these I added a special SWI to the ConvertBMP module which ConvertWMF could call to perform those operations.
One thing about the conversion of BMPs to sprites that I wasn't happy about was that it could be quite slow. I think this mainly hit the bit field formats as they had to be decoded in a generic function. I had considered - but not implemented - a couple of custom routines to handle the more common bit field formats.
The module also supported the reverse conversion - creating BMP files from sprites. This became very useful once the standard applications started offering the ability to save through the converters, and obviously would be available through the command line *ImgConvert tool.
The sprite to BMP conversion handled most of the sprite formats, finally getting CMYK support at the end (converted to RGB data, rather than being preserved). The conversion didn't support the 'left hand wastage' sprite data format, which shouldn't really have mattered as it was deprecated and should never appear in modern sprites. I only later discovered that the left hand wastage can very easily appear - if you capture a sprite from the screen, the left hand wastage will be used to make the operation very fast, and if you flip an image horizontally, the right hand wastage becomes left hand wastage, rather than being trimmed.
Both of these issues would have been addressed as part of Select 4, I think, or certainly early on in Select 5, but I never got around to them. I don't think that either is particularly difficult to get around, but the problem affects more than just the sprite to BMP conversion - at least the sprite to PNG conversion has a similar problem. I can't remember whether the others do; it's only because I've been converting the images for these rambles that I'd remembered this little issue.
One of the things that was quite painfully obvious about the Conversion modules was that they focused heavily on the bitmap file formats. There were quite a number of bitmap formats that we could convert to sprites, and some (like the JPEG conversions) which could convert between bitmap formats without involving the sprites.
For vector images there were only the DrawFile conversion (to SVG) and the !ArtWorks renderer - there wasn't a conversion for the !ArtWorks format at all. With this in mind, and having obtained a CD of Windows Clip Art for the purpose, I set out to create a WMF conversion module.
I've mentioned the module in passing before, as it converted Windows Meta Files to DrawFiles. DrawFile offered the most obvious parallel to the WMF files, as both were vector images and could contain bitmap data as well. Unlike DrawFiles, WMF files were intended to be rendered to a bitmap type of device, so included operations like flood fill and patterned fills. There were also features like clipping that weren't present in DrawFile at all, which were more noticeable as a missing feature on some images.
I had a copy of
LibWMF lying around from some very early
experiments I had done many years previously. It was old, but it was functional so I
decided to use that. It might have been better to pick a more recent library,
but I wanted to just make something work. The LGPL licence that it was under
wasn't ideal for the work, but partly this was an experiment, and partly I
wanted to release the example source for someone else to finish. There were a
couple of reasons for this.
Firstly, and more obviously, it wasn't as useful to have a generic image conversion system that anyone could create conversions for, if nobody other than me created them. Chris Bazley had created converters for a couple of formats (for Star Fighter image data, if I remember rightly), and Ian Jeffray had created a HTML renderer (not converter), which was equally interesting. But there wasn't much in the way of examples, and I hoped that the example code for ConvertWMF might help that. The more liberal LGPL licence was still unusable as part of the ROM, but could be used as provided independently by people.
Secondly, I didn't want to get into the problems of another conversion module. It wasn't so much that I was tired of it, but more than it took time to get things right, and that time might be better spent in other areas. The ability to create the conversion modules was available, so rather than doing everything myself it could be passed on to someone else.
The conversion itself was only for WMF files - EMF files were not supported. If I had used a later version of the library I think they would have been, but that's the penalty of using what is to hand over something newer and more developed. There were only a few primitives supported in the module, but they covered quite a lot of the common cases.
LibWMF code was really quite nice. It parsed the file and
produced callbacks to appropriate functions in a dispatch table. Usually
that would perform the relevant operations for the rendering or writing of
a different file format. We were doing the latter so a DrawFile was
constructed by the callback routines. Whilst many of the operations mapped
directly to DrawFile primitives, the fonts were the same kind of problem
as they had been when I had written the SVG to DrawFile converter. This time,
however, I had begun to attack the problem of different font naming schemes
by creating a module called FontMap which managed the
conversion from one scheme to another.
The text it produced, even without the font name mapping, looked pretty reasonable. There were a few bits of juggling in order to ensure that the font table was placed properly at the start of the DrawFile - as they cannot contain tables anywhere but at the start.
Bitmaps inside WMF files were similarly interesting. For obvious reasons, the bitmaps used BMP images internally, in the form of a DIB - a Device Independent Bitmap. These could be easily converted to sprite through the ConvertBMP module, which is what I did.
The module itself kept a cache of the converted content, so that if you tried to render the image a second time it would be processed quickly. This was most important because even the simplest of operations using ImageFileRender usually involved a call to read the bounding box of the content, which in the case of a format what was converted through an intermediary (in this case DrawFile) would mean converting the file multiple times - at least once to get the bounding box and then again to perform the render.
Nobody on the internal testers list took up the offer to take over the module's development. My notes say that the offer would be widened to the Select list later, but I don't know if that happened. I have a feeling that it was offered, but don't have any further notes to back up what came of it, if anything.
.ico conversion module shared a lot in common with the
ConvertBMP, but the file format had different restrictions and slightly
different usages of fields which made it difficult to implement them in
the same code. The file format would have had significantly less importance
if Internet Explorer had not begun using it as a way to store bookmark
icons for websites in the '
favicon.ico' files. Its continued
prominence is almost entirely due to this and Internet Explorer's inability
to display any other format for its shortcut icons - leading most websites
.ico format files for their shortcut icons despite most
other browsers supporting other file formats.
In any case, the
.ico decoder is quite similar to that for the
ConvertBMP decoder, but with special cases to allow for deviations from the
standard that were encountered in the wild. Because of the ubiquity of
favicon.ico' files, there were many variations, and the
compliance to the standard varied - probably due to the lenience of whatever
version of Windows the site was originally tested with.
.ico format is one of the few bitmap formats supported by
the conversions which has support for multiple images. Those images are meant
to be of the same icon, at different resolutions. The resolution that is
used in the file (generally) has to be one of few fixed sizes in
order to be recognised. Although it is allowable to have any size, certain
applications (and versions of Windows) would render the images wrongly if
the sizes were not exactly those expected - square 16, 32, 48, 64 pixel
images are generally supported. The ConvertICO module will actually allow
any sizes, but might display them slightly differently to other applications
which might round the size down or up.
.ico files are also one of the few file formats to support
a mask (a feature not supported by the ConvertBMP), which is converted to
a RISC OS sprite mask quite happily.
Late on, I added the ability to create
.ico files from sprites,
in order to make it easier to create the
favicon.ico files for
websites. It turned out rather well, and the ability to export images from
!Paint made it a lot easier to handle such things.
I'm quite pleased with the way that the module worked, but I always wanted to merge the ConvertICO and ConvertBMP modules into one, as they performed very similar jobs. The handling was sufficiently different that it just wasn't worth the extra work. Oh well.
There were a few other conversion modules. ConvertPCX would convert the (now very old) PCX file format, which was once a very popular way of transferring images on BBS systems. You still come across them now and then in older packages as an internal data format, although even that use seems to have vanished of late, with PNG being more commonly used.
ConvertSun would convert the Sun raster file format to sprites. Again this was a very old file format, and one that you would probably never really come across in general use. There's probably a good argument for removing it - it was never particularly common, even when the RiscPC was new. On the other hand, it's a reasonably simple format and was one of the first conversion modules to be created.
ConvertXBM is similar, and even less useful. XBM is a file format that
takes the form of a C source file defining the bits of a monochrome bitmap
with the size of the bitmap described in
It's pretty much obsolete now, but is a pretty simple format to decode.
ConvertPNM amused me. The PNM stands for 'Portable Any Map', and I do remember seeing an email from someone suggesting that we should proof read our releases more, as the format is PNG, not PNM. The PNM format comprises 3 major formats - PBM, PGM and PPM - covering monochrome, greyscale and true colour.
Each of these formats could be represented in either textual formats, or as binary, making for a total of 6 major formats. Because the format is unpaletted, it has limited use for low depth colour images, but it is very useful as a general interchange format because it is sufficiently simple that it is supported by most packages. Because it is not compressed, the files tend to be large, especially those that use the text encoding.
I used the initial code that I'd produced for the conversion modules to create a some example code which was to go out as part of an SDK for anyone wanting to produce conversion tools. In order to simplify the examples, and to convert a file format that was interesting enough to demonstrate the conversions, I created the ConvertDoom module. This would create sprites from the images contained in the Doom lumps from its data files.
It relied on the files being typed properly, and wasn't particularly clever in recognising that the files were the wrong format, but it was sufficient to provide a demonstration of how the converters were constructed.
Ian Jeffray wrote the ConvertClear module, which was quite nice - it was the first module that had been written that wasn't by me. 'Clear' was a file format that John Kortink had created for his general image conversion tools !Translator and !Creator as a way to retain the maximum colour resolution and to be a useful intermediate format. Sort of like the PNM format but only for its binary variants.
Clear files were generally easier to manipulate as I'd previously used them in my own image creation experiments. They weren't, generally, very widely used in the RISC OS world, as far as I could tell, but if you've got a general interchange format it would seem foolish to ignore it!
Ian also wrote a renderer for HTML files, which was rather fun. The renderer only handled the main HTML 2 elements, but that was sufficient to render a lot of documents, even if only in a quite simple way. Turning on the Filer thumbnailing and having a whole bunch of HTML and image files thumbnail was really quite cool.
I think I remember that Chris Bazley wrote some conversion tools for Star Fighter 3000 so you could view the graphics from the game as well. It was a one way process, but still quite useful.
The conversion and rendering system was all well and good, but it needed to be used by applications. The most obvious use for an image rendering system is a tool that lets you view images. That's pretty... well... it's so obvious that I probably didn't need to say it. The !ImgViewer application was intended for this - a basic viewer for the new formats that we could use in the Desktop. There had been a number of different viewer applications over the years, with many appearing when the JPEG support first appeared for RISC OS 3.5 - !SwiftJPEG and !EasyView spring to mind readily, but there were others and they all had strengths and weaknesses.
The purpose of !ImgViewer wasn't to replace any of those applications, but to provide a simple viewer that used the new renderers. I hoped that people would see that it was far more useful to be able to use the generic renderers than to provide their own. There are obviously times when you need your own conversion and rendering systems, but for many simple applications it was useful to use the system that was there - much as JPEGs were there for RISC OS 3.5.
The viewer itself was made simpler as I had created the ImageFileGadget Toolbox Window Gadget (see the later Toolbox Gadget ramble for more about it). All you had to do was to tell the gadget the location, size and type of the image data, and it would be rendered into the window for you. The application still had to manage loading the image file itself, but this was trivial.
The ImageFileRender interfaces were provided through the ImageFileGadget, which made it very simple to add the scaling support. The scale block was supplied to the gadget and that was all that was needed. More difficult was making sure that the gadget itself worked properly in the window. Because the viewing window needed to be large enough to show the image, it still needs to work out the size of the image when rendered - if the image is larger than the screen, the window extent would be increased to allow the scroll bars to move around the image. The gadget would be moved to reflect this.
The viewer supported the basic colour mapping operations as well. These were provided through the 'Options' window. Although all the colour mapping operations from ColourMap could have been used, only the Gamma, Contrast and Brightness seemed sensible to expose here. There's no reason why more couldn't have been added if necessary but, for most viewing, these were sufficient.
There was also the option to change the quality of the rendering. I had intended that the quality be a configurable, and this might have been useful to expose here - as a 'set as default' - but it didn't seem sufficiently important to pursue initially. Similarly it might have been useful to have a 'set as default for filetype', but again wasn't important.
One thing that had been done was to add an 'Export' option to the menu, to convert the loaded file to another type. This saved having to load the file into !Paint (or similar) and then use its export. Or, as I usually did, using the *ImgConvert -t type src dest command line.
The viewer could step through files in a directory, and took its cue from the work I had done on ControlAMPlayer to step into directories when viewing a sequence. This was most useful if you were trying to view a sequence of photos or similar, but I intentionally stopped short of providing a slide show facility. This felt like it was a feature that other people should provide in their own applications.
The viewer did have problems, though. Because it claimed the filetypes of all the convertable filetypes (through the ImageFileRender command ImageFileViewer), those types might not be made to load their editors if you ran them - the viewer might load. Similarly, if you wanted to view the file but had seen an editor, the editor might load. This was a general problem caused by the single 'Run' type action.
Previously I had investigated how we might provide additional actions, such as 'Edit', 'View', 'Mail' and 'Print' - the latter of which had partial support already, but this was not very well provided for. The difficulty with any large scale changes like this was that they would need to be supported by the primary types provided by the OS, and they would need to get sufficient traction with developers. The former was a reasonable amount of work - especially as the main applications weren't particularly up to date on modern features before they started - and the latter was increasingly difficult for even simple features, never mind a change to the differentiation of file operations.
In addition to this, there was precedent that it wouldn't be well used. The 'Print' action, as already mentioned, wasn't well supported, and if you ever look at the file type actions on Windows you'll see the types aren't always extended by applications (can't speak for later versions of Windows).
The ImageFileRender modules made it possible to draw images without having to know much about them. This was most useful with PNGs, which could be rendered with alpha-channel against any background. Only !Browse on RISC OS had support for this, and whilst it was pretty neat, the browser was dead since Acorn had closed up shop and wouldn't release that source.
The !NetSurf browser, on the other hand, was open source. I'm not even sure it had image routines at that point - it might have had its own, or it might have just had placeholders. In any case, I wanted to add some support for the graphics using ImageFileRender. This was made a little harder by a few differences from the intended environment that !NetSurf was built in.
- It expected to be built with GCC - not Norcroft.
- It expected to use a modern version of OSLib - rather than the older version that I had to hand and supported.
- It expected to use curl - whereas I preferred to use the modular URL Fetchers.
- It used an older version of LibXML2 - I was tracking the bleeding edge version of LibXML2 at that time.
- It used a UTF-8 library that I couldn't find.
These are minor issues, according to my notes. I worked around them quickly, building with Norcroft, replacing OSLib calls that were different, replacing the fetching libraries with URL Fetcher libraries instead, and stubbing out functions where necessary. It amuses me that each of those might be considered a problem in its own right (certainly if many of the comments I see on mailing lists and usenet are anything to go by), but it doesn't seem like they held me up much.
Whilst most of the changes were pretty hacky, !NetSurf is built quite modularly (with exceptions for some of the fetching routines which were a bit awkward). Despite my disagreeing with the choice of LibXML2 as a HTML parser, it was pretty reasonable.
The code I added to obtain the images and to render them was pretty simple - there was no caching of the decoded data, so every image would be decoded raw from the source image. This made it a bit slower but it simplified the implementation significantly.
After a few fun problems early on, the implementation worked out pretty well. I made some comparisons of Oregano 1 to !NetSurf for the PNG test pages. The test pages had a few different PNGs that showed off some real images on a web page, including the icicles image which was a favourite of mine.
The UTF-8 hard space didn't fare quite so well (due to my just stubbing the '
libutf8' library that I couldn't find).
I contacted James Bursa, who managed the !NetSurf project, and showed the screenshots of !NetSurf running with alpha-channel PNGs, once I'd got things to a state where they 'worked'. He was quite enthusiastic and encouraged me to submit things to their CVS. Sadly, I'm not sure that I ever did. I had only set aside a 'morning after real work' (apparently that stretched out to about 7 hours - I must have been pretty tired by the end!).
I gave them the diffs, and an explanation of what I'd done so that at least they'd have some idea of what I'd done. Things like the replacement fetcher code was pretty icky and incomplete, but it is nice to know that you could replace the fetcher system in such a short period.
I was really quite pleased with the way that the images came out and, whilst they were slow to render, for a tiny project to just see what I could do it it was really quite reassuring. These days I can't really imagine doing anything like that.
The !Paint application is one of the more obvious applications to update. Whilst it is functional, with the Wimp changes for the clipboard and the graphics system changes, there were new features available and some things that could be managed better.
Initially the changes to !Paint were quite small. I started out by adding support for 'multiple I/O' - a way to save out different file formats. In the main 'Save' dialogue it was possible to select saving as either a JPEG or a Sprite through radio icons that appeared in the Save dialogue box. Behind these, the save code had been reworked so that the JPEGs could be created through the CompressJPEG module.
I added a new system variable to control the quality of the JPEG produced.
Previously the 'configuration' had been made by setting the
Paint$Options variable, so using a system variable made sense.
Paint$JPEGQuality variable allowed the export quality to
be selected. Although the feature wasn't exported in a configuration
window, it was thoroughly documented in the !Help file.
At the same time (actually about 6 months earlier, but in the scale of things, that's about the same time!), basic support for PNG import was added using the PNG library to import the files. The basic structure of the import functions eventually became ConvertPNG, if not the implementation. !Paint uses the Flex memory manager, which is something of a fun beast to work with. As a shifting heap memory manager, it requires that you deal in references to memory areas, which can move as allocations take place. This makes some operations quite complicated and means that you have to be quite careful about the areas you are referencing.
JPEG import had been possible for some time, but the changes to be able to import PNGs meant that the import code was restructured significantly into a list of 'importers' in a similar way to the exporters which appeared in the save dialogue. A fun part of the JPEG import was that it redirected the output to a sprite and plotted the JPEG into the sprite. Usually this was fine, but because of a bug in the sprite system, redirecting to a sprite which was 1 pixel high would cause a crash. To avoid this problem, special case code in the importer would create a sprite 2 pixels high, and then delete the redundant row. A poor solution, but one that worked - the core issue was eventually fixed (see the 'crashes' section in the Sprites ramble which follows).
The initial rudimentary support for clipboard was just to have some simple menu entries for Cut, Copy and Paste which directly called the ClipboardHolder Module. It wasn't very pretty, but it did a nice job.
When CMYK sprites were introduced the internal representation of the colour number had to be changed, which caused quite a few changes across the source. Previously it had been a single integer which represented the colour number, mask or ECF pattern. This couldn't work when the colour number alone was 32bit, as would be for CMYK sprites. Consequently the colour number had to become a structure defining the type and colour.
Eventually all the special case importers and exporters were ripped out and replaced with more sensible code that used the ImageFileConvert stack. Ian Jeffray did a huge overhaul of !Paint, fixing many of the problems I had introduced and generally making things a whole lot better. He added the new toolbar, the more useful scale and colour selection pane, the hugely reworked configuration window and a more useful screen capture tool (amongst many other things).
One of the amusing things about the export and import through ImageFileConvert is that !Paint doesn't care what you give it - it will try its best to load the file. The screen capture window offers the option to save the capture in any format. Whilst doing the screen captures for the Rambles I regularly export a window capture as a PNG, and it's very nice to be able to do that by just clicking in the window and then saving the newly created PNG to disc - no manual conversion involved.
It is more amusing when changing to the capture a rectangle of the screen, though. Usually I'm doing this so that I can get a few windows into the image, and then mask out the background. If you accidentally left it set to export PNG, then it would do so - you'd end up saving a PNG of your captured rectangle. Which is all fine, except that you want to save this back on to !Paint so that you can trim off the edges and mask the region... which you do and it just works. Screen capture to sprite, converted to a PNG, then converted back to a sprite when loaded into !Paint. Not exactly the most efficient of operations, but it all works smoothly and without you really noticing.
I'm pretty pleased with it, to be honest. Although a lot of the end work which is really useful was done by Ian, I'm pleased that the ImageFileConvert stack work made the application more useful - after all that was the whole point.
The !Draw application didn't have as much opportunity to use the ImageFileConvert stack because there aren't as many vector formats that we can convert. The bitmap formats could be converted and imported into the documents using ImageFileConvert, in the later versions. At first, though the application gained the ability to export SVG from the DrawFiles. This code became, like the PNG code in !Paint, the basis for the SVG converter in the DrawFile module.
Like !Paint, the menus were reorganised when the clipboard operations were added. In particular, the clipboard operations were made the primary copy and paste operations - the old copy operation was renamed to 'Duplicate'.
One of the amusing effects of the ImageFileConvert generating Sprites or DrawFiles from the types imported was that even !ArtWorks files would import - albeit converted to a sprite.