Filer

Keyboard navigation

I had, for quite some time, wanted to include keyboard navigation in the Filer. This isn't entirely simple because not only do you have to implement it, but you have to get the interface right otherwise it will be an alien system that people don't want to use. As Filer is quite central to what people see as RISC OS (for reasons that I don't entirely see), this has to be done with care.

Adding thumbnails did not significantly change the way that the Filer worked, so couldn't really be viewed as detracting from the interface, but keyboard could find itself being a more intrusive change. I'd previously worked with some patches (and re-implementations) that used the keyboard and I didn't want to either merely emulate, or to follow the failings that they had.

Under RISC OS, applications gain the focus and tend to remain with it. You transfer files in to an application by dragging (let's say) and they appear either where they were dropped, or at a 'cursor'. The Drag And Drop protocol has some rules here, but doesn't talk about focus, so isn't strictly relevant - although its understanding of drop location may change in context if there is a current focus to target the drop.

For example, dropping on an inactive area of a window in an editor might normally insert at the current focus, but if there is no focus it might insert at the end of the document. (I'm not claiming that is a good or bad interface, just that nothing precludes it and it may make sense for some applications).

In such circumstances, if the Filer gained the focus when you clicked on it, dragging an object from a Filer to such an editor would mean that you could never drop on that inactive area to insert at the focus. Your initial click in the Filer window would give it the focus.

Similarly, the dropping a file on an editor wouldn't restore the caret (why should it ? There is no reason to believe that the click was intended to be transient for the Filer), so if you started typing, thinking you were now going to enter text at the focus in the window you had just dropped on to, but the focus would still be with the Filer. Unless you can include some simple to understand rules, the focus has to operate the same for Filer as it does elsewhere.

However, using a special modifier sequence to give the focus to the Filer was wrong - you don't do that anywhere else in RISC OS. Except, of course, in the Filer, where Alt-click had been added to rename files - a move that I disliked, but we were stuck with. 'Alt' modifiers were not meant to be used in general.

Similarly, cut and paste had to work properly. You should be able to cut a file, and paste it elsewhere. But also you should, logically, be able to paste non-file content (eg cut text, objects or other data) and have it appear as a file - the Filer should not be special.

This also implies that cut (or copied) data in the Filer should be pastable elsewhere. This raises implications for what you do if you want to cut a number of files (rather than just a single one) and then paste them elsewhere. Should the Filer be special in understanding that these files are cut, or should it expose this to other applications ? If it doesn't expose the ability for multi-file copy and paste to other applications, that means you cannot copy from Filer and paste to Pinboard (or vice-versa).

I had considered using a special filetype for 'File Selection' which was a text file containing the list of filenames in the selection to be operated on. This seemed the best of the worlds - copy or cut would create such a file, and paste would be delivered as a 'File Selection' which - if the application understood it - would be translated into either references to the files required (eg in Pinboard's case it would pin them), or operate on the actual files.

But there was still the question of what happened if you were to just create references, as would happen with Pinboard. A 'cut' operation followed by a paste to a reference taking application would result in the files being removed, and thus breaking things.

These issues, as well as how you present the focus in the window, whether there is a 'focus but no items highlighted', whether focus on a file implies that the file is selected, and all the issues about the right key bindings, were a constant source of frustration to me and I couldn't really resolve without properly sitting down with the issue - which was hard when there were a lot of other things that needed dealing with.

Having procrastinated about these issues for a few months (or more - the need for keyboard control had been there for ages) and having been unable to come to any sort of conclusion about the right way to do things, I decided to ignore the issues that were blocking me and get the basic support in to the application so that there would be something to try out. Hopefully then it would be easier to determine the correct way to go.

That kinda makes it sound like there wasn't a design for how it was going to work, and I was just crossing the bridges as I came to them. That's partly correct - I knew where I wanted to be, but some of the more specific hurdles, like the Cut-and-Paste model could be determined once there was a real implementation to look at. Different models could be tried out with some test applications to see what worked.

Easier to see the problem once it's in front of you, rather than far in the distance. I've never been good on detail in the distance. I usually see the shapes that they should be - and the shapes that they cannot be - but tying down detail is often harder. Often isolating the distant problem and tackling it directly helps. Logically I should have started by defining the cut and paste model, therefore, and trying to decide on focus issues.

I had previously done a lot of work with Cut-and-Paste - one of the big reviews during RISC OS 4 development was of the support for clipboard within applications. Somewhere there is a big text file containing a list of every single application which supported the protocol, whether released, test tool, or prototypes written by myself or others for diagnostics, and giving their conformance to the protocol and particular problems interacting with other applications. So I knew the protocol pretty well, and never felt a need to go further than that (it's tricky but needs thought).

The focus issue was probably impossible to resolve to everyone's satisfaction without making everything optional, which increased the diversity and number of tests that were required. I had previously played with enforcing focus-stacking so that you could always return to a previous focus window when the one you were finished with closed, with varying degrees of success. In generally such attempts were made more difficult by applications that tried to perform the same kind of stacking themselves. If applications were to just set the focus and never try to restore to another location, then all was well. Similarly, the restoration of focus when you complete an operation - other than a menu operation - was generally unexpected, so you often found that you were trying to put the focus where it had already appeared.

Amusingly, one of the experimental features that was found to interact badly with some applications did effectively that - if you tried an Alt-click rename of a file it could restore the focus back to the parent window. However this had particular issues if the focus was taken from a transient dialogue box as the focus would not return properly. That feature was, like the keyboard navigation, left disabled until it could be either fixed, or made less confusing.

Anyhow, aside from having played with, and understanding, some of the issues that I had raised, I ignored many of them and charged on with adding some keyboard support.

I chose a simple model where the focus could be given to a Filer window, and would initially start having highlighted no items (focus on window, no object highlighted). The would effectively be item 0, just top-left of the first item in the window, logically. Pressing the cursor keys would control the highlight, so that either the first or the last item became highlighted. There was no way to restore back to no-highlight, except by moving the focus elsewhere.

The highlight itself used a dashed box - the same way that I had presented the highlight in !AMFiler. Obviously !AMFiler provided a lot of the playground to try out different key combinations previously. I had played with using a solid background, with !AMFiler, but this looked garish and was soon abandoned. At no point with Filer did I consider that the highlight was a selected item, and similarly, moving the highlight would not discard the selection (q.v. icon selection model in the Wimp, where this was configurable).

New scrolling operations were required so that it was possible to highlight an item, and for it stay in the visible Filer window. Not particularly complex, but important from a user experience point of view. And there was a need for 'search as you type' - a vital feature that had to be right or it would just annoy. This was partially implemented, and would jump you to the matching name that you typed - so long as you typed it quite quickly. Pausing for more than a second would clear the text from the buffer.

There should have been a feedback on the search, though - an indication of what you had typed. This hadn't been done because the basic implementation worked and the whole feature was incomplete. When working out the key bindings, it seemed reasonable to use backspace to go up a directory. However, this made for a frustrating experience when using search-as-you-type. If you forgot that backspace moved to the parent and used it to correct a mistake in the search-as-you-type, you would find yourself cursing as the window changed under you.

The bindings themselves were mostly those that seemed sensible, and which could be implemented easily. In many cases the operations already existed as menu items, so just making them triggerable from a key press was relatively easy. However, the bindings were fixed. So if (I can't remember the way I set it up) 'change sort order' was on ctrl-S, that was the way that it stayed. This had issues in other languages, as 'S' might not be at all obvious for 'sort' in that language.

Because of the Cut-and-Paste issue, there was no way to trigger the delete or copy operations.

Like a few other features, this option was left disabled in all releases I made because it was just too unfinished. It had reached a useful point but needed those rough and missing areas to be smoothed and filled in properly, rather than delivering something half-baked that needed to be reworked later.

Panes

Of the features that people said they wanted, I don't remember that anyone actually asked for Panes. I'm pretty certain that it was something that I thought was useful and necessary for the future interfaces to the Filer. I could be wrong, but that's my current recollection.

Filer is a block of reasonably sane assembler which is a bit of a pain to maintain, but mostly sticks to some decent design internally. After I had reworked the code by moving things around and adding new bits, it was even pretty well organised. And, of course, parts of it were written in C, now, in order to make it easier to update.

One thing that is harder to do with assembler - not impossible, but generally more time consuming and error prone - is significant changes to the design. Higher level languages make that easier. So making larger changes to the way that Filer worked would mean significant and riskier changes to the code. As I was averse to writing more assembler than I needed to I was very keep to move away from the basic Filer implementation that was provided. However rewriting the Filer in C wasn't my idea of fun. Well, actually it would be reasonably fun, but compared to other things, it's not a great use of my time. I couldn't see it happening as there wasn't time and the likelihood of breaking functionality was pretty high.

Plus it would take significant time to the detriment of other things - whilst it would be useful, but it would very likely end up being canned because it took too long and wasn't actually as good as the original. As a result, I took the hybrid Assembler/C approach I've discussed.

So larger features needed to be done in a way that would not be to the detriment of the current system. The approach I took was similar to that of browsers and the like - to allow registrants to add nested windows to the directory displays to augment what is already shown.

Providers of Filer panes could attach themselves to windows using a very simple message to the Filer, containing the window handle that should be attached to the viewer. Each pane can be attached to one of the 4 edges of the window, top and bottom, left and right. The former pair are positioned first, and then the latter pair take up the remaining space.

Stacking panes vertically on the left and right was not supported - that would be an area that a pane handler would itself need to determine. For example a left side bar manager might offer the ability to register a number of regions which could be positioned vertically within the space the pane offered, which might be in the same form as the filer messages if so desired. This keeps the complexity out of the Filer and leaves the implementation of such styles to supporting applications.

The filer would automatically move the panes around (using the nested window interface) as the window size changed, or as other panes were removed. For example, if a top bar was removed, any left or right bars would need to be moved up the viewer. If two left bars were present and the leftmost was removed, the other would need to be moved left.

If any application exits without clearing up, its panes are considered removed (as obviously the windows they refer to would have gone). This was both more robust (clearly!) and meant that the isolation of the processes made all the operations safe in the face of problems during development.

There were messages in both directions for the Filer viewers. Notifications about viewers opening and closing were vital to ensuring that pane handlers could attach themselves to the windows, and know when they were no longer required. If panes wished to either shrink, or disable themselves if the viewer became small, they could do so by issuing a message to the Filer to change their size (either the height for the horizontal bars, or the width for the vertical bars).

The Filer would also send messages when the viewer state changed. So when the highlight or selection state changed in the viewer, a notification would allow panes to update themselves to reflect the new state. Although the panes would (probably) be interested in this information, the message was broadcast so that others could handle it as well. In some cases there might be applications that did not sit as a part of the Filer, but want to know about it. At the time I think I only had informational applications in mind, but you might also use it for a wider interface, similar to the styles that are currently being used for tablets. Probably the Filer would need other changes to handle this sort of thing - a window area larger than the directory being displayed (which, amusingly, was a feature I'd had to add to my Filer library when I created the !AMFiler browser plug-in) being a particular one that comes to mind.

The panes could request information about the viewer, such as the highlight, or the files that were currently selected. This allowed them to only change when a button was pressed or similar - think of an application that remembered the selection state, or more likely which wanted to know the state of the viewer when it started up.

The panes could also request changes to the viewer. Changing the highlight, selecting files, or performing any of the operations which were provided on the menu. Because all of these operations had been isolated previously by the tidy ups and then even more by the keyboard operations, it was incredibly simple for the Filer to be able to expose these as operations.

Prototype pane clients in a simple window
Prototype pane clients.
The whole interface was pretty simple really, and writing clients was incredibly easy. I knocked up a couple of simple examples which could be used to test the operations of the panes under more realistic use. Despite their seeming a little redundant - because I was used to RISC OS Filer windows not having them, and still did many things the old way - they showed that the interface worked. The applications were never meant to be released, at least not in the form that they stood. I am not sure if they ever were, but they did the job of testing the interface.

The prototype panes image shows the basic button bar and a location bar, together with a sidebar that would use ImageFileRender to show the content of any images highlighted. Because the highlight would only appear when the keyboard selection was used, this wasn't as useful. If the keyboard selection had been completed, this would probably have become more useful.

The button bar just takes a few sprites from Opera which were lying around from the IconBorders testing - they are not at all indicative of the style that would be used eventually. The buttons would probably be updated to match the style of the !Paint and !Draw applications. The prototype clients would shade buttons if they were unavailable, so the delete button would be shaded if nothing was selected, for example.

The different toolbars were defined to have specific purposes, so that when people added their panes they matched with the general locations that were used by others. The top bars were intended for accelerators and common operations on the window. The left bar should be viewer specific operations. The bottom bar should be status specific. The right was undefined - I guess it could be similar to the left bar, but I didn't want to tie it to that.

There was still some work to be done. Enumerating viewers wasn't possible, I believe, so you couldn't have an extra application that would generate a stacking list to select windows. There was no way to do 'scroll to' type operations in the viewers, except by changing the highlight. There were a whole bunch of things that hadn't been decided about how the Filer would react to operations - and a lot of testing to be done to make sure that it actually did work the way it was meant to. The whole panes implementation was left turned off because of this.

It could have been very useful though, particularly for the side bars providing information about the objects selected or highlighted. With a bit of work it could also have provided additional facilities (as suggested earlier) for multiple internal extensions to the side bar - allowing different information providers to give their own representations of the data.

Similarly, providing a tree structure of the filesystem leading up to the current viewer as other systems do, would be relatively easy. As an alternative to a tree structure, a 'breadcrumb' trail could be used in place of a location bar. Each component of the trail being able to provide a menu of alternative directories, the selection of which would relocate the viewer to that location.

One that I considered as a good candidate for a future work was to reuse the DrawChart to provide an indication of the disc space, maybe as part of the Free module. That would increase the complexity of the Free module, which was written in Assembler, so it might be a good opportunity for a rewrite in C. As it is a very simple application this would not be particularly difficult.

Experimental filer panes
Some example panes, a status bar, a toolbar and a side bar.

Filetypes menu

Back in the days when I used Filer+, there were some enhancements that were nice (well there were quite a few!), but one of them was the ability to have dynamic file types menus. This allowed you to easily set the type of files to those that you commonly used. Normally this wouldn't be done that often, but it wasn't unusual to create a text file and then set its type as HTML for viewing in a browser.

I added a system variable - Filer$Types - which was parsed to generate a list of types in the menu, below the main writable field. The value was just a comma-separated list of type numbers which should appear in the menu, in the order they should appear. If the list was empty or unset, no extended list would be shown, but when it was there you could select the types more easily.

Changing a Resource file's type, with Text and HTML in the menu
When set to FFF,FAF the menu includes 'Text' and 'HTML' entries.

The plan had been to create a simple configuration plug-in - with a scrolling list showing the filetypes which you could add to, and drag around - which could set up the list of types that were included in the list. It would also have been nice to include the 'small_xxx' sprites in the menu as well, to make it easier to identify the types.

Draw sprites

The Desktop rambles touched upon the different sprite styles that RISC OS used over the years and the problems that they caused for anyone wanting to theme the OS, and even simple application developers fitting in with the way that applications and data files looked. This was only going to get worse as we moved to using '11' style high resolution sprites. These would be 68 pixels by 68 pixels for standard file sprites, rather than the 34 pixels by 34 pixels they had been under '22' style.

Originally, the sprites were '24' which meant that the sprites were 34 by 17, which is both an odd number and hard to design for. Those original sprites had pixels that were twice as high vertically as they are horizontally, which meant that any border around the icons had to be 2 pixels wide but only 1 pixel high. Similarly any lines could be more detailed horizontally than vertically. This meant that there were some interesting design decisions made by developers to try to make the designs look right within those constraints.

As the sprites were generally created by hand, in !Paint, this also meant more thought and more work for the developers - especially when it came to creating the '22' variants later. Most people didn't bother with the '23' variants, as they were less commonly used. To make things more fun, the sprites used were originally expected to work in the standard 16 colour desktop palette. As anyone who has drawn low resolution sprites in the past will tell you, getting a design to look good under these constraints was a skill that took a bit to get right. I never did really <smile>.

As we moved to RISC OS 3, the sprites began to move towards using '22' as the dominant type, but the design features from the earlier style were still present. The Archimedes 'A' styling still pervaded over the system, the borders were still 2 pixels wide (and now 2 pixels high in the '22' style. There were some themes that people produced of different file type icons and associated applications. I used one for a long time, before it became too much effort to update the theme when other applications appeared.

With RISC OS 4, Acorn had made some decisions to cut some of the legacy ties for the icon style. The Archimedes 'A' was gone, replaced by a green Acorn logo. Sprites were now 256 colours, which gave much greater freedom to designers to produce more subtle icons if they wished. The 2 pixel border was replaced with a single pixel border (in the '22' style), and data type icons were given a little page curl in the corner. They still couldn't use deep colour (16 bit or 24 bit) sprites because these wouldn't render in the selected or shaded form due to the lack of colour mapping support.

Acorn also made a small break from the previous style of resource and support directories being a directory sprite with an indication of its use added (q.v. !System and !Boot). Instead, resource and support directories would be green, and regular directories would remain blue - albeit a softer blue with a slight raised edge.

I liked this distinction, but Paul didn't, and so RISC OS 4 was released with green directories as well. Hey-ho... I merely replaced the green with the earlier blue on my system and I was happy - I'm all for keeping things consistent if that's the way that things have been decided, but I didn't like the style myself.

People began to use deep modes more regularly for the desktop, and we wanted to be able to present true colour icons (and have thumbnails, as has been mentioned!), so Select introduced colour mapping, which allowed the deep sprites to be rendered highlighted when required. We weren't using this much, but it was important to have the capability and not be shackled by the limitations of the past.

Most of the sprites that were being created (certainly by myself, but I know that others were doing the same thing) were now being produced in !Draw or !ArtWorks (or other packages) and converted to sprites at the last stage. Thomas Olsson's mksprite tool was my preferred choice for the last stage, but !ChangeFSI fared well even given its age. Using vector formats meant that a variable resolution version was available, giving much greater flexibility for designing the icons. The resolution and colours were sufficient that it was much easier to render images without too much manual modification.

The DrawFiles (and Artworks files) don't include scaffold lines or hinting, so designing scalable sprites still took a little bit of effort and skill, but even simple designs tended to look reasonable within the new constraints. This would become more useful as we moved to the high resolution '11' type sprites - sprites with higher quality could be created by modifying the vector images to add more detail (if necessary).

For design purposes, all the later Select sprites were designed in !Draw and exported. Aside from a few minor tweaks, these were used as they came. This made generating new sprites significantly easier. Moving design components between images, so that the images looked consistent, was considerably easier in !Draw than using the paste brushes in !Paint. It was also simpler to replace a design component if an enhanced version was made - or the whole component was replaced with a design which looked different.

The Filer views themselves were of variable size - there was 'small icons', 'large icons', and 'thumbnails', which were significantly larger. The thumbnail mode couldn't use larger sprite icons, though, because there weren't any. In theory, I could have created a 'large_xxx' group of sprites, but I had decided that this was just going to cause more problems for designers in having to create them.

The solution was that the DrawFiles that were being used as part of the design process would become the images to render at larger scale. These would become the primary icons for applications. It would mean a little bit of a rework of the internals of some components - the Wimp and the Filer might bear the brunt of that - and some new things would be needed to support the new pools of icons at different resolutions. But it would be necessary to allow for a more scalable, more flexible and more modern desktop design.

It would be possible to make files highlight be increasing their size, retaining a reasonable quality, as well as making colour highlights. The old system would still need to be supported but the increased flexibility that would come about would also bring with it acceleration - the pools of icons would need to be managed, and if the pools are managed there is a greater scope for acceleration. Of course that doesn't hold all the time, but the current way in which sprite pools were manipulated made acceleration more difficult. All the work in this area would help, even if it wasn't immediately apparent.

External rendering

In preparation for providing a means by which icons in the desktop could be rendered differently, an 'external rendering' interface was added to the Filer. This was nowhere near ready for proper use. The changes, though, would prepare the Filer itself to be able to request rendering of an external handler. Part of the idea was that the thumbnailer could also become an external renderer, instead of operating inside the Filer proper.

The code wouldn't be used for Select 4, and might not even see the light of day for the next release even, as it wasn't specified well enough, and was highly specific to the way in which the Filer currently operated. There were a whole host of things that needed to be done to Filer to bring it up to modern standards, and the existing behaviour didn't fit that well. The changes were needed in quite a few places, because the data stored for processing the render data needed to be manipulated across a lot of the parts of the system.

The idea of external renderers goes way back, though. Back in the dim past (not that any of these rambles isn't already a dim past) Thomas Olsson had created !FilerPro. This was a complete Filer replacement, and it did an excellent job. There were a few downsides to it. It used Toolbox, so was expensive to use on my A5000. It cost money - only the demo had been released, and to get the full version you had to pay money. Being a poor impoverished student (aww), I looked upon this as a challenge. It wasn't possible to find the places where the code had been nobbled into being a demo. Thomas later explained what he'd done there and I doubt I could have fixed things without wasting huge amounts of time.

However, there was a separate benefit to the system. It had thumbnailing support. In the background it would thumbnail the files, and you could extend the things that it would thumbnail through some external rendering modules. A few were supplied - for JPEG, DrawFile and some others. The interface was documented in the instructions, as obviously Thomas wanted people to develop other renderers, and it was a great idea.

I took the interfaces and created some simple code that could call them and manage the interactions with the thumbnailers. A simple filter and WimpSWIve wrapped around the Filer to trap the rendering of icons in the Filer and diverted these through the thumbnailing code. If a thumbnailer could handle the file format, they would be processed and the resulting image rendered. Magic - Filer handling thumbnailing, and I'd actually done very little of the heavy lifting. Patching the Filer through WimpSWIve was (to me) a trivial process.

However, I couldn't do much with it because the thumbnailing modules were Thomas' and it wasn't a lot of use without it. I gave the results to David Thomas to have a play with. He rewrote the thumbnail parts, fixed, tidied up and otherwise recreated the whole thing himself. The result was !PhotoFiler, which did a far better job of thumbnailing than my dodgy hack had.

Filename filtering

In its rename dialogue, Filer had been updated to support the use of spaces. These would automatically be converted to hard spaces before the rename happened (just ignore the fact that this is ISO 8859 specific). This was acceptable as we had been requested to support spaces in filenames. As we couldn't actually use real spaces in filenames (due to oh-so-many interface issues that it would cause), this was the best substitute (again, ignoring that it tied you to ISO 8859).

However, if you entered a filename with a space in a dialogue box (eg the Filer 'Copy' dialogue, or any application's save boxes) and then tried to save a file to another application, you might find that the file wasn't saved properly - the space might confuse the application, files might be overwritten if the space was treated as a terminator, or the application might just crash. Saving to Filer was OK, because it would convert the name and reply appropriately.

To fix this problem, Filer places a filter over all tasks and traps the Message_DataSave (and friends) to replace the spaces with hard spaces appropriately. This meant that, assuming the application let the user enter a filename with a space in (many had validation strings that did not), the user was oblivious to the fact that the 'spaces' they were seeing were not actually spaces. They got the support for them.

Some network filesystems did get a bit confused at the names, but usually it wasn't so much the RISC OS filesystem interface, but the remote system which saw the hard space as something different. Usually it came out as a broken UTF-8 sequence, but it depended on the system.