DocumentsTab

= Introduction to "DocumentsTab" for Opie 2.0 = This is a small summary of a discussion by Stefan Eilers and Holger Freyther Copyleft by Stefan Eilers (eilers@handhelds.org) and Holger Freyther (zecke@handhelds.org)

The document centric idea is a very generic philosophy about handling documents without having to care about the physical structure of the device currently in use. Therefore the user should not care whether a mp3-file is on /mnt/Documents/whatever.. He just wants to use the file.. Trolltech introduced this idea in its Qtopia-platform, but its implementation caused the system to be nonresponsible for a long time as an external device was inserted. Sharp decided to remove this idea by showing a reduced directory structure (without configuration and system directories/files). Although the user is not irritated by strange system files, he has to tab through a filesystem to find his file..

Our motivation is to let the document centric idea survive, but make it more performant as the current Trolltech implementation will ever be.. AND .. let the user the decision to replace the document-tab with something as sharp has or with a real file-manager..

This modification will be implemented in Opie 2.0 due to the fact that it need a complete redesign of internal structures. The document-tab is a very basic element of the system, which needs more than just a patch to be satisfying..

Ok.. Let's start to introduce the implementation idea:

1. Remove the Document-Tab.. Well, yes you read it right! We want to replace the document-tab with an application/plugin which is shown _for example_ in a special tab in the launcher. Thus, we will call the "DocumentsTab" "DocumentsViewer"

2. The - so called - DocumentsViewer will just scan the fileystem when it is activated/used and just for the information which is selected (mimetype & category). Thus it will stop to scan the filesystem of an currently inserted media and delay the whole system, while nobody is interested in its results.. Basicly the application will be build around a widget which can be used as FileSelector replacement.

3. The idea of adding special files (*.desktop) for mimetype and category is too slow.. It will be replaced by a special fileystem structure:
 * Every document has to be underneath "/Documents" on external medias and "/home/$USERNAME/Documents" of the internal media.

Hint from Alwin: It would be better to use "/Opiedocuments" on external media so we will not conflict with possible other devices using this media eg digital cameras.

Hint from LJP: What if I put in a cf from an mp3 player or sd from a camera, it will most likely NOT have a Documents directory, and thus you wont see any of the files.

Answer from eilers: This is the reason we need an import function which is activated by demand. See MediaScan

Thus the previous mp3-file will be stored in the follwing directory if it categorised as "stupid_stuff": Documents/audio/mp3/stupid_stuff/Vater Abraham und die Schlümpfe.mp3
 * The directory Documents will be split in several subdirectories for available MimeTypes. (MimeTypes are already directory based like for instance "text/html" or "video/mpeg". Thus, a mp3-file will be stored on Documents/audio/mp3/Vater Abraham und die Schlümpfe.mp3 ;) )
 * Beneath the MimeType directores, there may be directories representing a category to classify or grouping the documents ( I would prefere just to take the name of the category itself (KISS)).  Files which are not in a category-subdirectory are shown as "unfiled".. If the user changes the category, the file will be moved into the corresponding subdirectory.

alwin: IMHO that makes no sense. As discussed with zecke I would prefer a structure for categories described within 'Document structure' on this page. eilers: Why ? The only disadvantage is, that we don't have multiple categories. I think the advantages are much more, especially due to the fact that this structure is simple and fast !


 * These kind of categories are the usual and we allow subcategories which are subdirectories of a category directory.
 * We merge categories of physical different locations into the one 'virtual' Category. Thus all files of a category are shown at the same time, independently to their physical location.
 * We allow scanning recursively from a Category for any MimeType ( even all )
 * We either show only known MimeTypes or we allow to show unknown as well.
 * Also we might have a virtual PIM folder which gives access ( Delete, Edit, Add, Beam, Print ) to your records.
 * Scanning and recursive scanning of files is only done by user action and not on creation.

The DocumentsViewer should provide the functions Add-Category, Rename-Category, Delete-Category; select category for a selected file and change category for selected file, and of course rename the file and move the document between medias (internal, external cf/sd, network, ...). When moving the structure is kept so if you move something from CF Ton Steine Scherben category to Internal Storage you also don't need to worry where it copies them. If you want to worry about it, go and use the AdvancedFM. ;) Internally these functions are mapped to file-movement ( for instance if a category or the media is changed ), adding or removing directories. The document function will be strongly tied to the concept of a service. A document capable application offers functionality it can do on a document-type. For Example: A media player could do read only audio play(play the song), add to play list, show tag info but the song is also a file (inheriting of mimetypes) so you can also do file operations on it delte and copy.(eilers: Den Satz verstehe ich nicht ! zecke:Better? ).

The OServiceManager would return a list of UI string of things it can do for a MimeType you could easily add them to a QPopupMenu. When a user choses an action you would ask the ServiceManager again to call the right application with your data.

There should exist a MediaScan which allows the DocumentViewer to search for Files outside of the "allowed" directories for "orphaned" files and to move it into the right subdirectory Documents/.

zecke: Hmm I would vote for AdvancedFM or 'Fixup external structure'. eilers: I would prefere the last one.

The advantage of this structure is the following:

The DocumentViewer doesn't has to search the whole media. It could exactly select the right subdirectory-tree to scan for corresponding files, if the user selects the _mimetype_ and _category_ he is interested in. And it don't care for any meta-files which has to be scanned and held to be in sync with the existing files.

This should prevent that the document centric philosophy will be unusable if somebody stores a lot of files on his device..

Besides that we also need a better setDocument function for document centric applications. Currently the problem with beaming is that you don't know if you should delete the object or leave it on disk. Either we add an additional bool value to the call or create a class like AppLnk/DocLnk.

Workflow on Apps: Document centric apps should either start with a new document or with the FileDialog with the Filter set to the Document supported MimeTypes. Or we could make the Document Browser a central entry point.

= Design of Document structure = small structure design draft from alwin (alwin@handhelds.org)

The first idea of store files was putting them into Opiedocuments/MimeType/categorie. But with this we will give up that a document can be member of more than one categorie.

So it would be better to hold two directories on media


 * FS/Opiedocuments As discussed all documents are here within Type/subtype/.
 * FS/Opiecategories Here we will store a sqlite db where to each document the categories are assigned. If no categories are assigned to a document, there will be no entry in db. This could be get the db smaller. The key is type/subtype/filename (in this order). Only files beneath Opiedocuments can get one or more categories assigned.

zecke: The problem with the extra categories is first we need to keep it in sync. If a document got removed we would need to update the db. Also for unfiled records we need to do reverse (expensive) look ups. The question is if multiple categories are really worth the trouble. What about supporting multiple categories only on ext2? (symlink/hardlink). With only one category we do not need any additonal book keeping. On the other hand with a db index we could keep additional information set by the user. We could also keep the mtime/dtime of the directories and then in a background job do the book keeping. The only two things that could happen are. 1) removal of a file 2) addition of a file. Ok some more discussion is needed

alwin: Well, we should try whats faster: Searching for categories specific documents via directory browse or via select inside a database. Means: User selects displaying documents of a specific categorie, but no specific mimetype. In such case we must go through the entire directories for finding all matching documents if not db based. With db its just a select ... where categorie = 'foo'. I believe the select is faster. Adding files: As said - documents with no db entry have no categories. means, if a user puts a file via a filemanager into that structure the file has no categorie. And if all categories are removed from a document, the db entry will be deleted.

Removing files: if a select ... where categorie... returns a file which not exists on storage, the entry will removed, eg., we can check the result of a select to get db clean.

When removed with the documentviewer the entry in db will removed, of course. Adding via sync or import via documentviewer will not add a db entry, 'cause in that case a document hasn't a categorie. And we can implement a "Clean document database" option user can call some times if wanted. But in most cases it will not needed, 'cause most user will use documentviewer for document related operations.

For getting other files not beneath Opiedocuments into categories there should exists a "Import documents" structure, for example if using a CF card from a digital camera. On that DocumentViewer looks through the directories setup in mediamount (or root of fs if not set) and moving that files into Opiedocuments/ (and it should realy be a move) (may be to another storage) and try to sort them into the right mime-directory. If no mimetype can assigend it should be application/unknown. eilers: The was ment with 'MediaScan' !

To be continued...