SyncingApi

= Opie PIM, Syncing an the future. =

This represents the result of chats, emails and telephone calls between tfootit, eilers and zecke.

Currently 3rdparty syncing applications communicate over the QCOP bridge to get the path and lock the editing applications and then fetch the whole files for contacts, todos and events and later uploads them again.

As we plan to switch to a different format in the future we need to think how we can a) try to be compatible and b) to make syncing more easy in the future.

Some goals that should be achieved are:
 * Introduce Created and Modified TimeStamp time_t in UTC
 * On the fly generations of complete listings ( Sync Plugins )
 * QCOP notification of live changes. ( Edit in Opie... notify over QCOP )
 * Allow multiple Computers to "Fast-Sync"
 * Give XML journalfiles for changes since time_t in UTC

First Step:
 * Change transferserver to allow filtering of get/put requests for filenames and allow sending custom data. This should be used to generate XML files on the fly. Also we could add a quirk mode there to write/read XML files for a specefic partner. We can also do check ups on upload not to lose not supported attributes.

Second Step:
 * Introduce Created and Modified timestamp to OPimRecord and to the OAccessTemplate so it is set all the time
 * Start with the Notification calls for QCOP on QPE/Pim for add, edit, deleted and map them back to a signal in the concrete API for Contacts, Events and Todos

Third Step:
 * Route notification to the outside introduce a QCOP/PIM channel for export
 * Listen on that channel for queries like contacts/todos/events but also have a generic one for currently not known formats
 * Either put the XML file directly to the stream or think of a better solution ( ftp? with filename from QCOP?)
 * Use journal file like method for uploading and downloading sync_state=[added,modified,removed] where removed will be missing on downloading
 * If the addition of a record was later then the last sync the record will be marked added even if it was modified later
 * Send a list of currently used UIDs

The pros for that journal file solution are:
 * Multiple Sync Partners
 * Only two timestamps

The cons for that journal file solution are:
 * Currently problems with reassigning of UIDs
 * We need the host to assign good UIDs ( not used yet )
 * How to get deleted

How we can eliminate the problems with?
 * The partner needs to save the list of used uids
 * So if he knows a UID and it is not in the used UIDs list anymore the partner is capable of recognizing the record was deleted.
 * What is when the UID gets reassigned? This is why we've the added timestamp in each record. If the UID was known on a sync before but now is marked as ADDED the partner is capable of recognizing that in order to be able to reassign a UID the former record had to be deleted.

In the end we will have life changes notification, and faster sync because we only need to transfer one line of XML for each added or modified record. We do not need to keep track of removed entries and because all goes through the PIM API we make sure these timestamps are set.