Wednesday, December 30, 2009

FileShare on Sugarlab's Activity Page

Today I uploaded the activity to the sugarlabs activity page and sent out another announcement to the sugar devel mailing list.

I am looking for testing, feedback, bugs, and any other comments you may have.

It can be found here: http://activities.sugarlabs.org/en-US/sugar/addon/4266

Saturday, December 26, 2009

FileShare Dev Release 5

Released a bug fix and added the ability for multiple selected items.  This update will allow easier use of the system as you can now select multiple rows for downloading, copying to the journal, or removing from the list.

Changes:
* Added ability for multiple selections.
* Fixed Bug: Fixed bug on client side when server adds file after join.

New Build: FileShare-5.xo

Friday, December 18, 2009

FileShare Dev Release 4

Found a few major bugs today which needed to be fixed
as well as added a new feature.

* Fixed bug with removing files added in last release
* Activity now asks if files should be saved in the
   activity entry inside the journal
  * This is because there is no way to prevent a
     save, so if the user doesn't want to keep a
     copy of all the files transfered with the activity
     they don't have to.
     Otherwise it would just be a waste of space.
* Fixed bug with formated debug strings due to
   new id system

New Build: FileShare-4.xo

Tuesday, December 15, 2009

FileShare Dev Release 3

Today I got the FileShare activity able to keep and resume.  So now when the user closes the activity, it saves its files to the journal.  This can be useful for teachers to build a file list and then share them at a later time.  One advantage to this system is it stores the file stat at the time it was added and not a link.  This means if the file is modified or deleted it does not impact the file in the activity at the time it was saved.

New build: FileShare-3.xo

Here is a change log:
* Moved tmp to instance directory
  * Tmp is kept in ram where instance is kept in flash
    memory and deleted on close.

* Keep/Resume Activity
  * On resume, only shows what files they got if they
    closed as a client
  * Keeps a COPY of each file being shared.  This allows
    it to have the files even if they were removed from
    journal.
  * If was server and now client, doesn't allow download
    of files they added when they started.

* Install files back to journal
  * Useful if it is a resumed activity.
  * This allows the activity to also be shaired and they
    will have copy of all files which they can install to
    the journal if they wish.

INTRODUCES BUG:
  client downloads file, disconnects, server removes file,
  re-adds file (after changing it), client re-connects (resumes),
  the client will not know that the file changed and not allow
  them to download the new one.
  Work around: close the activity and delete it's entry in the journal
  when re-connecting, it will allow all downloads as expected.




Monday, December 14, 2009

FileShare Dev Release 2

Today I worked on the FileShare activity and released another build.

2009-12-14: FileShare-2.xo
* Added Localization Support
* Server notifies client when new file is added
* Server notifies client when file is removed
* Client ignores remove request when transfer
 is in progress or completed
* Server now deletes file bundle for removed files
* Files are now refered to by a sha1 hash instead of int count
* Server will attempt to prevent duplicate shared entries
* By activity id in datastore metadata
* If no activity id in metadata, sha1 file given by get_file_path
* If file path is empty, no check can be done, create random sha1
 no duplicate check can be done at this time

Friday, December 11, 2009

File Share Activity Initial Dev Release

A lot has happened today.  After hours of debugging and a few chats on the sugarlab's irc channel, I have finally gotten a version of the code that seems to be stable enough for people to test and preview.  Because of this a few things have happened:

First I created a wiki page (http://wiki.sugarlabs.org/go/FileShare) for this specific activity instead of just using the one for my server code (which will need to be rewritten).

I also created a group page for the activity on Google Groups (http://groups.google.com/group/fileshareactivity). This group page will host discussions, bug reports, as well as XO activity files.  This group also holds some screenshots.

Afterward, I uploaded an xo build to the group page which can be found here: http://fileshareactivity.googlegroups.com/web/FileShare-2009-12-11.xo

And to end the day, I submitted an announcement to the sugar-devel mailing list. An archive of this email can be found here: http://lists.sugarlabs.org/archive/sugar-devel/2009-December/021377.html as well as on the activities Google Group page.

Thursday, December 10, 2009

Dec 10: File Share Activity Update

The file share activity seems to be moving along now and it is able to transfer more types of files then last update.

Bugs Fixed
Server crashes when adding files that don't exists.  This was fixed by adding a check to the journalbundleentry file. Fixed Commit

Problem unzipping large files.  This bug was actually a problem with transferring non-installed files.  Example, xo files.  There was a bug in the journalbundleentry file where if the journal entry does not have an activity id or an id from the data store, it had an empty id.  This empty id was problematic when building the zip xo file.  This was fixed by building our own id for the file as we are building the xoj file.  Fixed Commit

Fixed the problem with sending the file list between the xo and the soas.  The problems were caused by the xo using hacked up and old libraries that gave many problems.  After a few hours trying to fix them I decided to just substitute the existing json with simplejson.  Then I switched my initial file transfer to json instead of pickle as well.

New Bugs Found
Some files have problem installing some transmitted files (for example, the log activity).  This will require some more research

Only one transfer can be conducted at a time.  After the transfer is complete, the tube is left in an unusable state.  The server will need to create more tubes when the tube is taken and then the client needs to release the tube after the transfer.

Wednesday, December 9, 2009

Dec 9: File Share Activity Update

The file share activity is starting to show some results.  In its current state, it is able to transfer some journal entries.  There are still quite a few bugs that prevents it from being ready for any stable testing.

Current Bugs
One major bug is that any journal entires that do not have a file associated with them crashes when the bundle section tries to zip files that do not exists.  This means that currently it can only transfer journal entries that has a file associated with it like the Pippy and Read activities.

It also seems to have problems with unzipping large files that were transfered which I will have to investigate further.

Another bug that is hard to figure out is that for some reason, transferring the file list between my soas in the vm and my xo seems to work fine, but if transferring the other way it is failing when trying to read the file list.  I am guessing it may have something to do with the different versions of the pickle library between them.

Some problems I had to overcome
When writing code this past week, I ran into a disturbing problem between the version of sugar on my XO and the version on my vm of SOAS.  The version of the json python library on the xo was a hacked library from the looks of it.  It was very minimalistic and was missing a lot of features, including a __version__ attribute.  It also uses a read and write method where all other versions of this library seem to use loads and dumps.

That I found very annoying as I had to then modify one of my files to check if it had the read or loads method.

Another problem I found is the documentation about using stream tubes is literally "See the Read activity's code for an example."  Otherwise just lots of debugging problems that seem to be because of incompatible libraries between versions of sugar.

Future Tasks and Improvement
I need to solve the bug about transferring file lists, zipping the file for transfer with an empty file, and unzipping large files.

I also wish to add better UI, for example, progress bars during transfers on the client and possibly a list of users downloading files on the server (on this note, it should prevent disconnecting when a user is downloading).

The file share activity should also be more chatty over the mesh.  For example, as the system is set up now, the client gets the file list once they connect.  The system should allow the user to add and remove files after clients connect.  At this moment, the client gets access to all the files they were told about when they connected.  If the server removes a file, the client still can get it.  If the server adds a file, the client is not notified about the new file.

Friday, December 4, 2009

FileShare Activity

Well I have officially started working on an activity I am calling FileShare.

At the current stage, I have the activity able to select files from the journal and share the file list with other xo's that have joined the activity.  My next task is to be able to implement the file transfer system.

During the week, I spent most of my time trying to figure out little details about the sharing system and the pygtk for the gui.  Now that I have the gui up, now I just need to get the file transfer to work.  Once I have that working, I plan on making a post on the dev mailing list to get initial feedback from the community.  From there, I will then try to interface this with the school server so that the server will be able to host files as a library.

Here is a screenshot of the file list ready to be shared.



Here is a screenshot asking the user what journal entry/file they wish to share.

Tuesday, December 1, 2009

Initial Client Plans

I have started working on the client of the file distribution system. My original first task was to get the system to work with the server, instead I have decided the first step will be to get the system to share between xo's. This is because I found a proof of concept activity called Distribute (http://dev.laptop.org/~bemasc/Distribute-1.xo) as well as.

This Distribute activity was designed as a proof of concept which means it is very limited. In its current state, when it is opened it asks for a single file which once it is supplied, the activity waits for someone to join. When someone joins the shared activity the selected file is transfered.

I am planning to write my client activity based off this Distribute activity. First I would like to allow the user to be able to build a list of files they wish to share and to allow the person receiving the files to decide what files they would like. This would allow the user to have a mini library they wish to share. The next step would be to extend the system to use the server. The reason I think it would be better to get between xo sharing to work is that it would be easier for me to adapt my system's api to the system the activity will use to transfer files.