Final report - Google Tasks API support to libgdata/Evolution - there's trees behind forest

So here comes last final summary post about my this year's project. This year I took a break from working with Gstreamer related things like Jokosher or libwaveform. It started quite innocent - I really wanted to add Google Tasks support to Evolution. Yeah, I know, many now assume that NSA reads all our stuff (be serious - it isn't), but really, my task list for all things open source and free software or that I have to buy a kitchen sink really not top secret information. Still, it would be nice to add tasks support for let's say ownCloud in the future too. Another huge reason for this was that I wanted to improve my C knowledge a lot, and for this reason alone these were four months good spent. Also thanks to my mentor Philip I learned one thing or two about writing much cleaner code, thanks to his regular pokes at reviewing commits I got some of these rules in my blood. It's worth to remember that every language and even every project in same language could have different writing styles.

First look at the libgdata gave false sense of assurance, but in fact I stumbled on a lot of issues I have never thought of before. First we had to choose how to implement general JSON support in libgdata. There's lot of places which would require a lot of code refactoring if we would do it full intrusion style. Instead, we choose to "sneak in" support, sometimes using clever hacks in places, as functionality of code was the same and adding something would mean duplicate a lot of stuff. When my last patches will be comitted to master I'll probably write in-depth summary about some of these hacks, in case someone will want to join in improving JSON support in future. I must admit that when first confusion weared off, libgdata code seems very nicely written and it was quite easy to test my code when I got my test right. One of more annoying problems I had was with Google authorization, as we looked for ways how to get authorizer created, as it kept coming back with errors. Google Tasks and other newer APIs quite insist on using OAuth 2.0. In the end I used GDataGoaAuthorizer in test code (which involves manually place client id in code), but it's not a real solution and I will try to get it working with OAuth 1.0 so we can get Google Tasks support tested automatically.

As Google moves to next generation of protocols which uses JSON, I hope my work on this will allow to add support for them in libgdata more easily. I have my branch published at GitHub here and you can follow GNOME Bugzilla bug #657539 who has all my patches attached.

If honest, I hadn't researched Evolution part of my project to full extent before I started hack on eds and "friends", and in result I was a little bit afraid of complexity when discovering that adding GT support will be more challenging than I thought. It took lot of time for me to fully understand how things works in Evolution and EDS, but slowly I pulled picture together. As I mentioned in my previous blog post, all Google ESource objects resides in modules/google-backend. I added tasklist to EGoogleBackend collection object, and defined new backend 'gtasks', in which implementation complexity hides. Evolution relies on this backend implementing several virtual functions. Implementing them properly would end with Evolution automatically recognizing ESource, and reading objects from it to show in task list.
Current state of my Evolution and EDS fixes is not fully ready for "screenshot prime time", however it is positive I will finish implement required changes till the end of September. Currently I finally got my jhbuild working and fixed bugs for last days like crazy. You can see final (for GSoC anyway) state of my evolution-data-server branch here.

Report #4 - Google Tasks API support to libgdata/Evolution - hacking Evolution Data Server

As Google Summer Of Code 2013 finish line closing in I'm completing evolution-data-server part of my project, and cleaning up second part of libgdata changes. Evolution part of project turned out more challenging than I expected, as it has many "moving parts", which I have to take into account.

First of all, Evolution-data-server has central daemon, but it also has of factory registry and factories for addressbook and calendar. In Evolution all communcation between elements are done via D-Bus. To this factory registry we should use ESource to add another resource of information. All related Esources can be stored together in ECollectionBackend subclass. In my case all Google sources can be found in EGoogleBackend (which in turn is hidden in modules/google-backend directory), which I have extended to support tasklists already. However, it's not whole story. Essentially, ESource is just a shell. You add extensions to it to ensure different functionality. Also each ESource has backend, who does all heavy lifting - actual data retrieval and synchronization with remote source.

As in many systems Evolution included tasks are seen as part of calendaring system. In result all calendaring backends include support (if possible) for calendars and tasks at once (notes and journals also can be added) and can be found in calendar/backends. This time though I'm making brand new backend just for Google Tasks, as it has completely different way to access Google APIs and retrieve results. I don't have to worry that calendar may try to use this backend, because in EGoogleBackend you can identify in code that only tasklists should use 'gtasks'.

Currently I'm trying to get 'gtasks' working with OAuth2 authorization. For this I follow how Contacts 'google' backend is implemented. I would like to use GOA extention for source (which EDS has), but as it means changing whole Google source collection backend, I will have to stick with OAuth2 for now.

Additionally to get backend working for read only mode, I have to implement open, get_object, get_object_list, start_view virtual classes. Their functionality is rather simple and I have bunch of them already implemented. As starting platform I used 'caldav' backend, but this caused some issues with understanding fully how everything works (thanks my mentor Milan for clearing up most of things). A little bit more complicated is synchronization between local storage and actual server, but it's still doable.

During weekend I will write one or two final entries for GSoC with details how everything works together so far. Then I hope to hand in my branches to mentors for review and commit. Then all work will continue on GNOME git server after GSoC.

Report #3 - Google Tasks API support to libgdata/Evolution - compiling, testing and believing

Again after longer pause there's some status update about Google Tasks support in libgdata/Evolution. We got all August past us, and there have been some good news and surprises along the way, so it's worth to blog about it.

First, in a crazy travel stunt which included traveling in the car non-stop for 18 hours to and back with only one driver, my friend Rudolfs Mazurs (and our significant ones) in torching 37C hotness in a car without conditioning, we went to first GUADEC in my life in Brno. As I had to miss first two days due of theatre performance I had to do on Friday, we arrived at Saturday's evening. After regrouping at dormitories and arriving late at conference place Faculty of Information Technology at the Brno University of Technology, we met Gstreamer's crowd - Transmageddon hacker and unofficial leader of sorts Christian Schaller, my past year's mentor and current de facto maintainer Sebastian Dröge and other Gstreamer core guys with addition of Zeeshan Ali of Boxes and Maps fame, and went for hunting food. For driving 18 hours without sleep it was nice ending of the day, with chats going around about beautifully geeky stuff like Doctor Who.

Last day consisted of lot of interesting speeches, I missed Sunday's keynote, but got essence of it later from buzz and back at home watching it in recording. I attended such good presentations like provocative and excellent Stef Walter's "More secure with less "security"", or nicely expressed Jeff Fortin's "Pitiv and GES, towards 1.x", which included most precise video clip explaining migration from Gstreamer 0.10 to 1.0 and bug hunting involved ever (yeah, that was *that* bad :)).

In the middle of all this I also attended Evolution hackfest and met and had a chat with my mentors libgdata maintainer Philip Whitnall and Evolution superhacker Milan Crha. Unfortunately due of sudden changes in planning - and willing to avoid driving back in a deadly heat during daytime, so in result we decided to leave on Monday's evening, not Tuesday as planned - my visit was cut short. Overall GUADEC was a blast, and I really enjoyed it even for such short visit.

I started to work on libgdata testing during GUADEC, and as usual ran into several interesting things. First of all, when Google recommends you something when using their APIs and products, you better use it, because trying other kinda "supported" things will yield inconsistent results. In this case for new JSON based APIs Google "recommends" to use OAuth2. In result with one week of trial and error I settled for using GNOME Online Accounts as way to build Authorizer (using a GDataGoaAuthorizer subclass), which I need to build working Service object (in this case GDataTasksService) so I can check if things are really working as they should. That included modifying GOA code for including Google Tasks authorization scope in get_scope method of GoaGoogleProvider, and building it with separated application keys using --with-google-client-id and --with-google-client-secret configure flags (I needed them because you have to enable tasks support for your application in Google API Console, it's not done yet for GNOME application, which is managed by Debarshi 'Rishi' Ray). In result I had my own GOA daemon, which I run instead of system's installed. I also replaced several symlinks to libgoa and libgoabackend libraries as I installed goa in /opt. Now I was ready to run the tests.

Getting my own code to actual testing was kinda fun, because I learned a lot. While I'm not totally new to C and Glib, there's still lot of things for me to learn. It was most rewarding thing too, because to see it actually working after long session of debugging and compiling is very uplifting moment.

While I was testing GDataTasks code Philip started to merge my core libgdata improvements (along with his own additions) to include in special branch and after 0.14 release - in master. Currently my plan for next week is to complete libgdata Google Tasks support by finishing insert/delete/update methods to GDataTasksService for both tasks and tasklists. Before that though I will rebase my work on Github on newest version of master as it include all JSON I need already.

On Wednesday I will blog about EDS aka Evolution data server changes, and what else I will need to have actually working Google Tasks support in Evolution. My GitHub branch for libgdata can be find here and for evolution-data server here.