Winter is coming!
This blog post marks the end of an amazing summer, the summer of '16.
This summer, I got an opportunity to share knowledge, views and code with some of the most brilliant people I have come across, contributing to an amazing application that has the potential to take neuroscience to the next level.
My Google Summer of Code project with DIPY, under the umbrella of Python Software Foundation is one of the most interesting projects that I have ever worked on, which was reason enough to keep me motivated through all these months.
Currently, if you have an OpenGL interface, you need to use Qt/GTK or some UI library to create a window and focus out of the OpenGL window to do simple UI tasks like fill forms, click on a button, save a file, etc. Our idea is to get rid of the external interfaces and have the UI built in.
So, all the interaction happens within the 3D world interface.
Currently, no library in Python offers such a functionality, much needed during scientific visualisations.
So, we built this cross platform minimal interface on top of VTK, which provides a very simple but powerful API, and can be tweaked to create futuristic interfaces.
I had heard about DIPY through a friend of mine (who himself is a Google Summer of Code student for DIPY this year) and I was intrigued by what they were doing. Over my years as an undergrad, I had developed an interest in Visual Computing and this was one organisation that was doing some really cool work in the domain. So I headed over to have a look at the list of projects and sent across a mail to my (to-be) mentor, Eleftherios Garyfallidis.
He and Marc had an idea of building a futuristic GUI for DIPY using VTK. I loved the idea and I instantly began working on making it a reality. And over the course of the next 3 months, we came really close to it.
As with any project, the first step is setting things up. This took some time given that I was on OS X and my mentors were on Linux. Setting up VTK on OS X has a really convoluted procedure, but finally, working together we got it up and running in a week or two.
VTK is an amazing framework, but there isn’t much documentation and learning resources (for a complete newbie) on the internet. And therefore, I got off to a slow start. Over time, I realised that there is a world beyond documentation and StackOverflow and open mailing lists are one of the best things that happened to Open Source and Software Development in general.
And then there were these situations.
After nearly a month of struggling, we were finally able to get a click-able button working.
Once we had a working button ready, the subsequent work went on smoothly. Every time we modelled a new UI element, we found out a better way to do the whole thing, and ended up rewriting older elements. There were times when we completely rewrote elements to add a single small functionality. We let go of a lot of code and came up with simpler and more efficient ways to do things.
Since we were building a programmable UI, we tried to keep each element as generic as possible, exposing as many parameter to the user as we could. To make things simple, we also gave the best possible default values to these parameters, so that the user can simply instantiate a UI element object without compromising the amount of control he/she has on it.
When I say smooth, I don’t mean there weren’t roadblocks. There were several times when I was stuck on a problem for days, but my mentors never shied away from discussing the problems with me, and I could ping them any time to get a solution.
3 months hence, we have a good UI framework in place, built entirely in VTK-Python and it’s built in such a way that it can potentially be plugged into any VTK-based application.
The project involved knowledge about how OpenGL works, a good knowledge of Python, ability to read through documetation and mailing list archives and 3D coordinate geometry (all that effort 5 years ago finally paid off!).
vtkTexturedActor2D we built a button with functionalities to change icons and add callbacks. This is what we got.
A button overlay
We built a textbox using
vtkTextActor and added ways to edit the text in the text box. Starting from an editable actor, we ended up with a multi-line text box. We rewrote a lot of code while building this and this is where we ended up with the idea of having a generic UI super class for all the UI elements. This element also introduced ui parameters (to pass between the element and the interactor) which were later deprecated.
A text box
While building the line slider, we realised the need for multiple elements within one element. This is where the idea of a common
set_center method came up. This element also introduced changes in the way we added elements to the renderer. We also introduced a
ui_list for each element that carries all the sub-elements in that element. We ended up with this.
The Line Slider
Using techniques similar to above, we built a circular slider, using a lot of math. The circular slider underwent a lot of modifications while adding it to the panel because we wanted to maintain a constant value while moving it around.
The Circular Slider
Moving on, the idea was to use the existing elements in 3D. Using
vtkFollower, the former taking a lot of time to understand (but thanks to efforts by my mentors, it turned out to be not so convoluted), we successfully ported several 2D elements to 3D. We couldn’t do 3D sliding, so that is something we will be appending to future work.
The Orbital Menu
A More Complex Orbital Menu
A 2D Panel is basically a collection of 2D elements. Built in such a way that they relatively stay the same, not depending on the size of the panel, the panel turned out to be more useful than we thought after we managed to set up a panel of panels.
We also used the
set_centers recursively to move the panel with all its elements around. We also used it to align panels to the left or right of the screen.
A Right-Aligned Panel
The time had come to build a file dialog. Using
glob.glob we built a file menu for displaying files in the current folder and changing directories when clicked.
The File Menu
To put all that we had done to test, we built a file save dialog. This used almost everything we had built till now - panels, buttons, text box, etc. Here, for inter-object communication, we introduced optional parent references for each element. In the end, it all worked out well :)
The File Menu
Here’s what we want to do in the future:
While the final certificate would bear my name, there are countless brilliant brains that went behind this project.
Both my mentors, Eleftherios Garyfallidis and Marc-Alexandre Côté have stood by my side, all through the summer, with regular (and at times more than that) meetings, reviews and suggestions. I could ping them at any odd time of the day and they would promptly reply to all my doubts. The Story of the Rabbit’s PhD Thesis holds true :D
The people who built VTK and those who built the Python wrapper for it have done an amazing job. It’s an amazing framework, still in development. I am also indebted to the people who discuss all their doubts online and leave breadcrumbs leaving to the right resource.
And lastly, my colleagues who gave me valuable feedback like, “Try red!”.
Thank you all!