A Lightweight UI for Medical Visualizations (GSoC 2016)
Currently, if you have an OpenGL interface, you need to use Qt/GTK or some UI library to create a window and focus out of the OpenGL window to do simple UI tasks like fill forms, click on a button, save a file, etc. Our idea is to get rid of the external interfaces and have the UI built in.
So, all the interaction happens within the 3D world interface.
Currently, no library in Python offers such a functionality, much needed during scientific visualisations.
So, we built this cross platform minimal interface on top of VTK, which provides a very simple but powerful API, and can be tweaked to create futuristic interfaces.
The Project
A complete list of all my commits into the project are listed here.
Here’s the PR [#1111] with all the changes.
Here’s what we did:
Building a Button
Using vtkTexturedActor2D we built a button with functionalities to change icons and add callbacks. This is what we got.
A button overlay
A Text Input Field
We built a textbox using vtkTextActor and added ways to edit the text in the text box. Starting from an editable actor, we ended up with a multi-line text box. We rewrote a lot of code while building this and this is where we ended up with the idea of having a generic UI super class for all the UI elements. This element also introduced ui parameters (to pass between the element and the interactor) which were later deprecated.
A text box
Line Sliding
While building the line slider, we realised the need for multiple elements within one element. This is where the idea of a common set_center method came up. This element also introduced changes in the way we added elements to the renderer. We also introduced a ui_list for each element that carries all the sub-elements in that element. We ended up with this.
The Line Slider
Circular Slider
Using techniques similar to above, we built a circular slider, using a lot of math. The circular slider underwent a lot of modifications while adding it to the panel because we wanted to maintain a constant value while moving it around.
The Circular Slider
3D Menus
Moving on, the idea was to use the existing elements in 3D. Using vtkAssembly and vtkFollower, the former taking a lot of time to understand (but thanks to efforts by my mentors, it turned out to be not so convoluted), we successfully ported several 2D elements to 3D. We couldn’t do 3D sliding, so that is something we will be appending to future work.
The Orbital Menu
A More Complex Orbital Menu
A 2D Panel
A 2D Panel is basically a collection of 2D elements. Built in such a way that they relatively stay the same, not depending on the size of the panel, the panel turned out to be more useful than we thought after we managed to set up a panel of panels.
We also used the set_centers recursively to move the panel with all its elements around. We also used it to align panels to the left or right of the screen.
A Panel
A Right-Aligned Panel
A File Menu
The time had come to build a file dialog. Using os and glob.glob we built a file menu for displaying files in the current folder and changing directories when clicked.
The File Menu
A File Dialog
To put all that we had done to test, we built a file save dialog. This used almost everything we had built till now - panels, buttons, text box, etc. Here, for inter-object communication, we introduced optional parent references for each element. In the end, it all worked out well :)
The File Menu
Future Work
Here’s what we want to do in the future:
- Build robust 3D elements
- Convert the prototype elements to futuristic looking elements
- Add unit tests - we don’t have a unit testing framework right now, we are looking for one
Acknowledgements
There are countless brilliant brains that went behind this project.
Both my mentors, Eleftherios Garyfallidis and Marc-Alexandre Côté have stood by my side, all through the summer, with regular (and at times more than that) meetings, reviews and suggestions. I could ping them at any odd time of the day and they would promptly reply to all my doubts. The Story of the Rabbit’s PhD Thesis holds true.
The people who built VTK and those who built the Python wrapper for it have done an amazing job. It’s an amazing framework, still in development. I am also indebted to the people who discuss all their doubts online and leave breadcrumbs leaving to the right resource.
And lastly, my colleagues who gave me valuable feedback like, “Try red!”.
Thank you all!