I am incredibly excited to announce that Oz is receiving funding from Clojurists together to help take data visualization and scientific document support for Clojure to the next level!
Before I go on, many thanks to everyone who contributed to Clojurists Together, and who voiced support for projects in the data science space. Clojure has such amazing potential as a language for data science, and it's exciting to see two other projects in this space receive funding this quarter (Congrats to Dragan Djurick's Deep Diamond and Chris Nuernberger's Libpython-clj!) Again, thanks to everyone who helped make this happen!
What I hope to accomplish
I initially submitted to Clojurists Together for Q3 2019, but didn't get the award that quarter. So I was pleasantly surprised when I received an award for Q4! In the mean time though, I had already started in on some of the key features initially proposed:
- static site generation
- live code reloading
However, I have yet to release a non-beta version of these features, and there is much to be done still in making them more robust and complete. Moreover, a number of ideas have come since then which I would like to take time on.
So without further adieu, here is my amended agenda for the next three months:
- Mint a release ASAP:
- Oz has fallen behind a bit on the latest versions of Vega-Lite/Vega (hard to keep up actually), and there are a lot of exciting new features in Vega-Lite
4.0.x, so it'll be exciting to start working with these. - It's come to my attention that the uberjar builds I've been publishing have been making dependency issues harder for folks. This has actually bitten me a time or two now, and I'm looking forward to unblocking people on this front.
- Improve Reagent API:
- There are several things the Reagent API is currently not capable of, but which are possible with the Vega View API:
- Attaching signal handlers: This makes it possible to use interactive data visualizations as app-state controls
- Streaming data: It's possible to stream data into a Vega-Lite/Vega data visualization, which can be useful for building real-time data visualizations
- There's a challenge here in exposing all of these features in a manner compatible with the Reagent model, but I'm excited to tackle it!
- Leaftlet Vega:
- Vega is great for geo-spatial data, but being able to layer on top of Leaflet opens up a huge number of possibilities. There is currently a project for doing this from vanilla JS, so I'll be looking to wrap this functionality.
- Add voyager API:
- If you haven't taken a look at Voyager, you should. It's more or less an Open Soure, Vega-based take on Tableau, with less lock-in (data driven visualizations FTW!).
- Voyager is a library, and should thus it should be possible to add a function for sending data to a Voyager instance from the REPL for exploration.
- Static plot rendering:
- Static plot rendering is important to many folks who need to programmatically render lots of content as static assets, as is often the case in academic contexts.
- There are a few ways we can potentially go about this:
- Call out to the system
vega-cli utilities for converting files statically. This is conceptually simple, if not a bit crude, but ultimately comes with the downside of having to separately install a system utility (though it's pretty easy with npm now). - Use Nashorn (JS in JVM) to run Vega from within Java! This will be great if it works, but may depend on JVM 11/12 for full ECMA6 support (which Vega now requires)
- Static site generation:
- As mentioned above, this functionality already exists in some basic form now (since , but there is there much room for improvement:
- Given static plot rendering (above), the ability to insert static images in documents will make it possible to generate fully static scientific documents, and unlock the ability to export to PDF. In dynamic contexts, this also gives us the ability to display a pre-rendered version of a visualization before the JS has finished loading to generate the interactive/live viz.
- More control over styling and increased templating flexibility.
- Table view helpers:
- It's possible to create table views now just with regular old hiccup, but this is frequently inconvenient, and takes a long time to load for large tables.
- Ideally, we'd have a
[:table {opts...} data] component, virtualized for largish data.
- Separate out the client side library:
- Folks just using the client-side code for data-viz apps shouldn't have to clutter their classpaths with everything needed to run the REPL/JVM tooling.
- Improved documentation:
- More videos, blog posts, tutorials and demo repositories would go a long way towards helping folks get up and going!
Please help me prioritize!
There's way more here than I'm sure I'll be able to accomplish before the end of the three month period during which I'll be funded. With that in mind, I'd love to hear from the community about which issues folks would find most valuable. If any of these issues sound particularly exciting to you, please follow the relevant GitHub link above, and show your support with a thumbs up. You can also check out https://github.com/metasoarous/oz/issues for anything I may have missed, or to make suggestions of your own.
Status updates
Clojurists Together requires recipients to submit biweekly reports on progress. To the extent I can, I'll be attempting to make mini-blog posts out of this content.
Conclusion
Thanks again, and please stay tuned!