Oz Updates - Finishing Oz release & working on Vega-Leaflet

Jan 17, 2020

Now that I have Cljsjs releases under way for Vega, I look forward to being able mint a new release of Oz. However, I still have to wait for the releases to come out the other side. So until then, I'm going to take advantage of the fact that the Cljsjs build process is fresh in my head, and look at extending it towards some of the other projects I'd like Oz to be able to integrate with:

• Vega-Leaflet
• Voyager

First up of these is Leaflet-Vega. It turns out this depends on another project by the same author, Vega Spec Injector, so I'm starting there. Process looks more or less like copying over the bin/update-cljsjs.sh script and editing everything over to match the other library.

I was able to get a PR up for https://github.com/cljsjs/packages/pull/2042, and now mostly have to wait and see on it.

As for Voyager, I started doing some research, and it turns out the project has not been updated in a while (no release since the last alpha.24 1.5 years ago), and there are issues on GitHub suggesting its not possible to run it anymore with the latest version of React. I'm rather sad about this, as Voyager is a really exciting and powerful project, and I was hoping to use it!

Jan 18, 2020

Going about seeing if I can't set up a test project to see if Vega Spec Injector works.

There has been talk in the past of using shadow-cljs for the client compilation, and that it might potentially have some nice features for us as far as updating dependencies goes. I haven't had a full understanding of what Shadow can do so I decided to go down the path of doing this little demo project using Shadow.

Unfortunately, upon setting up a basic shadow project depending on the Vega Spec Injector Cljsjs lib, I found that

The required namespace "cljsjs.vega-spec-injector" is not available, it was required by "example/core.cljs".
The namespace was provided via :foreign-libs which is not supported.
Please refer to https://shadow-cljs.github.io/docs/UsersGuide.html#cljsjs for more information.

:-/

Jan 24, 2020

This week has been crazy :-/

I'm just now getting back to things, and have been looking into whether Shadow makes sense and what it might look like to use it. Here's what I've been able to gather:

• It looks like cljsjs projects can be used from Shadow, but it requires a little bit of work.
• If you build your project with Shadow, you should be able to use the result from a non-shadow project.
• Overall, the externs handling process is much more streamlined, both as far as being more or less automated when possible, but also as far as allowing for customizing/fixing simply when needed (typehints, externs.txt file, etc), not to mention alarms along the way to try and help you catch instances where the automated part of the tooling isn't doing the right thing.
• It seems that this will allow users to specify their own npm versions of the various Vega libraries for debugging!

I've worked with shadow on a client project before, and thought the tooling was nice. But I wasn't sure how all these details would pan out in the context of a library. So far, everything looks very positive. However, it would mean some major changes to the way that Oz is structured, and might mean several new repos (depending on how we want to handle this).

Jan 31, 2020

At long last, my PRs have been merged! So, without further adieu, I'd like to get on about cutting a new release of Oz with the latest Vega/Vega-Lite versions. And as mentioned in my initial post, I plan to do this as a regular distribution instead of an uberjar, to mitigate some dependency issues which have cropped up for users (myself included).

However, longer term, the long time it took to get the Cljsjs releases through a PR, together with what I've learned over the last couple of weeks about Shadow has me thinking that it may be a better fit for wrapping the core Vega/Vega-Lite functionality. I wouldn't be dependent on anyone else to cut a release, and wouldn't have to go through the ridiculous song and dance I've had to do with Cljsjs.

As such, its likely this may be the last release I do before switching over to Shadow-cljs. This dovetails with a general move towards having a separate oz-cljs project for folks who don't want the server/JVM side dependencies (as also mentioned in my initial post.

I do feel somewhat bad to have spent so much time updating the Cljsjs process, only to now be dropping it. But ultimately, sorting things out has given me a chance to reflect on the problems inherent in (+ JavaScript Closure ClojureScript) compilation, and has helped convince me that Shadow is probably the right choice for this project.

Feb 2, 2020

I got a deploy to Clojars set up today. However, I am finding testing it out that it's not working. Will have to look into this more tomorrow.

Feb 3, 2020

I've tracked down the issue from yesterday. It seems that in the process of tweaking my build process to avoid creating an uberjar, I thought it would be safe to remove the line :auto-clean false from my project.clj. This turns out not to be the case, and was found to ultimately be what was causing the JS to not load.

This only seemed to get me part of the way till I realized that one of the API function calls in the vega-lite component was using js/vl instead of js/vegaLite. This seems to have been one of the "changes" in Vega-Lite 4, which at the very least makes this more consistent with the other Vega projects.

Fortunately, the code that directly called out to Vega-Lite, as apposed to operating through the Vega-Embed API, was only being used for debugging purposes to extract the compiled Vega. This isn't strictly necessary now that there's a button in the default menu for displaying the compiled Vega. There are some future uses for the vegaLite API which I have in mind, so I may have to revisit this decision. But for now, the easiest thing to do in this case was to simply remove the call.

Feb 7, 2020

Eek! I'm only just now getting around to taking the project out of SNAPSHOT and minting an actual release! At long last though, you should now be able to require [metasoarous/oz "1.6.0-alpha6"]. Again, this release comes with all of the most up to date Vega libraries, and is no longer uberjar'd, which will hopefully ease folks development/dependency pains with the project.

Conclusion

I hope that folks try out "1.6.0-alpha6"! Please especially let me know if it solves any problems for you, or as always, if there is anything you think would make Oz better.

Now that I have things generally working again, I'm hoping to make some more rapid improvements to Oz in the weeks to come. Please stay tuned!