LK Weekly: Preparing for scale (Patreon)

This week Adam got back from Xmas holiday and had a light week dusting off his brain and his desk, Braden pushed forwards a new billing system (with a short interruption from Bloated Boards), and Justin slacked us some elegant interface mockups for billing management.

New Billing system

As we mentioned last time, we are working on a way for people to more easily start using LegendKeeper. This is an essential step in our business to encourage the growth we need to accomplish our big ambitions.

As part of the change, we are moving to a new billing system. Braden spent the week integrating it and testing the many, many edge cases that come from something like this.

For anyone currently subscribed through Patreon, don't worry - nothing will be happening to your account status yet. We will let you know when it's time to switch over anything, and we will work hard to make it SUPER EASY for you.

Bloated Boards

Some users reported awfully long load times for Boards. Braden investigated and found a bug that caused the size of saved Board data to expand at an alarming rate. This was like a ticking time bomb for every user, so he jumped onto that and shipped a quick fix.  There's a technical deep-dive into this fix at the end of this post, for those of you interested.

As always, please make sure you click the "Update" button whenever you see it 🙂

Scaling communications

As our user base grows (thank you!) and our team remains the same size, we need to start looking for ways to stay in touch with everyone and make sure your feedback is heard. This week Adam started thinking about new approaches to collecting product feedback and bug reports.

As LegendKeeper grows we want to make sure that we have a clear way to reach everyone when we need to, and a clear way to hear from you when you need us.

Roadmap

Our entire effort is focused on the billing change right now. It will be done in the coming week or two. Until we launch that and see what happens, we can't make decisions about timing for our next features.

Team LK will be meeting up again in person after this change to talk about the goals of this year and we will share everything we come up with.

As a reminder, our most immediate priorities for new features are:

• New Editor UX
• Module Sharing
• Public projects (and integrations with a new API layer)

After the billing thing settles, we can get more specific with estimates.

That's all for now. Happy New Year!

-TEAM LK

P.S. Boards quick-fix deep dive

Every change to a file in LK is stored in the file's permanent, compressed history log. This allows LK's offline, collaborative, conflict-free magic to work. (more info on that here) These logs can get pretty chonky, especially with Boards, which have tons of oft-changing 2d positional data.

On the first iteration of this saving system, v1, when we changed a shape, we simply saved the entire modified shape again, even if the only thing that changed was, for example, its color or position. While this made for an easy saving algorithm, it caused lots of redundant data to get re-written. This would rapidly increase the size of large boards, especially when modifying lots of shapes at once. This would overuse disk space and drastically increase loading times.

In board-saving v2, on each shape change, we simply iterate over changes, and only write changes that are different than the current "canonical" state. We also simplified a few data structures. This small change resulted in much better disk-usage and loading times.

To prove that this change worked, I did a few benchmarks. The benchmark board for this change was a pretty big one with 50+ post-it notes and lots of arrows, weighing in around 100kb.

In V1, after moving a single shape 5 times, the board had grown 10 kilobytes. While this seems like a small amount, keep in mind moving a shape is just changing two numbers, an x and y coordinate. That shouldn't take up 10kb!

After moving the board's 50+ shapes around 5 times, the file exploded to 400kb, quadrupling in size in less than a couple of seconds! While our collab system is smart enough to see some redundancy, and eventually compressed its size back down to 196kb, this is still an unacceptable rate of growth.

In V2, after 5 single-shape moves, the board grew just 191 bytes. After 5 50+ shape moves, it grew by 5 kilobytes. I'd love to get that even smaller, but we'll take an 80x improvement for now!