Back in 2014 I worked on a project to integrate Yammer with Office 365. For an entire year, our team avoided using e-mail and chat apps and tried to use Yammer for all communications. It was a fun experiment! The Yammer application still had the mentality of a startup company, however. Its design was dogmatically tied to customer engagement metrics. The statistical bulk of customers turned out to be relatively casual users. By contrast, at a big company, it’s normal to receive hundreds of emails each day. This firehose got instantly projected into our Yammer feeds. I struggled with tracking my reading progress accurately. I frequently needed to dig up previous conversations and refer to them, sometimes from months ago. I needed some equivalent of email filters. At the time, the Yammer UI didn’t handle any of these problems very elegantly. So I started to wonder... What if I made my own personal desktop client for Yammer?
Of course I didAt first this sounds like a ton of work. The official client apps were fairly closely coupled with the server. Most UI operations involved specialized REST calls. Changing anything seemed to require server changes. But I realized there was a shortcut: Since Yammer messages are mostly plain text, the relevant data set was fairly compact. What if I made a sync engine that would simply download all the content and store it locally in a big JSON file? Then my client would just be a simple viewer for this data structure. I prototyped this idea during our next employee “hackathon” event. It worked surprisingly well! For fun I coded it using MonoDevelop and GTK#, which made the app portable to Mac/Windows/Linux almost for free. This also enabled me to use C# async/await, avoiding the hassle of threading. It came together so easily that I continued building out the UI over the next year or so.
When my local data set eventually reached 100,000 messages, the JSON file became a bottleneck. Every update required writing 100 MB to disk! I tried BSON (a more compact binary JSON representation), but later replaced it with a SQLite database. The desktop app still loads everything into RAM, but syncing incrementally from the SQL tables, which in turn are incrementally synced from the service. This unidirectional data flow made the overall architecture very concise and solid. When a new message arrives, the filtered views and aggregated counters update in constant time, using simple logic. The biggest lesson from Yamster was how radically easier it is to work with data when loaded in RAM. Views, queries, searches, filters — these problems all reduce to for-loops and maps. It completely bypassed all the complexity of REST services and caching. Besides supporting much more flexible queries, basic interactions with Yamster were significantly faster than the official apps. We tend to think of client/server interactions as “queries:” The client asks the questions and the server gives the answers. Yamster taught me that “syncing” is often a simpler and more powerful model that querying. If your data is small, it might be a better bet. The code: https://github.com/octogonz/yamster