As heavily hinted in my 4th goal for 2021, I've decided that the Always Developing podcast was not working out. So instead I am "live coding" on YouTube.

I'll let the blurb from the about page explain what it's all about...

Live coding an app with SvelteJS, Go, CockroachDB, and Juju Charms in Python. Sometimes Linux desktop app or WordPress plugin development too.

Before starting every work day, I do half an hour of noodling on side projects, making sure to keep them moving forward, albeit slowly.

Working on these side projects is how I learn new programming languages, frameworks and technologies.

Chances are I'll fail miserably each and every day, so at the very least you can take comfort in seeing a so-called senior developer fumbling through some development with little success.

Because live streaming sounds like an absolute nightmare for concentration, and the goal is to lean and keep progressing my side projects, it's recorded coding sessions. However, I am recording them "live" with OBS, and not editing them except for editing out any sensitive data (once so far), or should there be some sort of interruption (none yet ).

And true to the description on YouTube, there has already been plenty of opportunities to learn from my mistakes in the 10 episodes I've recorded so far (11 if you include the short follow-up episode 9.1).

The idea of doing half an hour of development every day was inspired by one of Andreas Kling's Car talk episodes. In Car talk: My technique for discipline he was answering a question on how he is able to be so disciplined. His answer was basically that he's not disciplined, he just has a routine that works well for him. He also mentioned that even if you could do half an hour of something every day, it would add up nicely over a year.

Given that I work approx 46 weeks in a year, that's 115 hours worth of half hours before work. That amounts to about 15 professional days for me (I do 7.5 hours per day). 3 weeks of pure coding days (which never ever happens in a real job) should produce something of worth, and at the very least I will learn a lot.

I'm doing the coding before work for a couple of reasons:

1. The first and most important is that it's easy for me to slot in half an hour as part of my pre-work routine, just after having breakfast while catching up on that there internet thing. If I tried to schedule something in the evening I would forever be skipping the session for some reason or another as evenings are much less structured for me. There's always the problem of motivation in the evenings too, YouTube isn't going to watch itself you know, someone's got to do it.

2. The second is that I have a daily 10am stand-up meeting with my team mate Erik which forces a hard stop, giving me a sense of time scarcity to concentrate my mind and reason to step away and come back with fresh eyes.

And so far it's worked out well. My very first attempt on April 1st 2021 didn't go so well though, and has been trashed. In that joke of a recording session I was trying to also code at my standing desk, which I never do, and it just didn't work at all. I struggled to use the keyboard properly, and what with trying to code and talk at the same time too, it was a fool's errand. I should have known given the date.

Since then I've been sitting at my desk, so am way more comfortable, and haven't been trying so hard to explain absolutely everything that's gong on. I haven't really properly introduced what I'm working on either, as each session needs to stand on it's own feet.

It's still very very early days and I've a lot to learn about live coding and recording etc, let alone learn all the development technologies I'm working with, but I'm enjoying it and making progress.

So head on over to YouTube and start watching me fumble my way through learning some stuff!

For a software developer, seeing this level of rapid deployment of a new release is awesome. Having users using the latest version quickly and easily helps get bug fixes out rapidly, which in turn reduces the chances of the user having problems and needing support.

Here's what the 1.5.0 released included:

• Added search of Snippets in main window.
• Added last used Snippets shown first in Search and Paste window.
• Added shortcuts for main window actions such as Add Snippet (Ctrl+n).
• Added support for system Light and Dark appearance preference.
• Improved theming of window controls to be more consistent with system.

Being able to search snippets in the main window is very useful while adding or editing snippets, especially once you've accumulated quite a few.

Over time I've found that I tend to use the same handful of snippets over and over again while working in a particular context. So this release now orders the entries shown in the Search & Paste window by most recently used first, whether there's a search or not. This has proven to be a huge productivity boost for me.

Once you've got a good library of snippets created, adding shortcuts to the main window in this release doesn't really make much difference to you as you rarely using that window. However, when you're starting out with Snippet Pixie or in a phase of editing a bunch of existing snippets, this new feature will be very helpful.

The last two entries in the release notes relate to how the main and search & paste windows are themed. If a Linux desktop supports light and dark themes, Snippet Pixie should now switch its styling according to the user's preference. Along with that came a reduction in custom styling for Snippet Pixie, it's now using standard colours for the header bar and pretty much everywhere else so that it can easily switch between light and dark, and generally fit in with whatever desktop is in use.

Snippet Pixie 1.5.0 is available from the elementary OS AppCenter, the Snap Store, and nixpkgs.

To me, goals are worthless unless they're S.M.A.R.T. There's a few variants of what SMART stands for, but the version I learnt many moons ago is as follows:

• Specific - targeting a specific area for improvement
• Measurable - progress can be measured numerically
• Achievable - has to be attainable
• Realistic - has to be achievable within the alotted time span
• Time-boxed - there must be a target end date for reaching the goal

So, given the above, I'm setting goals for 2021 that cover areas of my life where I feel I need to make significant improvements, or keep on top of and simply not screw up.

Business

My business is software development, that's what I do, and love doing it. However, the vast majority of my income comes from being a contract software developer where I effectively trade time for money.

For a very long time now I've wanted to at least partially supplement my consultancy income with passive income from software products. While I've had some success with this in the past, at present I'm failing bad, with only WP Table Pixie and Snippet Pixie available as a way of generating passive income. The annual revenue from them combined might just stretch to a nice meal out for the wife and I (finger's crossed we can do that this year).

I don't think either of the above products have much chance of making any significant revenue in the long term, but my Secret Project might make something worthwhile eventually.

Goal 1: Release 1 new software product by 01/11/2021

My little secret project should be available for anyone to sign up for and use by 1st November 2021.

It's important to get the software to Minimum Viable Product and available for use by people I don't know. If it's not available for anyone to use, I can't possibly make any money from it.

Goal 2: Secret Project earns $10 MRR by 31/12/2021

My little secret project should have however many paying customers it needs to be earning $10 USD per month by 31st December 2021.

Just having that validation is key here, I can worry about growing revenue in the future. I need to do whatever it takes to get people interested in, trying and paying for it. There's a lot to that beyond simply making the product available for sign up.

The product may not even be something you pay for on a monthly basis, but there'll be some way of working out a monthly recurring revenue figure.

Personal Growth

While I've learnt a lot in relation to programming in 2020 having picked up Go, Svelte, and some ancillary technologies, I've really not grown my social skills as I had hoped.

On the face of things you'd think growing my social skills is a bit of a hard goal to quantify, but I think I have two goals that will achieve what I want in terms of helping me communicate better and improve my interactions with the World at large.

Goal 3: Write 12 blog posts by 31/12/2021

There should be a new blog post published on this site every month, for a total of 12 posts by the end of the year. While the goal is to have 12 new posts by the end of the year, I really mean to publish something every month.

I need to greatly improve my writing skills, and just writing something that is publicly available at least once per month should help with that.

I'm not even worried about how big each post is, or how profound it may be, I just need to write something on this blog.

I'm already over achieving on this goal as this post and 2020 Year in Review are definitely going to count!

Goal 4: Release 6 episodes of Always Developing by 31/12/2021

It doesn't matter whether I decide to release audio or video, but by the end of the year there must be 6 new episodes of Always Developing available.

Health & Fitness

I've been really lazy in 2020, and my strength and suppleness have suffered as a consequence. I've noticed this simply when going for longer walks with my daughter, or doing various DIY tasks around the house this year. Getting more fit and healthy will not only directly improve my quality of life, but is also very important for those that I love.

Goal 5: Practice strength yoga 156 times by 31/12/2021

That's 3 times per week of doing either Yoga Strength Basics for Beginners or Beginner Yoga Strength Classes.

These classes should help me achieve both increased strength and suppleness over time. I'm intending to start my day off by following these classes before sitting down for breakfast, catching up on the internet, and starting my work day. The classes have built in rest days, but my intent is to limit the sessions to work days, and leave weekends for other activities.

Summary

While five goals is quite a bit to juggle at one time, there's two sets of two that go hand in hand, and I feel all three areas balance each other and hopefully help one another too.

In January 2022 I'll write a 2021 Year in Review post and include a section reviewing how I got on with these goals.

I might even do an interim post at some point in the year to see how things are going and whether any course corrections have been required. That'll at least help with goal 3 anyway!

That's not to say that 2020 has been a terrible year for me and my family, or for my business. In a lot of respects it's been a relatively positive year compared to some people's, but still, there's been ups and downs.

Business

I'm very fortunate to have been working with Delicious Brains as a senior software developer for over 6 years now. As a fully 100% remote company, the COVID-19 pandemic didn't disrupt the daily work routine too much as we're all already working from home.

I continued to work on WP Offload Media this year, a WordPress plugin that I'm very passionate about and keen to keep on moving forward.

At the beginning of the year I was the primary developer on WP Offload Media, in a team with Matt, who primarily worked on WP Offload SES. We bounced ideas off of each other and reviewed each other's code etc, but for the most part just got on with our work.

This worked ok, but WP Offload Media has become a bit of a victim of its own success, as the support load has increased significantly over time as more and more WordPress site owners rather than site developers have started using the plugin, and they tend to have less experience with cloud storage, content delivery networks, DNS, and general WordPress admin. Matt and I found ourselves doing quite a lot of support, and much less development on our respective plugins.

We weren't the only ones at Delicious Brains to be struggling with support detracting from development, WP Migrate DB Pro and SpinupWP also had the same problem as they saw increased popularity too.

Luckily Brad (owner of Delicious Brains) recognised the problem and has been effectively on full time recruitment duty for as long as I can remember now!

Since early on in the year we've had dedicated 1st line support staff trialled and come on board to help with the more administrative and technical sales style support requests for all our products. They've seriously improved the life for us software developers, and for Caillie who was helping us out as much as she could, detracting from her primary role too.

We've also had developers come on board this year to help speed up the development of all our products at Delicious Brains, along with a little shake up of the teams.

For me this meant losing Matt as he's now working with Pete on WP Migrate DB Pro, but getting a brand new team member on WP Offload Media, Erik.

Erik has gotten off to a flying start, we threw him in at the deep end and he's taken to it like a shark.

With there being no chance of a company retreat in 2020, and a general feeling of increased isolation, we've also started doing per team 10 minute stand-up video calls early in the day. For me this has been a massive improvement. Getting just a bit of face time with Erik each day helps immensely, not just for sorting out development related stuff, but for getting to know my Swedish team mate.

Another change at Delicious Brains was Iain switching to being a product manager rather than developer. While this happened relatively recently, this too has been a welcome change as Iain has already taken on some of the general product management tasks that detract from development time, but also brings in a different slant on things that I struggle to see myself.

Erik, Iain and I are working on some big things for WP Offload Media, and I'm particularly looking forward to 2021 as I've got some interesting and fun development lined up.

Side Projects

For the most part my side projects have suffered from a lack of attention this year. While I've managed to throw myself into my work at Delicious Brains, all the goings-on in the World have been quite distracting and left me less inclined to work on stuff in my spare time.

Snippet Pixie

The most active of my side projects this year was Snippet Pixie, the little text snippet expander application for Linux that has been used quite a lot while writing this post!

At various times throughout the year I've got the madness for improving Snippet Pixie and worked many hours to release almost complete rewrites or bug fix releases. Apparently seven in total.

Developing a snippet manager for Linux has turned out to be quite a tall order, it's harder than it might at first seem. However, once I switched full time to Linux in the summer, it became incredibly important that I had a reliable desktop text expander, and version 1.4.0 was born. This release of Snippet Pixie threw away the notion of universal automatic expansion and conceded that hot-key expansion was a much more reliable mechanism for the many non-native applications that are used on Linux these days.

I'm very proud of Snippet Pixie, and use it many times every day. I think in 2021 I need to work on making it better known in the Linux community, listen out for what people want from it, but otherwise just keep it true to its core reason for being and keep it ticking along.

There are some potential ways I could make a little income from Snippet Pixie in the future by offering some adjacent services that also help push it forward, but I'm not sure it's quite ready for that yet.

WordPress Plugins

I have two WordPress plugins that I maintain on the side, WP Table Pixie and WP Cron Pixie. Neither are particularly popular, or have received much attention from me this year.

WP Cron Pixie is completely free, and was originally born from a blog series I did for Delicious Brains. It just ticks along and does its thing. I use it quite a bit, have kept it up to date as the underlying technology changes, but otherwise it just exists. I have a vague notion of revamping it by rewriting the front end with Svelte, and back end with WP-REST, but that's maybe best left until I get the green light to add a part 4 to the blog series for it (which is unlikely as there's many more things we can blog about).

WP Table Pixie is freemium, there's a read-only plugin on wp.org, and you can upgrade to the premium version to add editing of postmeta, options and other metadata records. Apart from keeping it compatible with WP I've not worked on WP Table Pixie this year, mainly because it seems to be a very niche product, makes very little money, and I'm not sure it will ever have much of an audience. It's a little frustrating as it's a plugin I use virtually every day when helping WP Offload Media customers, it's great for checking what's really going on in a WordPress site's Media Library data. I seriously considered just making WP Table Pixie completely free this past year, but lack of time, a few more sales than I expected, and that little voice in the back of my head saying I should maybe give it another go kept me from doing it. I'm not sure what's in store for WP Table Pixie in 2021, it may be a make or break year.

Podcasting

In August 2018 I started the RustyElm podcast, a microcast journal of my attempt to learn the Rust and Elm programming languages.

In March 2020 I finally admitted defeat and started a new podcast called Always Developing.

And then there were tumble weeds.

I've not podcasted again since that "reboot".

It's weird because I've been continuously developing software and enjoying it ever since that first Always Developing podcast episode. The problem is I've just not felt that my rambling on about what I've been up to is particularly interesting.

I seriously considered doing some (not so) live streaming of me working on Snippet Pixie or my Secret Project, even have an OBS setup for it, but just not the confidence to actually do it.

I'm a pretty private person, and so social media and tooting my own trumpet in public does not come easy. While my colleagues at Delicious Brains hear my views on subjects more readily than they probably wish, I'm not one for spouting my opinions to people that I don't know. Yes, I'm your typical nerdy introvert, until comfortable enough.

Podcasting was going to be a means for me to combat these tendencies, to improve my social skills, get a few things off my chest and journal, kinda. So far it hasn't worked out, but maybe I'll find a format for podcasting or similar in 2021 that'll help me come out of my shell.

Secret Project

There's a little side project that I've worked on every now and then for at least 16 years now. It's been a tiny little bit of software that I use as a project to test new programming languages and technologies on. It's a great vehicle for learning, has had many incarnations in desktop, mobile and web forms with many languages, and has never ever got past prototype status, even though I've had spells of using it consistently to great success.

In 2020 I started playing with it again, this time using Go for the back end with CockroachDB as the database, and Svelte (Sapper) for the front end. I've thoroughly enjoyed working on it, including taking a bit of a detour into writing Juju Charms in Python for deployment.

For Go and Svelte I've done my usual deep dive into the technology, thoroughly absorbing them, reading through tutorials and getting to know them well before properly launching into developing with them. It's been a blast, they're both fantastic platforms that just make sense to me.

When I decided to write a custom Juju Charm for the project (which has now ballooned to 3 charms by the way), I took a very different tact to picking up Python in order to work with the Operator framework. For this I tried what I call "Stackoverflow" mode, no deep diving, just trying to get somewhere by searching DuckDuckGo or Stackoverflow for answers and perusing existing code on GitHub.

I'm not sure I would ever recommend this style of learning, but it's been relatively effective for the task at hand given that the Juju Charms I've been working on aren't the primary focus of development. As I write this it's been the part of the project I've worked on the most recently, partly because it's been just such fun. Juju is a wonderful system, and Python is a very easy language to accomplish tasks with.

This little side project has had enough iterations now that I think I should probably just make a commitment to it and take it seriously this coming year. It's nothing Earth shattering, but it may be helpful to some people, enough that they're willing to pay a few quid for it. So I think 2021 might be the year that it finally gets properly worked on and released.

Personal

If you've got this far you may be under the impression that 2020 has been a bit of a quiet year for me in terms of business and side projects. Thankfully it's also been very quite in my personal and family life too.

Although the pandemic has seriously dampened our plans for celebrating some big family birthdays and anniversaries this year, my family and friends have all managed to stay safe and weather the storm.

There's been a lot of "in 2021 we'll ...", and hopefully that'll all come to pass, but really I just hope we all continue to stay safe and healthy throughout the year, and the World in general gets back on its feet.

The one thing I've not really taken care of this year is my fitness. Although I've kept a good healthy weight, somehow, I'm pretty sure I've lost a good bit of strength and suppleness through a lack of exercise this year. That is probably the one major thing I intend to address in 2021 to ensure I'm fitter and healthier for both myself and my family. But that kind of talk is probably better suited to a "goals" post.

2021 Goals

Seeing as I've broken the habit of not writing year in review posts, I guess I might as well also write a looking forward post too.

Stay tuned for a much much shorter 2021 Goals post, where I'll briefly go over what I hope to achieve in 2021, so that this time next year you and I can laugh at them in a review post. Assuming my goal of writing a 2021 year in review post should come to pass!

I tried monitoring keystrokes at a higher level for tell-tale snippet triggers, then faking keystrokes to backtrack select characters and check the select clipboard for an abbreviation match. This turned out to be a rather resource intensive and flaky method of working out whether a snippet should then be stuffed into the clipboard and pasted over the selection.

I iterated on that method a lot, came up with some routines that made it more and more reliable and performant, but still, some times you had to wait for the checking to do its thing and could often see the selection walking when the error correction and retries kicked in to combat the temperamental async Linux clipboard. If you created any abbreviations that ended in commonly used letters or punctuation characters then you could often find yourself frustratedly waiting for Snippet Pixie to fail so you could carry on typing.

I created a snippet on purpose that I knew would piss me off, it ended in a "$". I did this to drive my desire to improve the algorithm whenever I was writing code for my day job in PhpStorm, you use $ a lot in PHP. It worked to some extent, it caused me enough headaches that I often ended up right clicking Snippet Pixie's icon and hitting "Stop Snippet Pixie" so I could get on with my day. This would impact other areas of my day as I would seriously miss not having a text expander while answering support requests etc. Just silly things, like not wanting to type "WP Offload Media" over and over again, "ome`" is way shorter and easier to type.

A little over a fortnight ago I was using the most excellent Ideogram app by Cassidy James Blaede to pop open an emoji picker and pick for the millionth time that day, when it occurred to me that maybe I should just give in, stop fighting non-accessible apps, and also create a shortcut popup window to search for and paste snippets. So that's what I've done.

You can open Snippet Pixie's "Search and Paste" window at any time by hitting its shortcut, which by default is Ctrl+` (grave).

You can then just start typing the abbreviation or anything you remember from the abbreviation or its contents to narrow down the list of snippets. Hit tab to focus the list, and then either navigate the list with the usual up and down arrows etc and hit Enter on the one you want to use, or hit its number. You can just press Escape or the Ctrl+` shortcut again if you change your mind and don't want to paste a snippet.

When you do choose a snippet from the Search and Paste window it'll close and the expanded text will be sent to the system clipboard and then pasted into whatever app you had open before hitting the shortcut.

It works really well, and the window and general usage pattern may look really familiar to some people. That's because a lot of the code for the Search and Paste window and how it safely closes and pastes is adapted/liberated/borrowed/stolen from Clipped by David Hewitt.

Until I started this little exercise a couple of weeks ago I'd never actually used Clipped before, but had used some of its code when trying to get Snippet Pixie compatible with browsers etc previously. I was led to that code via comments in Ideogram's own code that first time, it worked well for what I needed those bits to do. When I started thinking about how to do a popup snippet search and paste window I first took a look at Ideogram, but that quickly led me to Clipped again. This time I took a much deeper look at the app's code, and had a play with it, it's fantastic! As much as I love the convenience of snippets, I'd not really used a clipboard manager before, but now that I've got Clipped I'm a convert, it's saved me having to go back and re-copy and paste stuff many times.

Anyway, back to Snippet Pixie. It wasn't all plain sailing, but I was able to adapt a good chunk of code from Clipped for the new search and paste window, sprinkle in some code for working with snippets, showing their abbreviations and bodies, and a few little tweaks here and there, and Bob's your uncle, we have a search and paste window opening from a shortcut.

While I've now been able to rip out all the janky clipboard code I was using for searching entered text for an abbreviation to auto-expand, auto-expansion is still in place for accessible apps. So if you're a fan of Quilter, LibreOffice, or any other native Linux app that is accessible, then you're all good, type your abbreviation and Snippet Pixie will expand it in-place for you. You don't have to use the shortcut.

However, if you decide that you prefer the shortcut, or maybe still have issues with auto-expansion in accessible apps , then Snippet Pixie now has a new "Auto expand snippets" setting you can toggle on and off in its revamped preferences menu, which has also gained a "Shortcut" submenu.

Oh, yeah, the shortcut, that too was initially inspired by some of Ideogram's code, but I also found some interesting stuff in Clipped regarding being able to change the shortcut. In Snippet Pixie's Shortcut preferences submenu you can change the shortcut from Ctrl+` to whatever suits you best. Just click on the existing shortcut and you'll be asked to type a new one.

In the above screenshot you can also see a couple of settings related to how the shortcut works.

You can turn on "Search selected text" if you like. When on, if you have any text selected in an app before you hit the Snippet Pixie search and paste shortcut, then that will be used as the initial search for snippets. It's off by default for a reason though. While at first I thought it was an awesome idea, in reality I often found that the search text was being populated with text I didn't expect, such as previously copied text. This is an unfortunate side effect of how the Linux clipboard works, it's quite easy for the "select" (a.k.a Primary) clipboard to end up with the contents of the copy clipboard. That kind of makes sense when you think about how you first select text and then copy it, but it is a bit annoying some times.

However, if you also use the on by default "Focus search box" setting, then it's not a huge deal as you can just start typing to replace what was automatically inserted into the search box. This is a diversion from how Clipped works, which naturally selects the list of previous clipboard entries as you're very likely to want something just recently clipped and therefore it's more convenient. But with Snippet Pixie I find I'm generally searching for an abbreviation anyway, so by default I've switched initial focus to the search box. However, turn off that "Focus search box" setting and the first entry in the list of snippets will be selected when the window opens. This does have its attractions too when "Search selected text" is enabled as you could find you've got a narrow list of snippets to hand and can quickly select the one you want without having to hit the tab key to focus the list first.

So that's it really, Snippet Pixie 1.4.0 is now available on the elementary OS AppCenter, and will be available on the Snap Store soon¹.

Here's the changelog for Snippet Pixie 1.4.0:

• Added Search and Paste window, opened with shortcut Ctrl+` by default.
• Added "Auto expand snippets" checkbox to preferences menu for enabling/disabling snippet expansion while typing in accessible apps.
• Added ability to change Search and Paste shortcut in preferences menu.
• Added preference for whether text selected before using shortcut is used for initial search.
• Added option to focus search box when using the search and paste shortcut.
• Fixed support for Wayland.
• Removed auto expanding of snippets in non-accessible applications such as browsers and electron apps (use shortcut for search and paste window instead).

Enjoy!

¹ I'm holding off on the Snap just now as there's a couple of improvements in 1.4.1 that are important to Snaps, once through review and test I'll promote it to the Snap's stable channel.

WP Cron Pixie 1.4.1 changelog:

• Fixed wrong data refreshing into non-primary subsite of directory multisite
• Minor updates to framework and build tools.
• Tested with WP 5.4

Liked the format of 2 main talks + 3 lightning talks after a break.

Great talks, and very well organised event, a lot of work must have gone into it, congrats to all!

Umm, what? How could you bump Snippet Pixie up a version, and claim to have spent "many many hours" on development, but not have any new features?

And isn't this post titled "Snippet Pixie 1.3.1 Released"?

Well, take a look at the gif attached to this post, that's a snippet expanding in Standard Notes, an Electron based application.

Ok, to some people that might not mean anything, but believe me, it's a major relief to have Snippet Pixie able to work with Electron based apps, and of course, Chrome and Chromium too.

Before v1.3, Snippet Pixie relied on apps using the ATK library, or at the very least, doing what's needed to work with the accessibility DBus interfaces that AT-SPI interacts with. Further still, the user had to be typing in an EditableText and Text interface compliant control. There are many Linux applications that are ATK/AT-SPI compliant, but unfortunately a few very popular ones are not.

The biggest trouble maker for Snippet Pixie was Chrome, turns out it's quite a popular browser.

And many companies build cross platform apps with Electron, a technology that basically uses Chrome under the hood, so they weren't working well with Snippet Pixie.

I tried all kinds of things to try and get Chrome and Electron apps to play nice, and there was the odd occasion when a control here and there would actually work with the accessibility framework, but in the end I had to throw out the accessibility stuff and go "old skool".

Well, I say "throw out the accessibility stuff", but that's not quite true, and is the reason for v1.3.1, but we'll come to that in a minute.

Snippet Pixie still uses AT-SPI for monitoring keystrokes, but it's using a method that is a little more greedy in its scope, monitoring "all windows" for the current app rather than just the currently focused text control. With this method, Snippet Pixie can see when a key is pressed that matches the last character of some snippet's abbreviation, and then go look to see whether previously entered text matches an abbreviation.

This may sound less performant than monitoring the currently focused text control, and technically it likely is, but, it turns out that with this method I was able to eliminate a huge performance problem that earlier versions of Snippet Pixie suffered.

Snippet Pixie used to have to notice when the current app changed, then go look through all the widgets in that app to try and find the currently focused one, which hopefully was an editable text control. When that app happens to be a spreadsheet with thousands of widgets, that search for the currently focused control (cell) could take a noticeable amount of time. Now, we're not talking minutes here, not even seconds, but if it took even anything near half a second, you'd notice it.

Because Snippet Pixie is now monitoring at a higher level by default, it does not need to go look for a widget to attach monitoring code to. That's a big boost in performance, it no longer feels like Snippet Pixie is slowing down app switching for some apps.

Hang on a minute, if you no longer know where the text is being entered, how are you checking whether an abbreviation precedes that key stroke that matches the end of one or more abbreviations?

Blimey, you do ask some very intelligent questions don't you?

So this is where a huge amount of time was spent in developing this release. When I first thought of building Snippet Pixie, one of the initial ideas I had for how to get the just entered text was to see if there was a way to walk back through the text with some sort of selection method, and then check the contents of the selection. I didn't get too far in that investigation before finding the accessibility interfaces, which seemed like a much saner approach.

Recently I found out about another open source text expander style app called Espanso. There was an issue raised on the project about the clipboard being overwritten when an expansion happened, or something like that, can't quite remember. But anyway, seeing that made me think that maybe I should take another look at my mad idea of selecting text, stuffing it into the clipboard and then checking the contents of the clipboard.

After much research and experimentation, I eventually managed to get Snippet Pixie to start a text selection by "pressing" the shift key and then "pressing and releasing" the left cursor key, then fake a copy shortcut, check the contents of the clipboard, and then keep hitting the left cursor key until an abbreviation was found, or the max length of known abbreviations was exceeded.

This kind of worked, and once I had an abbreviation match it was a cinch to then stuff the expanded text into the clipboard and fake a paste hotkey. Thanks for the initial code Clipped!

Grabbing the selected text into the clipboard wasn't ideal though. It was very flakey, using the copy shortcut for every extra character was hit and miss, and there was some weirdness with the clipboard not always updating.

I learnt a lot about the async mechanisms that the Linux desktop (GTK+ based in this case) clipboard uses, and that there's such a thing as a clipboard that most people think about, but also a second selection clipboard that auto populates when text is selected.

With that knowledge to hand, and after much experimentation with threads and strategic yielding and microscopic usleeps (I hate handling threads, but who doesn't?), I finally got a nice mechanism that recognises the abbreviation via selection, and then expands via the clipboard.

Over time I fine tuned the mechanism, and added a bunch of speed ups and error correcting retries to the way abbreviations are searched for, and ensured that the current clipboard is saved away and restored before and after the expansion respectively. The attached gif is intentionally slow so you can see the walk back working, even error correcting, but in general the mechanism is way quicker.

It worked really well with Chrome, Chromium and even Electron apps, but when I tested LibreOffice Writer, which had worked fine with the previous version of Snippet Pixie, it did the most weird things and lost text selection all the time, making abbreviation recognition impossible.

So I brought back the core abbreviation recognition engine of the old accessibility framework dependent mechanism, and sure enough, that still worked with LibreOffice Writer. Well, it wasn't quite that simple, I also had to add back a mechanism for being notified of when an accessible control was focused, and do a bunch of work to dynamically swap between the "all windows" monitoring method and the control based one as appropriate when apps are switched.

I did not need to re-add the old widget lookup code though, and I was able to improve the accessibility based mechanism with the abbreviation lookup speedups and threading from the new text selection mechanism.

In the end I got to a place where the previously supported applications still worked with the faster accessibility based text editing, and added support for a much wider set of applications via the new text selection mechanism.

When I applied those speed ups to the old mechanism that I created while developing the new one, I did miss one problem that necessitated v1.3.1. I used a slightly different monitoring mode with the re-implemented accessibility based code, async rather than blocking, kinda needed as part of the performance improvements and thread usage. Sometimes the control where the text was being entered would report the caret position incorrectly, especially near the beginning of the control, and after initial switch into the app. Turns out I needed to yield the abbreviation checking thread for a split second to let the app get itself together.

There are unfortunately a few apps that remain incompatible with Snippet Pixie 1.3.1.

The one that hurts the most for me is Firefox, my preferred web browser, which at some point in its last few releases started to misbehave when Snippet Pixie was in use. The old accessible method that Snippet Pixie used stopped working for some reason, something must have changed in Firefox to stop keystroke monitoring. I wouldn't be surprised if this was an intentional security measure, but I bet it also stops screen readers from working now.

When forcing use of the new text selection mechanism with Firefox, it keeps losing focus while you're typing, which breaks everything, text stops being entered. To get around this problem temporarily, until I can find a proper solution, or Firefox gets fixed, Snippet Pixie now blacklists Firefox so at least Firefox is usable, even if snippets don't expand.

Similarly, Snippet Pixie has never worked with terminal emulators, their text handling mechanism is fundamentally different. To avoid any potential issues with the new text selection mechanism that Snippet Pixie uses, a bunch of popular terminal emulators are now blacklisted by Snippet Pixie and it turns off abbreviation checking while they're the active application.

This new blacklisting mechanism actually brought to the fore another performance improvement for Snippet Pixie, as turning off the abbreviation checking speeds things up in incompatible apps. And along with this change, Snippet Pixie also monitors less keystrokes in general now too anyway, as if you're using either the Ctrl or left Alt key, as far as I can tell you can't be entering a character. The right Alt Gr key is still monitored in combination with normal keys though as that's how you generally enter extended characters such as € and ¢.

The final incompatibility with Snippet Pixie 1.3.1 is Wayland compositors. This too hurts as I'm a fan of Sway, but alas the way I've had to implement active window / app checking is via libwnck, which unfortunately only supports X11.

I'm actively trying to find either an alternative robust method of tracking the current application that works in X11 and Wayland, or an additional Wayland only method that I can use when Wayland is detected.

So I guess you could maybe class the expanded compatibility and vastly improved performance as new features, but even if you don't, I'm sure you can see why I bumped the version from 1.2.x to 1.3.x.

Anyway, that was a bit of a rambling retrospective into what went into Snippet Pixie 1.3.1. Hope you enjoy this release, please submit bugs and feature requests in the GitHub repo.

If you use the command line at all, you should check it out!

So, I've finally had some time over the last week or so to get stuck into my Rust and Elm project, starting by trying to get the Rust based server up and running.

Getting a server up and running with a simple "Hello World" is a doddle with Rocket, and I had fun playing with the basics of request guards etc. to easily handle calls to /hello/ and similar play things to get a grip on how it hangs together.

However, I then made the mistake of jumping into taking a look at how to use the Juniper crate along with Rocket for creating a GraphQL server. Things very quickly came unstuck as I tried to work through a seemingly endless amount of errors that frankly made no sense to me as I have very little experience with the Rust compiler. I tried following the quick start guide but even that had problems when implemented in Rocket with slight modifications as per the integration docs, and its example on GitHub. The problems seemed to be related to dependencies, I could never get the derive macros to work properly.

Eventually I came to my senses and decided that maybe I should attempt setting up a couple of simple REST routes that return JSON first, that's built into Rocket, and in the simple case doesn't need hardly any dependencies.

It worked fine, changing the existing play routes to return JSON was easy. Tiny steps for the win!

Then it was time to connect up a database and grab some data from it. I thought it would be relatively simple, as again, I was going to use functionality that is effectively built into Rocket for accessing a database via the Diesel ORM.

I have some data I can use from an existing prototype of the application I'm building, the prototype currently uses an SQLite database and I have a couple of shell scripts and accompanying SQL scripts built that can extract the data and import it into a CockroachDB database. I want to use CockroachDB because it uses the PostgreSQL wire protocol and is mostly SQL compatible with PostgreSQL too, so is easy to use with most languages and frameworks, but can easily scale and have data GEO partitioned. I don't need that scale out capability or GEO partitioning just now, but seeing as Cockroach is a breeze to set up and frankly just cool to play with, I'm going to use it. Have I mentioned that this project is all about learning how to use technologies that I find very interesting and see a great future for?

Anyway, installing the Diesel CLI was dead easy, a simple cargo install diesel_cli and we were done as I had already made sure I had the required PostgreSQL client bits and bobs installed as per the Quick Start Guide. And the basic setup of its dotenv file along with similar configuration in Rocket.toml was a no-brainer too. A quick run of diesel setup and I had the expected new migrations directory and schema.rs file in my project.

Before getting stuck into using Diesel with Rocket I took a detour to make a few tweaks to my data migration scripts to use UUID columns instead of plain integer style serial columns. It's highly recommended when data can be written to multiple nodes in a cluster, like with CockroachDB, to use random primary keys as it prevents data writes from being clumped together in its range based data setup. However, using UUIDs turned out to cause more dependency headaches further down the line that required some more quick learning, but we'll come to that in a second.

Diesel's migrations feature allowed me to convert the schema definition bits from my data migrations scripts into proper migration steps. The first step to set up a basic schema with extra "old ID" fields in tables, the next step being used for data import so I can re-create it easily from the SQLite database, and a follow on migration that then fixed up the data to convert the pre-existing integer based primary/foreign key pairs to use the new UUID setup. That last step also dropped the "old ID" fields on completion.

Then came the task of creating structs in the code that could be hooked up to the schema with derive(Queryable) macros etc. These macros allow the Diesel ORM to build queries and with the right use statements the fields from the structs can easily be used in code to form predicates. This turned out to be another source of Dependency Hell, largely due to the UUID and TIMESTAMP columns in the schema requiring new crates that exposed the underlying problem I had before.

There were a lot of errors related to traits not being satisfied one way or another, and infuriatingly it would often show trait signatures that exactly matched what I saw in the docs for the UUID or chrono crates that I was using. Every time I searched Stackoverflow or other sources for anything related, they always said that dependencies hadn't been properly added to Cargo.toml or as use statements, and then showed me exactly what I already had. Eventually I pieced together enough information to realise that if the same crate is imported as a dependency by a couple of crates, and/or explicitly by your code, if they happen to import slightly different versions that seemingly have exactly the same function signatures etc, just by being even a debug level different, Rust will consider the traits completely different and incompatible.

I used the excellent cargo-tree plugin to inspect the dependency tree and saw these tiny differences in crate dependencies. At first I was at a bit of a loss as to what to do, then on a whim I decided to relax the version numbers I'd used in the dependencies sections of the Cargo.toml file. So instead of setting Rocket to be version 0.4.2, I instead used version 0.4, and so on for all the other dependencies. And boom, just like that, cargo test compiled and ran! I had previously set tight versions to try and match what I'd seen in the dependency tree, slightly coaxing versions I had control over to match the versions I saw as automatic imports. That was around the wrong way, a better strategy was to relax the versions a little so that Cargo had wiggle room and could recognise satisfied dependencies based on slightly looser version requirements. Not quite sure where my head was on that one, it's obvious in retrospect, I blame late night programming!

With that I was able to get a couple of routes set up to return information in JSON format from the database based on UUID or other field types I played with. I even set things up so that custom JSON errors were returned when supplied query params weren't as expected and so on.

I've seen it mentioned somewhere before that one of the hardest hurdles to get over when starting with languages like Rust is simply getting past the endless stream of compile time errors you encounter when first trying to find your feet. I very much felt that pain over the last few days, and I expect it's one of the reasons Elm tries so hard to have nice helpful compiler errors that try to suggest fixes. To be fair, in a lot of cases there were helpful hints from the Rust complier too, there were times where I knocked out a big bunch of errors by following the tips it showed.

So I now had the basics of a REST-API server set up, albeit querying a single database table, but alas my GitLab CI pipeline was failing due to the new database stuff. How I got that fixed up and cut down from a run time of 26 minutes to under 3 minutes is a story for another day as I've run out of time.

As always, you can find me on...

Micro.blog: @ianmjones

Twitter: @ianmjones

... goes to register domain ...

Me: Dammit, domain's already taken, and not being used!

... later updating some DNS settings and spots ... that sweet domain that I've had for years! ...

Me:

Not something like "runtime configuration" like I've always thought.

sudo snap install snippetpixie --classic

I even tested it with LibreOffice Writer and gedit on KDE Neon, but remember folks, Snippet Pixie is a GTK app, so your mileage will vary greatly.

Now to create 300k+ posts/media for @wpoffloadmedia testing!

There's a couple of wrinkles due to naming.

The snap is called snippetpixie and uses that in the snapcraft.yaml apps: entry, but the binary and canonical name for the app (as used in meson build etc.) is com.github.bytepixie.snippetpixie, and that's the app's "ID" too. It's a GTK app built with Vala, initially for elementary OS.

This means the .desktop file is usually called com.github.bytepixie.snippetpixie.desktop. So in the app it uses this as the desktop file name when trying to copy and fix it up to create an autostart version, but the DesktopAppInfo class failed to find the file by that name.

Turns out snapd/snapcraft takes that com.github.bytepixie.snippetpixie.desktop file and renames it to snippetpixie_snippetpixie.desktop at some point (I think install time by snapd), making some changes to the Exec line along the way.

This means the DesktopAppInfo class doesn't recognise com.github.bytepixie.snippetpixie.desktop as valid.

In the end I had to add specific code for running as a snap to change the desktop file name to look for, and also be careful about the way the Exec line is changed for the new autostart version to carefully insert the required --start param in the altered command line instead of blindly changing it to com.github.bytepixie.snippetpixie --start.

You can see some of the altered (not very clever, but it works) code in snippetpixie's update_autostart function.

Hope this helps someone else having troubles getting autostart to work for a snap.

One More Thing...

I guess I'm also officially announcing that there's candidate availability of a snap for Snippet Pixie!

It can be installed from the candidate channel with…

sudo snap install snippetpixie --classic --candidate

Any feedback on the snap is warmly welcomed via either the Snapcraft forum's "Call for testing" post, or Snippet Pixie's GitHub Issues.

Get It!!!

I had a draft podcast way back in mid September 2018, but never finished it as I was just too damn busy.

Things have been kinda crazy, starting with working on multiple huge releases of WP Offload Media, and helping out with WP Offload SES. Giving my best 8 hours or more of mental energy a day to my work for Delicious Brains doesn't leave much in the tank for late night hacking. I very much enjoy working on WP Offload Media, it's an awesome plugin, and I enjoy having primary responsibility for it's development, so I naturally put a lot of my energy into it and am always thinking about it.

Having said that, I did get an itch for something I needed to build that also took my eye off of RustyElm and learning Rust and Elm, that being: Snippet Pixie.

Snippet Pixie is "Your little expandable text snippet helper", for Linux. I couldn't find a good Linux application alternative to the likes of Text Expander or Dash (which I currently use on macOS). So, in the end, I kinda just had to build it, otherwise known as scratching my itch. My primary desktop Linux distribution is elementary OS, so that's what I built it for, in Vala.

Turns out that may have helped me get my shrivelled little brain around the Rust concept of Ownership, as Vala also doesn't let you throw objects around without taking into consideration ownership. It caught me out a few times when the complier complained about not being able to assign an unowned reference etc, and reminded me a lot of how Rust handles ownership. Even though things aren't exactly the same, I'm going to chalk that one up as some progress in my learning of Rust, alright?!

Anyhow, Snippet Pixie recently had a momentous release that included date/time, clipboard, embedded snippets and cursor placeholders, adding a whole new level of functionality. And even though I'm in the middle of trying to get a snap sorted so it'll be usable on many more Linux distributions, and still have plans for more improvements to Snippet Pixie, I feel like I no longer have a huge pressure to keep adding functionality and can relax a little bit on it.

As such, I'm starting to look at getting started with my Rust and Elm based project for my evening recreational programming. Stay tuned.

Micro.blog: @ianmjones

Twitter: @ianmjones

This release took an immense amount of work, with lots of thought taken in the design of the placeholders feature, hopefully you find these new super powers useful!

• Date/Time: Insert the current or calculated date/time with configurable format.
• Clipboard: Insert the text contents of the clipboard.
• Snippets: Insert snippets in your snippets!
• Cursor: Set where the cursor should end up after the snippet has expanded.

Read on for a summary of the placeholders feature and how to use the different types of placeholder.

Placeholders

All placeholders are delimited (wrapped) by $$, with the placeholder name starting with an @ symbol.

For example, today's date can be inserted with $$@date$$.

Some placeholders allow for extra arguments when : follows their name and that is followed by the argument. For example a format for a date, or the abbreviation for a snippet. Check the following descriptions for each placeholder for more details.

To use $$ in your snippet body, escape the second $ with a backslash like so: $\$.

@date

Quick Examples:

• Today's date with system format: $$@date$$
• Today's date with custom format: $$@date:%Y-%m-%d %H:%M:%S$$
• Tomorrow's date with system format: $$@date@+1D$$
• Date 2 weeks from today with custom format: $$@date@+2W:%x$$
• Time 3 hours from now: $$@time@+3H$$

@time is an alias for @date, with one important difference, the default output if no format specified is the default time format (%X) rather than default date format (%x).

The optional format specified after : can take a format string as detailed in the GLib.DateTime.format function's docs.

The optional date calculation starts with an @ after the placeholder name, followed by a signed integer and unit. The unit types are as follows:

• Y: Years
• M: Months
• W: Weeks
• D: Days
• h: Hours
• m: Minutes
• s: Seconds

You can apply more than one date calculation, for example +2h+30m adds 2 hours and 30 minutes to the current time.

You can use both positive (+) and negative calculations, for example -3D takes 3 days from the current date.

@clipboard

When $$@clipboard$$ is part of a snippet's body, when its abbreviation is expanded the current text contents of the clipboard will replace the placeholder.

@snippet

You can have up to three levels of embedded snippets with the @snippet placeholder.

The abbreviation for the snippet to be embedded is entered after :, for example $$@snippet:sigg;$$ expands the snippet with abbreviation sigg; in place of the placeholder.

@cursor

Adding $$@cursor$$ to a snippet's body will put the cursor in its place after expansion instead of at the end of the expanded text.

If $$@cursor$$ is entered more than once in a snippet's body or via snippet embedding, then the last occurrence of the cursor placeholder wins.

Get It!!!

deliciousbrains.com/s3-compatible-storage-provider-minio

• Improved performance, compatibility, and stability.
• Added French translations. Huge thanks to @NathanBnm on GitHub!

This release includes two features that go hand in hand, snippets export and import.

Snippet Pixie is a great way to build a library of time and keystroke saving short text abbreviations that automatically expand while you type, but up until this release you could only add snippets to your second machine by either manually adding them again or finding and copying the settings database.

Export

It's now possible to export your snippets to a JSON format file for the purpose of backup or transfer to another machine for import. Now when you add new snippets on one machine you can quickly add them to another machine.

Import

When using the import feature Snippet Pixie by default will not overwrite existing snippets with the same abbreviation, but you can optionally do that if you've made changes to the abbreviations on one installation and want to apply them to another.

For safety, the import feature also does not delete any existing snippets.

UI

The welcome screen in Snippet Pixie now includes an Import Snippets option to get new installations up and running quickly. If you've already got some snippets in Snippet Pixie then you can use the "Import snippets..." option from the cog menu.

The cog menu is where you'll find the "Export snippets..." option too, it's enabled if you have some snippets to export.

CLI

Want to script an automatic backup or import? Snippet Pixie has your back as it also has new command line options for export and import.

-e, --export=filename
Export snippets to file

-i, --import=filename
Import snippets from file, skips snippets where abbreviation already exists

--force
If used in conjunction with import, existing snippets with same abbreviation are updated

File Format

Some people might be wondering why I picked JSON as the file format. The primary reason is flexability and general support for the format. There are something like eleventy million utilities for handling JSON, so if you want to manipulate or generate your own file of snippets you'll be able to do it easily.

I'll document the format properly at some point, but it's very simple as this vastly trimmed down export of some of my snippets shows...

{
  "generator" : "com.github.bytepixie.snippetpixie",
  "version" : 101,
  "data" : [
    {
      "snippets" : [
        {
          "abbreviation" : "joy`",
          "body" : "😂"
        },
        {
          "abbreviation" : "ksig`",
          "body" : "Kind regards,\n\nIan"
        },
        {
          "abbreviation" : "shrug`",
          "body" : "¯\\_(ツ)_/¯"
        },
        {
          "abbreviation" : "sp`",
          "body" : "Snippet Pixie"
        },
        {
          "abbreviation" : "spu`",
          "body" : "[www.snippetpixie.com](https://www.snippetpixie.com)"
        }
      ]
    }
  ]
}

The generator and version elements are very important, Snippet Pixie will not currently import a file that doesn't have those two elements with those same values as they signify a format that it understands.

The data array contains a list of data types with data that can be imported, currently there's only snippets.

Each of the objects in the snippets array contains only an abbreviation and body, on export they are ordered alphabetically by the abbreviation and if there are duplicates (it's possible) then by a hidden id which typically means by order of addition unless entries have been renamed. Due to this ordering, if overwriting duplicates on import is allowed then last duplicate in the file will "win", but by default first entry will "win" if the abbreviation does not exist in Snippet Pixie's library already.

As you can see from the example, it's UTF-8 and supports emoji and new lines (those \n bits), and characters like \ are escaped etc.

Get It!

As always, you can get Snippet Pixie from the elementary AppCenter.

Hope you like it, if you have any feature requests or problems with Snippet Pixie, please stop by the GitHub repository and share your thoughts!

Now autostarts on login by default, and tabbing into the body entry field selects all text, making adding new items a little bit quicker.

initiativeq.com/invite/BB...

In this release we have the following changes...

• Added checkbox to control whether example cron events should be added to cron.
• Added checkbox to control whether the display should auto refresh.
• Added "Refresh" icon for manual refresh of data.
• Fixed not all strings in UI being translatable.
• Elm 0.19 frontend.

A couple of things not mentioned in the changelog are that the "Refresh" icon does a nifty little spin as data is being refreshed or events are being run manually to give a little UI feedback, and this version has been tested with WordPress 5.0 beta 2.

It's available from the WordPress plugin repository, or from its GitHub repository.

I had a lot of fun developing this version, using the Elm language for web development is so fast and efficient. Elm is a small language packed with power, the combination of a functional and static typed compiled language with a well thought out application architecture make for a great development platform.

TinySeed. The First Startup Accelerator Designed for Bootstrappers.

tinyseedfund.com

It used to be called WP Offload S3 as it worked exclusively with Amazon S3, but now that we've added support for offloading your WordPress Media Library to DigitalOcean Spaces too, seemed like a great time to change the name!

I made a lot of changes to the internals of WP Offload Media in preparation for this release, previously the plugin was scattered with S3 specific code, now it's much more adaptable to alternate storage providers. In fact it should be a lot easier to work in a different SDK than the AWS PHP SDK that we're currently using for both Amazon S3 and DigitalOcean Spaces, you know, should we decide to add something like Google Cloud Storage support via the Google Cloud Client Library for PHP.

Micro.blog: @ianmjones

Twitter: @ianmjones

Its Timeline page is a great idea.

Nice one @TwoBitHistory!

rustyelm in Apple Podcasts

https://www.ianmjones.com/alright-lets-do-this/

Micro.blog: @ianmjones

Twitter: @ianmjones

Turns out it was February 2002, via the now defunct CityDesk desktop application. In that first blog post I make mention of having been building some now long lost CMS of my own, I don't even remember that!

I'm feverishly trying to learn Rust and brush up on Elm so that I can build a project with those two programming languages.

It occurs to me that it might be worthwhile trying to record my fumbling progress in the form of a microcast.

It would be very ad-hoc, no schedule, very much a journal for myself.

It'll initially cover how I get on with learning Rust as I plan to build out a basic backend for the product first, then build out a basic frontend in Elm and iterate on both from there.

There will likely be rumination on the actual product idea too.

So I wonder, would anyone else be interested in listening, or should I just keep this journal to myself?

Decisions, decisions!

Micro.blog: @ianmjones

Twitter: @ianmjones

It's been a long time coming, I should have taken the plunge over a year ago when I first got the bug, but decided at the time to keep my WordPress site and cross post to Micro.blog. You can tell from my distinct lack of posts that I never really got into that, I mainly used the old site for announcing stuff (a bit like I'm doing now I suppose!), and only sporadically posted to Micro.blog itself, or Twitter for that matter.

So this move is part of an attempt to be more social, something that Brad Touesnard has said I suck at big time!

It's also prompted by a desire to consolidate and clean up a few of my "web properties", with an emphasis on simplifying my hosting etc. There's some stuff coming down the pipe from Delicious Brains that I intend to use for my business stuff, but this personal stuff should hopefully flourish here.

So, when I introduced Meta Pixie my friend and colleague at Delicious Brains, Ashley Rich, had just two comments...

Nice, it supports user meta and everything. How come you decided to make it a separate plugin?

Apart from obviously loving it ;-), his second comment was basically asking how come I released a brand new plugin for accessing metdata tables rather than adding the functionality to the Options Pixie plugin?

My answer boiled down to how the WordPress database's options and various *meta tables are a different structure, the name Options Pixie didn't really fit for tables like postmeta, and frankly it was nice to have more plugins for people to be able to purchase a Pro addon for and for me to cross promote.

A year and bit later, my opinion has changed in a number of ways, and I've decided to reboot work on my personal WordPress plugin development by merging Options Pixie and Meta Pixie into a single plugin called WP Table Pixie.

At first, WP Table Pixie supports the following tables: commentmeta, options, postmeta, termmeta, sitemeta (multisite) and usermeta. However, the name "WP Table Pixie" is obviously a little more generic than that initial list of tables might suggest. I'm not promising anything, and I haven't figured out a good UI yet, but I'm keen to try and expand the plugin to handle more than just those options and metadata tables, plenty of plugins have custom tables that it would be nice to be able to drill down into and manage. If that's something that appeals, let me know! However, I realised that although the options and metadata tables have obviously different structures, I could mung things a bit in the UI and swap out column labels such as Related ID and Autoload as appropriate as long as I did some jiggery pokery in the background to make things sane.

I've come to very much dislike how having two free plugins with Pro addons means a total of four plugins needing to be installed when wanting to inspect and edit both options and metadata for a customer (I use these plugins a lot when doing support for Delicious Brains).

Hence WP Table Pixie is now a single free plugin that can do the job of the free Options & Meta Pixie plugins to VIEW ALL THE THINGS, but can also be replaced by the Premium version to also EDIT ALL THE THINGS.

This means customers need only install the one plugin, there's no need to install the free plugin before installing the version of WP Table Pixie they receive after purchase. And believe me, there's a lot of benefit to developing a free and expanded premium plugin rather than an addon for a free plugin, I squashed many subtle and not so subtle bugs during the development of WP Table Pixie from a single codebase. With WP Table Pixie I'm moving away from running my own store on my business site, and instead I am using Freemius for both in-plugin upgrades and sales initiated from the (really rough at the moment) product page. There are a few reasons why I've made this move from an Easy Digital Downloads based sales site to using the Freemius service, it feels like I should probably go into those reasons in a follow-up post.

If you're an existing customer that's bought the Pro addon for Options Pixie, Meta Pixie or both (thank you), am I leaving you out to dry? Of course not! By now every Options or Meta Pixie Pro customer should have received an email with a WP Table Pixie license for a multiple sites plan (because the previous plugins started at 10 site licenses) that expires a year after their current license expires. If you had bought both of the previous Pro addons, then you're getting a two years extension to your license as big thank you!

If you have any comments or questions about WP Table Pixie, please drop me a tweet.

It's a thin wrapper around the most excellent toastr JavaScript library to enable the display of notifications. You know, those little growl like notifications to display information, success, warning and error messages.

It's under the MIT license, and you can grab it from GitHub.

Yep, just one week after releasing Meta Pixie I've managed to finish off and release Meta Pixie Pro.

To be honest, I was already working on the Pro addon for Meta Pixie when I announced Meta Pixie's release last week, as I had to wait for Meta Pixie to be approved and added to the WordPress Plugin Directory. It took a few late nights of development and much testing, but I wanted to get it out before Christmas and I'm pretty stoked at how well development went.

Meta Pixie Pro adds the following features to Meta Pixie:

• Add, edit, delete records.
• Fix broken serialized values.
• Bulk deleting and fixing of records.
• Priority email support.

List, sort, search, view, add, edit, delete and fix your WordPress site\'s commentmeta, postmeta, termmeta, sitemeta and usermeta records with style.

Like Options Pixie Pro, you can pick up a Meta Pixie Pro license for 10, 25 or 50 sites, starting at the super low price of $39. If you want a license for more than 50 sites, drop us line.

If you stick both Options Pixie Pro and Meta Pixie Pro in your cart, you'll get a 20% discount on the second item! (tip: if you're buying a larger pack for one of the plugins, stick it in your cart second for a bigger discount!)

Well, a couple of weeks ago I finally found a few spare evenings to work on a new plugin that enables viewing of not only postmeta table records, but also commentmeta, termmeta, usermeta, and on multisite installs, sitemeta too.

Meta Pixie is now available from the WordPress Plugins repository, and once installed you'll find a Meta Pixie menu option within the Settings menu.

Now you can inspect your WordPress metadata records to find out what's really going on. For example, you can easily check that a Media Library item has the correct thumbnail metadata.

You can also check the integrity of serialized values, Meta Pixie will highlight broken serialized values (i.e. where it expects a string to be 256 characters long but its actually 255).

For us on the WP Offload S3 team it's awesome to be able to view the plugin specific metadata that we need to generate, as well as settings stored in the sitemeta table of a multisite without having to resort to database admin tools. The rich view means we can easily see the structure of a serialized record without having to unpick plain text.

I'm considering creating a Meta Pixie Pro addon that enables add, edit, delete of metadata records, as well as fixing of broken serialized values. If this sounds like something you could use, please let me know by adding your vote to its product page.

After reading everything I could find on ActorDB, and still having a bit of time to kill before having to go transfer our daughter from one place to another yet again, I figured I'd have a quick play with it.

As is rapidly becoming my norm, I went to Docker Hub to find a Docker image to spin up. To my utter shock and horror (there might be a little bit of exaggeration there), there was not a single image for ActorDB to be found!

My first thought was that maybe there was a very good reason for there not being a Docker image available, but after a little look around that there interwebs I came to the conclusion that it was simply because it's a young project, and generally run on real machines in production. Nothing made me think it was a terrible idea to run ActorDB in a Docker container, so why not give it a bash? ActorDB for Docker was born!

The Dockerfile

Having recently done a bunch of work for Delicious Brains that involved using Docker, I wasn\'t too phased by the idea of spinning up a new Dockerfile, and luckily there's an ActorDB package for Debian.

I built a basic image based on Debian Jessie, and then logged into it and experimented with the steps I needed to run to get ActorDB installed and running. It took much less time than I expected as ActorDB is pretty self contained, the biggest issues I had were with making sure curl had all its dependencies in place to actually download the .deb file from within the running container (it needs some extra root certificates installed via a package).

I also rather stupidly forgot that the actordb program runs as a daemon by default, so the container kept just running and exiting cleanly. Once I realised what was going on, I simply ran the container with --entrypoint /bin/bash and then ran actordb without any arguments to get its usage info. Turns out you can run actordb forground rather than actordb start to get it to run in the current shell. Just what I needed.

I've been working with a PhantomJS image recently, and noticed that it used a script to ensure that the phantomjs binary did not run as the first process in the container (PID 1) as it may cause issues with shutting it down. So I nicked that idea.

The final Dockerfile is pretty simple, makes sure to expose the required ports and volume primary paths that ActorDB uses, and the image is available as an automated build on Docker Hub.

Docker Compose

A distributed database is no good without multiple nodes to distribute its data around, and of course I wanted to ensure the image could be used in a network of containers. I used Docker Compose to set up a very rudimentary cluster of three nodes, you can see the setup in the project root's docker-compose.yml file.

While the containers seemed to work (on second start, see issue #1 for details), could ping each other by their service names (e.g. actordb-server-1 could ping actordb-server-2), I spent ages trying to get the instances of ActorDB to actually replicate data between them. I eventually sorted it out with the following two important realisations:

1. The -name setting in each node's vm.args file must have a unique name before the @, the ip address/domain name after the @ doesn't make it unique.
2. You must update your initialisation script that is run on the leader to also register the other nodes in the cluster (you can register them manually later, but will need to copy the leader's lmdb file to them first).

Once I'd worked that out, I was super happy to have a fully working cluster of ActorDB nodes!

Quick Example

The following text is a quick run through of bringing up a cluster, initialising it, writing some data to the first node, switching to the second node and showing the data replicated from the first, creating some more data, and then showing it all on the third node, but using the MySQL client. Enjoy!

Ians-MBP:actordb-for-docker ian$ ./up.sh
Creating network actordbfordocker_default with the default driver
Creating actordbfordocker_actordb-server-1_1
Creating actordbfordocker_actordb-server-2_1
Creating actordbfordocker_actordb-server-3_1

actordb-server-1 - Use actordb_console to run the SQL script that initialises the database:

Ians-MBP:actordb-for-docker ian$ docker-compose exec actordb-server-1 actordb_console -f /etc/actordb/init.example.sql
Config updated.
Config updated.
Schema updated.

actordb-server-1 - Use actordb_console to show the schema, create and select some data:

Ians-MBP:actordb-for-docker ian$ docker-compose exec actordb-server-1 actordb_console -u myuser -pw mypass
*******************************************************************
Databases:
use config (use c)  initialize/add nodes and user account management
use schema (use s)  set schema
use actordb (use a) (default) run queries on database
*******************************************************************
Commands:
open         (windows only) open and execute .sql file
q            exit
h            print this header
commit (c)   execute transaction
rollback (r) abort transaction
print (p)    print transaction
show (s)     show schema
show status  show database status
show queries show currently running queries
show shards  show shards on node
*******************************************************************

actordb> actor type1(hello_world);
actordb (1)> insert into tab(id, txt) values(1,'Hello World');
actordb (2)> c
Error: {error,{nocreate,>}}
actordb> show
****************************************************************************************************************************************************************
sql                                                                                                                                                 type       |
----------------------------------------------------------------------------------------------------------------------------------------------------------------
$CREATE TABLE tab (id INTEGER PRIMARY KEY, txt TEXT);                                                                                               type1      |
$CREATE TABLE tab1 (id INTEGER PRIMARY KEY, txt TEXT);                                                                                              type1      |
$ALTER TABLE tab ADD i INTEGER;                                                                                                                     type1      |
$CREATE TABLE tabx (id INTEGER PRIMARY KEY CHECK (typeof(id) == 'integer'), txt TEXT CHECK (typeof(id) == 'text'));                                 type1      |
$CREATE TABLE asdf (id INTEGER PRIMARY KEY AUTOINCREMENT, txt BLOB);                                                                                type2      |
$CREATE TABLE actors (id TEXT PRIMARY KEY, hash INTEGER, val INTEGER) WITHOUT ROWID;                                                                counters   |
$CREATE TABLE actors (id TEXT PRIMARY KEY, hash INTEGER, size INTEGER)  WITHOUT ROWID;                                                              filesystem |
$CREATE TABLE users (id INTEGER PRIMARY KEY AUTOINCREMENT, fileid TEXT, uid INTEGER, FOREIGN KEY (fileid) REFERENCES actors(id) ON DELETE CASCADE); filesystem |
----------------------------------------------------------------------------------------------------------------------------------------------------------------
actordb> actor type1(hello_world) create;                 
actordb (1)> insert into tab(id, txt) values(1,'Hello World');
actordb (2)> c
Rowid: 1, Rows changed: 1
actordb> actor type1(hello_world) create;                 
actordb (1)> select * from tab;
actordb (2)> c
*********************
i    id txt         |
---------------------
null 1  Hello World |
---------------------
actordb> q
Bye!     

actordb-server-2 - Use actordb_console to see the data created on actordb-server-1, and create some more:

Ians-MBP:actordb-for-docker ian$ docker-compose exec actordb-server-2 actordb_console -u myuser -pw mypass
*******************************************************************
Databases:
use config (use c)  initialize/add nodes and user account management
use schema (use s)  set schema
use actordb (use a) (default) run queries on database
*******************************************************************
Commands:
open         (windows only) open and execute .sql file
q            exit
h            print this header
commit (c)   execute transaction
rollback (r) abort transaction
print (p)    print transaction
show (s)     show schema
show status  show database status
show queries show currently running queries
show shards  show shards on node
*******************************************************************

actordb> actor type1(hello_world) create;
actordb (1)> select * from tab;
actordb (2)> c
*********************
i    id txt         |
---------------------
null 1  Hello World |
---------------------
actordb> actor type1(hello_world) create;
actordb (1)> insert into tab(id, txt) values(2, 'ActorDB Rules!');
actordb (2)> c
Rowid: 2, Rows changed: 1
actordb> q                                                    
Bye!     

actordb-server-3 - Use the mysql client to connect to actordb-server-3's MySQL protocol port:

Ians-MBP:actordb-for-docker ian$ mysql -u myuser -p -h127.0.0.1 -P33337
Enter password: 

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 0
Server version: 5.5.0-myactor-proto 5.5.0-myactor-proto

Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> actor type1(hello_world) create; select * from tab;
Query OK, 0 rows affected (0.01 sec)

+------+----------------+------+
| id   | txt            | i    |
+------+----------------+------+
|    1 | Hello World    | NULL |
|    2 | ActorDB Rules! | NULL |
+------+----------------+------+
2 rows in set (0.00 sec)

mysql> actor type1(hello_world) create; update tab set i = id * 100;
Query OK, 0 rows affected (0.00 sec)

Query OK, 2 rows affected (0.02 sec)

mysql> actor type1(hello_world) create; select * from tab;
Query OK, 0 rows affected (0.01 sec)

+------+----------------+------+
| id   | txt            | i    |
+------+----------------+------+
|    1 | Hello World    |  100 |
|    2 | ActorDB Rules! |  200 |
+------+----------------+------+
2 rows in set (0.00 sec)

mysql> exit;
Bye

After many late nights (but no weekends), I finally got to a point where I was very happy with how Options Pixie Pro was working, and confident that it provided enough value to be worthy of release.

It's pretty awesome to have a new product out in the wild, it's been over 10 years since I released my last product on my own, through pinching time here and there in the evenings.

It feels a bit weird too when I realise that Options Pixie Pro is new to everybody. I've been using it for months during my normal work day, especially when testing WP Offload S3 as it's been perfect for inspecting the options records created for its settings and processing queues. It's very handy too when you need to add, edit or delete options records to mimic various scenarios while testing a plugin.

Favourite Feature

I think one of my favourite features of Options Pixie Pro is how you can edit a Base64 encoded serialized array, or fix a broken serialized array hidden behind Base64 encoding. Options Pixie Pro lets you see the hidden serialized string, edit or auto-fix it, and re-encodes it for you on save. I can only assume that the reason theme developers Base64 encode their settings is to preserve extended character sets regardless of MySQL's character encoding settings, but now you can edit and auto-fix those values. And if you can't be bothered to count the number of characters in the string you've just edited deep within a serialized array or object string, no problem, as long as the format is still valid Options Pixie Pro will fix those pesky character counts for you on save.

Row Actions

Of course, there are row level and bulk actions for fixing serialized values...

Bulk Actions

And the same goes for row and bulk actions for deleting records...

Multisite Support

And if you manage a multisite install of WordPress, you can inspect, add, edit, delete and fix options records across all your subsites...

Quick Demo

Below I've embedded a quick animated gif of me exercising some of the features of Options Pixie Pro, click it to get a larger format version.

Learning Experience

It's been quite some learning experience. There's a lot involved in developing any kind of software product that you intend to sell and support. Apart from the actual software development, getting a website set up to sell direct downloads is generally more involved than it might look at first. But the good thing is, once it's set up for the first product, the second should be a lot easier to add.

If you're looking to List, sort, search, view, add, edit, delete and fix your WordPress site's options records with style, then please pop over to my business site and take a look at Options Pixie Pro!

There are of course a few existing WordPress plugins for working with the wp_options database table, but none have quite fit my needs, either by not showing values in a usable way, not having good search and sort, or simply throwing a million errors when in debug mode (which you always seem to want on when the need arises for checking your options records).

Although I've been developing WordPress plugins for Delicious Brains since mid 2014, I've never actually released a plugin of my own. That doesn't seem right, especially as most of my team mates have excellent plugins of their own.

So I've scratched my itch as they say, and have developed and released my first open source WordPress plugin called Options Pixie.

Actually, as I write this, Options Pixie has been released and available from the WordPress plugins repository, GitHub and my own business site for over 7 months, as I released it at the beginning of July 2015. I guess the fact that I haven't spoken about it here, only once on twitter, is a testament to my awesome marketing skills. Maybe not.

I'm not sure why I haven't been jumping up and down and shouting about Options Pixie, as I'm really quite proud of it. It works very well for what I need it to do, and I went to great pains to make it a very high quality and robust WordPress plugin. Maybe I can just blame my natural and very British reserve?

Regardless, I've finally finished off this post (I started it just a few days after releasing Options Pixie, 7 months ago), and encourage anyone that finds themselves in the position of needing to check the contents of their WordPress site's options table to search the WordPress plugins repository and install Options Pixie. It's currently at version 1.0.1 having had a few bug fixes, and there's a new version in the wings which improves some of the behind the scenes functionality, enhances working with base64 encoded values, and is tested with the latest versions of WordPress.

Here's the highlights of what Options Pixie offers:

List, filter, sort and view options records, even serialized and base64 encoded values.

• List, sort and search options
• “Rich view” of serialized and JSON string values
• Works with base64 encoded serialized and JSON string values
• Highlights broken serialized values
• Supports Multisites

With Options Pixie you can find out what is really going on with your WordPress options. I'm also working on Options Pixie Pro, a paid addon that adds bulk actions such as delete and fix serialized, add, edit and delete functionality, and of course priority email support.

Today I had an email from Apple to say that my first iOS game, Missed Three, had entered In Review status, which I thought was pretty good as it'd only been submitted for review a week ago. Then, just 30 minutes later I received another email to tell me that Missed Three was ready for sale through the App Store.

I guess when you make what is probably the simplest game on the App Store, it doesn't take very long to review!

It really is a very simple game, keep tapping the targets as they appear, when you've missed a total of three the game is over. Of course, it starts off at a reasonably slow pace, giving you a little while to tap a target before it disappears, but as the game goes on the targets disappear in an ever shorter time. It really does get pretty hectic if you're doing well.

If you get a new high score, you get an opportunity to share it with a friend by sending them an email. I intentionally did not add sharing by Twitter, Facebook or any other social media as I don't like seeing those kinds of things pollute my timeline, so why would I make it easy to do that in my game? Just challenge your friends directly, you know who will or will not take up the challenge, so why annoy anyone else?

If you're looking for something short and sweet to challenge friends with, you can purchase Missed Three from the App Store.

On re-appraisal of the old ianmjones.net as was, I felt that the site did not reflect who I am now, so I made the decision to start from scratch, to reboot if you will.

You are now reading the first post on that clean slate.

Some might say starting again with a clean slate is not in the spirit of blogging, surely a blog should morph and transform over time as you do, and still others might comment that at the very least I will be losing valuable search ranking.

I’m in the process of cleaning out and simplifying many aspects of my life, and this site is most definitely needing my attention, along with all my other web sites.

The majority of traffic to this site over the last couple of years has been to “off topic” posts about matters that have no long-term interest for me from people who in general won’t have any reason to ever return. That’s not very useful, to me, or to those visitors arriving by searches that I would never have occasion to try myself.

As the last post on the old site notes, my old site succumbed to the WordPress worm that has attacked older versions of WordPress and laid dormant until just recently. It's a good time to clean house.

Also, you may or may not have noticed, but this site does not have any form of commenting mechanism. This is not an over-sight, it is intentional. If you would like to comment on anything I should put up on this site, please feel free to send me an email, tweet, IM or even a dead tree letter, or better still write something thought-provoking on your own site.

Often, when this site has been re-born I’ve made some promises or shared some hopes as to what I expected to do with it, this time I make no such promises or declarations of intent.

Previous versions of this site can be found at:

old.ianmjones.com (covering dates 2004-09-30 to 2009-09-16)
old.ianmjones.net (covering dates 2002-02-11 to 2004-09-30)