Practical Fragment Shaders in Flutter | Guide – Improving Blurhash

Previously in the Practical Fragment Shaders series, we’ve been learning how to create stunning real-time visual effects and animations that we can apply to our widgets or even a device’s camera feed. And while that’s awesome to look at and very satisfying to implement, you might argue that the practical applications of such shaders are limited, as not every project has the requirements or time to develop such features. So this time, let’s apply our knowledge into writing a shader that every single app could benefit from.

Meet Blurhash

Isn’t it annoying when an application takes a while to load its content, and all you see during the wait is a blank screen? What if I told you that there is an easy solution for that? Blurhash!

But what is Blurhash exactly? You could say it’s an image thumbnail generation algorithm. It was invented by Wolt. The idea is that when you need to load and display a picture, while the full resolution image is loading, you could show a placeholder image, to fill the empty space. But not just any placeholder – It’s a low-res, blurred representation of the original picture! And that’s what Blurhash does – it converts a full-size image into a short encoded string, usually around 20-30 characters in length, that can then be decoded to display said thumbnail.

The way you would use Blurhash is that, when an image gets uploaded to your app’s back-end, the back-end would need to compute and save a Blurhash string along with the original image. Then, when the app requests to load the image, the server would first respond with the Blurhash string, which would allow the app to draw the placeholder image right away, and then proceed to load the full resolution picture.

As you can see, it’s a very elegant and easily implementable improvement to the overall user experience of any app.

Why bother with shaders?

While Wolt didn’t officially release their implementation of Blurhash for Flutter, a quick search on pub.dev might tell you that there’s already a number of Blurhash packages available and ready to use. And yet, none of them seem to be using shaders. They either rely on another platform’s native libraries and Flutter’s method channels to access them, or just write their custom implementations entirely in Dart. What’s the deal with that?

The short answer is: it’s just easier that way. But is it better? Let’s see:

• Using platform native packages is sure to provide great performance, but it’s a liability – you have to keep track of all the versions of native libraries for each platform and possibly their dependencies, too. And if just one of them breaks after an update, you might be screwed.
• The pure Dart implementation benefits from not needing to maintain native code, but this comes at the cost of performance, as Dart lacks any tools to parallelize drawing the blurred image.

But what if we use shaders? We could keep the implementation free of native dependencies and still have the image drawn to the screen in a super performant way benefitting from shader’s parallelization and GPU usage.

The code

First, let’s set up a benchmark, so that we’ll have something to compare to. I’ll be using the flutter_blurhash package, which is by far the most popular Dart implementation of Blurhash. It’s also written in pure Dart, which will make for a great performance comparison for our shader.

Let’s import flutter_blurhash into our project, prepare a bunch of Blurhash strings to display and create an infinite scroll page displaying those images:

You’ll notice I’ve set the decodingHeight and decodingWidth to pretty high values – that’s in order to make the performance difference more obvious when we run the benchmark.

When we run it, we should get something like this:

As you can see, there’s a noticeable load time when we scroll down. Let’s see if we can do something about it.

Inside Blurhash

Let’s take a peek inside the flutter_blurhash package to learn how it actually works and where we can make some improvements.

Our entrypoint is the BlurHash widget, which takes our hash as one of the parameters. This hash is then put through a _decodeImage() method, that calls the blurHashDecodeImage() function.

That function then calls another one called blurHashDecode() that does the actual decoding.

Can you see if anything stands out here? Yes, the nested for loop! That’s a perfect spot to introduce a fragment shader, because then, instead of going through the image’s pixels one-by-one, we’ll do them all in parallel!

So how do we go about it?

First, we need to modify how the blurHashDecode() function is structured a little bit – we need to split it in two parts:

1. The decoding part, which will stay as is.
2. The image generation part, that we’ll convert into shader.

Let’s first move the decoding part into a new method:

Then we can change the BlurHash widget code to be able to use a shader implementation if we choose to:

Let’s run it and see if it has improved anything:

Smooth as butter!

The drawbacks

Our solution works great and yields a very nice UX improvement, with the images loading nearly instantly, but if you read the shader code carefully, you’ve probably already noticed that it’s not perfect. How so? The additional two for-loops inside the shader have hardcoded i and j limits, instead of parametrized ones. Those are the parameters that are representative of BlurHash’s “components” parameters during encoding.

So why did we have to code it that way?

Because of Flutter’s Fragment Shader API’s limitations.

As of writing this article, it doesn’t support dynamic loops, which means we can only use hardcoded values when looping over something. And even if it did, there’s no support for UBOs (Uniform Buffer Objects) and SSBOs (Shader Storage Buffer Objects), which means we have no good options to input any dynamically sized data structure (apart from textures) into our shaders.

So is all we’ve learned useless right now?

Of course not!

We’ve got a BlurHash shader that works perfectly for a specified encoding configuration. I think that, when it comes to real-life projects, it’s perfectly reasonable to make some assumptions about the data we’ll be receiving and processing. It’s very rare that you’d need an algorithm that would need to handle every possible configuration of all possible data in a controlled environment like a Flutter app. And if you need to handle multiple configurations, there’s nothing stopping you from taking our original shader and tweaking it to handle a different case – and you can even have both shaders within your project.

The end

This article concludes our Practical Fragment Shaders in Flutter series. We’ve started with the absolute basics by just drawing some colors to the screen before learning the ins and outs of fragment shader programming in Flutter and actually utilizing those skills in practical scenarios.

I hope you’ve had as much fun reading those tutorials and learning along the way as I had while writing.

As always – feel free to write any comments and questions and check out my repositories with the shaders and BlurHash fork.

Related posts