Practical Fragment Shaders in Flutter | Guide – Introduction

Over the past few years, Flutter has established itself as the leading framework for creating stunning cross-platform UIs, which encouraged many developers and companies to try their hand at it. 

When learning Flutter, you probably heard the phrase “In Flutter, everything is a widget” on more than one occasion. 

That might have led you to believe that widgets are the limit of Flutter’s capabilities. 

Well, I’m here to tell you that was a lie. 

In this series of articles, we’re going to explore Flutter beyond widgets and (hopefully) find some practical use cases for what we learn. 

Let me introduce… Fragment Shaders!

Related reads:

• Pros and Cons of Flutter App Development

So… What are shaders anyway?

If you’re a gamer like me, you probably heard the word before in the context of enhancing the graphics of your favorite video games. That’s not too far off. To answer what exactly shaders are, let’s head to my go-to tool for learning what stuff is… Wikipedia. Here’s what it says:

Let’s focus on the first sentence for a minute. 

We learn that shaders are computer programs that are used for calculating the lighting of 3D scenes. You can see why that would be helpful for video games. And that’s actually the origin of shaders. 

Way back in the era of computers having barely enough computing power to run the early examples of real-time 3D, developers needed some tools to perform their calculations as efficiently as possible. That’s when a new category of graphics processing focused programs appeared. Since the output of such a program at that time was usually a shaded scene, the name stuck – a shader.

That’s cool and all, but that doesn’t seem useful for our Flutter projects, right? 

We’re pretty much never doing any actual 3D, so why bother with shaders? 

Reading the Wikipedia definition further, we learn that while shaders were originally built for 3D graphics, they were later repurposed to other applications. Among these, we find special effects – exactly what we need for our Flutter UIs!

• Related content: Flutter App Development by Droids On Roids

And what about the “Fragment” part of “Fragment Shader”?

So, now that we know what we can use shaders for, let’s see where they fall in the process of drawing UI to the screen. 

Even though the Flutter Development Team only just started boasting about Fragment Shader API with the release of Flutter 3.7^[2], shaders were an essential part of Flutter since the very beginning. And Flutter itself uses them all the time – without even telling you – for stuff like gradients, blurs, drawing images, etc.^[3] 

When you put a widget in Flutter’s widget tree, Flutter represents this widget as two separate entities under the hood – Element and RenderObject.

The RenderObject is the one responsible for explaining to Flutter Engine how the widget should map to pixels on screen. And it does that using the Canvas API.^[4] If you’ve ever encountered a CustomPaint widget, the idea is essentially the same. Those draw commands are then passed through what is called a rendering pipeline.

And if you pay attention to this pipeline, you might notice that there’s more than one type of shader. 

Let’s deal with the Vertex Shader first.

The name suggests what it’s used for. It takes some geometry as an input, and executes some operation for every vertex of that geometry. That operation might be moving the vertices around, for example, or just doing some useful calculations that may come in handy in the following steps. But since we’re mostly operating on simple rectangles in Flutter, vertex shaders are not very practical for us.

So Fragment Shader, again, as the name suggests, is a shader that works on fragments. 

What fragments are, exactly, is not that simple to explain, but we can just think of a fragment as if it was a single pixel on the screen that we’re trying to draw.

As you get more advanced with the whole programming graphics rendering pipelines ordeal, you’ll learn that’s not always true, but it’s good enough of a lie for what we’re doing here. 

Why not just use Dart?

When you write and run Dart code, the compiler transforms your code into machine code. It essentially translates human-readable language into a language that your hardware will understand. And for most cases, that hardware is your CPU. 

CPUs are built to be versatile tools for dozens of different use cases. They are fast too, with modern consumer CPUs easily breaking hundreds of GFLOPs (that’s hundreds of billions of operations per second). But that’s not good enough. 

See, CPUs are not that great at doing loads of repeatable work at the same time.

Let’s look at a typical screen resolution of 1920×1080, for example. That’s over 2 million pixels that we need to take care of. And ideally, we want to do it 60 times per second. That’s a significant amount of processing power wasted on just updating each pixel on the screen.

We need something better – a GPU.

Since it’s a different piece of hardware, we can’t use the same machine code to give it commands. That’s why most graphics APIs require writing shaders using special programming languages. 

There are a lot of those languages, but they are mostly very similar to each other. And their syntax resembles the classic C syntax a lot, so if you’re comfortable with Dart, you’ll feel right at home reading shader code. The language we’ll use in Flutter is GLSL – a shading language from the OpenGL library.

At this point, a second wave of doubts might be creeping in.

When I say GPU, you probably imagine something like this:

A bulky and expensive brick that you put inside your PC to make video games run faster.

Since most Flutter projects target mobile devices, aren’t shaders useless for us then?

Fortunately, smartphone producers are ahead of our expectations. A typical mobile chip nowadays usually contains a built-in GPU that we can take advantage of.

And don’t be fooled by their small sizes – they are plenty powerful for what we will do.

Let’s get down to business!

It’s time to write some Fragment Shaders in Flutter! 

What’s the first thing you do when you learn a new technology? Write a “HelloWorld” program of course! Except there’s a problem. 

See, the GPU does not have a console that we can print out our “Hello World!” to. And since we’re operating on a level of a single pixel, there’s really no easy way to draw text on the screen. 

So what do we do? 

The answer is simple – for the reasons stated above, the general consensus in the graphics programming world is that the “HelloWorld” equivalent is displaying some color on screen.

And that’s what we’ll do – let’s create a helloworld.frag file in our project directory:

Seems simple enough, right? Let’s talk about what each line of code does. The first one tells the compiler which version of GLSL we’ll be using. Unless you’re after some specific functionality from a specific version, always keep this line the same. 

Next, we need to declare an output variable. It’s a variable where, at the end of our shader, we have to store an output color that will be displayed on screen. We do that by using the out keyword before our variable declaration. You might also notice that the variable I declared is of some weird vec4 type. What’s up with that?

In GLSL, we’re writing code on a way lower level than in Dart. Due to that, we don’t really have a bunch of fancy classes to represent complex stuff like color. Instead, we work directly on data that that color consists of. We’re not completely on our own though – we have vectors. 

Vectors are data structures that basically compose a bunch of numbers into a single entity, and then make your life easier by exposing some useful getters and operators that make implementing algorithms just a little more pleasant. So, to represent a color, we would need four numbers – red, green, blue and alpha channel components – hence the vec4.

Last but not least, we get to the main function. Here, the idea is pretty much the same as in every other programming language – it’s the function that gets automatically invoked when you run the code. Inside it, we’re assigning a new 4-element vector to the output variable, with ones in red and alpha parameters, and zeroes in green and blue. 

When writing fragment shaders in Flutter, you’ll always want to be working with normalized values – meaning 1.0 is the maximum intensity of a parameter, and 0.0 is minimum.

And we’re done with the shader! 

Next, let’s see what we need to do on the Flutter side to get it up and running. First, the pubspec.yaml:

Here, two notable things happen. At the bottom, in a similar fashion to handling assets, we tell Flutter that it should package our helloworld.frag file into the app. In the dependencies section, we add the flutter_shaders package, which is not mandatory but, as you’ll learn soon, will make our life easier.

Now, we need a widget.

The first thing you’ll notice is the ShaderBuilder widget. It’s the first addition we’ll use from the flutter_shaders package. It basically just loads the shader for us and gives us a convenient builder method with the shader ready to use. Inside this builder, we create a CustomPaint widget, and give it both a size and a painter. Now, on to the painter…

…which, as you can see, is as simple as it gets. It takes a loaded shader as a constructor parameter, and then just adds that shader to a Paint object that is then used to paint a rectangle over the whole available canvas.

Let’s run it and see what we get!

Great! Our very first custom shader works!

What’s next?

We’ve learned that building a working fragment shader in Flutter is not as scary as one might think. 

Now that we’re experts in writing some very basic shaders, I think we’re ready to apply this knowledge in some real-life use cases.

That’s what the next articles in this series are going to be about – a guide to creating practical fragment shaders for your Flutter projects. 

But before that, be sure to play around with the code we just wrote to get a better understanding of how it works. 

If you want to explore a bit more, you can also visit my GitHub repository with the code containing examples of custom Flutter shaders I’ll be talking about in the next articles.

Check out also:

• Flutter App Development by Droids On Roids
• Pros and Cons of Flutter App Development

In the meantime, if you have any questions, let me know in the comments!

Discover also: svg_optimizer: Droids on Roids’ Open Source Flutter Package

Related posts