Building Real-time Audio Processing App with SKSL Shaders in .NET MAUI
Development insights for real-time pitch and BPM detection with SkiaCamera audio processing, along with using SKSL shaders.
Building SolTempo: Audio Processing App with SKSL Shaders
Recent enhancements shipped with DrawnUI’s SkiaCamera control (real-time video + audio processing) made creating this app possible. I will touch video processing with real-time encoding in the next article, meanwhile let’s have some fun with the incoming audio stream: this control can work in audio-only monitoring mode.
For correctly singing a double octave you get a golden rays achievement SKSL shader running under two glass-effect shaders, Android.
SolTempo, a published open-source .NET MAUI app for iOS, Mac Catalyst, Android, and Windows does real-time note pitch+BPM detection, it showcases a cross-platform audio pipeline:
- capture mic audio
- apply transforms (Gain +5 in this app, it could be anything: voice changer, EQ, noise gate…),
- analyze audio samples (notes / BPM)
- render visuals from that state
An additional challenge when building this (took about a week) app was to bring out some neat SKSL shaders, similar to a liquid glass simulation and some more. SkiaSharp makes this all possible for .NET MAUI.
App is currently available in AppStore and GooglePlay, you might consider installing it before further reading.
SolTempo Features (Quick Overview)
Here is what the app does:
- Real-time note pitch detection for voice and instruments
- Tuning indicator: shows how sharp/flat you are relative to the nearest semitone
- Multiple note notations: Letters, Solfeggio (fixed/movable), Cyrillic, Numbers
- Optional semitones (C#, Eb, etc.) or “natural notes only” mode
- BPM / tempo detection (roughly 40–260 BPM)
- Audio settings: choose input device (or System Default) and enable Gain (+5) for low signals
- Streak achievements (“Full Octave” / “Perfect Streak”) with confetti and a fullscreen shader celebration
The Single Canvas Approach
SolTempo is completely drawn on a single hardware-accelerated SkiaSharp-backed Canvas. The other native control we use is the one presented via DisplayActionSheet - it’s cool to use this one to keep platform-native feel for users.
All the navigation, modals, and popups happen inside the canvas. To make the UI feel pleasant we use:
- shader-based transitions when switching modules
- shaders for entrance/exit of popups instead of usual scale/fade transforms
- a dynamic liquid glass-like shader backdrop behind the main panel
- a dynamic liquid glass-like shader backdrop behind the bottom icons menu panel
- a constantly drawn audio equalizer at the bottom
- a simple confetti helper when you hit a “Full Octave” streak
- a neat animated shader for the “Perfect Streak” achievement
The UI itself is assembled in code (.NET HotReload-friendly), no XAML this time, and uses DrawnUI for layouts, gestures, shaders etc.
For example creating the glass panel behind the notes module looks like this:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
new SkiaBackdrop()
{
HorizontalOptions = LayoutOptions.Fill,
VerticalOptions = LayoutOptions.Fill,
Blur = 0,
VisualEffects = new List<SkiaEffect>
{
new GlassBackdropEffect()
{
EdgeOpacity = 0.55f,
EdgeGlow = 0.95f,
Emboss = 9.2f,
BlurStrength = 1.0f,
Opacity = 0.9f,
Tint = Colors.Black.WithAlpha(0.33f),
CornerRadius = 24,
Depth = 1.66f
}
}
}
The backdrop captures the background, a custom visal effect applies a shader to it. You can easily create your effects from scratch or subclassing some of the existing. The GlassBackdropEffect took a SkiaShaderEffect and wired up some custom properties on top for use with the glass.sksl shader shipped inside the Resources\Raw MAUI app folder. We would see it in more details later in this article.
You could also use XAML too for DrawnUI as demonstrated by other articles/apps, today i am mainly using code-behind, i like it much how .NET HotReload works with code-behind:
.NET HotReload in action: we are changing module card emboss.
Real-Time Audio with SkiaCamera
If you’ve seen previous camera/shader experiments (like Filters Camera), you might already know a bit about the SkiaCamera control. But here we use it in a slightly different way: audio-only monitoring.
SkiaCamera can provide incoming audio buffers directly, and we can build a cross-platform audio pipeline on top of it. App continuously captures the audio, applies transforms if needed (like an optional +5 audio gain), and analyzes samples to detect pitch / compute BPM. Everything is processed on-device and then fed to UI visualizers.
Audio-only mode
SolTempo defines a SkiaCamera subclass that disables video and enables audio monitoring. I am including it in the UI tree as hidden, but normally for audio monitoring you could create and use it inside a ViewModel like you would use a service.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
public partial class AudioRecorder : SkiaCamera
{
public AudioRecorder()
{
// flags for permissions that will be required when turning control On.
NeedPermissionsSet = NeedPermissions.Microphone;
// turn on AUDIO recorder mode
EnableAudioMonitoring = true;
EnableAudioRecording = true;
// turn off VIDEO
EnableVideoPreview = false;
EnableVideoRecording = false;
}
public float GainFactor { get; set; } = 5.0f;
public bool UseGain { get; set; }
// this is where you can hook whatever processing you want
protected override AudioSample OnAudioSampleAvailable(AudioSample sample)
{
if (UseGain && sample.Data != null && sample.Data.Length > 1)
{
// Amplify PCM16 audio data in-place, no allocations
AmplifyPcm16(sample.Data, GainFactor);
}
OnAudioSample?.Invoke(sample);
return base.OnAudioSampleAvailable(sample);
}
}
Notice that while in SolTempo we do not record anything, processing hook OnAudioSampleAvailable would be also used to transform audio before it would go to real-time audio encoder for saving recorded audio.
Now that the sample is ready to be consumed:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
//we hooked Recorder.OnAudioSample += OnAudioSample;
// now a sample that passed through processing comes in
private void OnAudioSample(AudioSample sample)
{
// notes detector module
if (_musicNotesWrapper.IsVisible)
NotesModule.AddSample(sample);
// our BPM module
if (_musicBPMDetectorWrapper.IsVisible)
_musicBPMDetector?.AddSample(sample);
// EQ drawn on bottom
if (_equalizer.IsVisible)
_equalizer.AddSample(sample);
}
Audio analysis routines can be easily created with AI upon your needs, let us focus on presentation.
Rendering Modules
After the data received via AddSample was analyzed we need to paint our UI. We use DrawnUI for NET MAUI to be able to unleash the power of SkiaSharp here, it brings its own Canvas handlers, focused on UI rendering, fps-control and display sync, plus a comfortable WPF/MAUI-like layout system with gestures support etc.
SolTempo modules use SkiaSharp in two main ways:
- Using DrawnUI controls (
SkiaLabel,SkiaShape, layouts) for everything that feels like UI - Painting directly on SKCanvas for drawings like waveforms, EQ shapes etc..
Drawn Controls
A DrawnUI control would be painted every frame if it and all of its parents are not cached or it’s cache is invalidated. Why caching? Intead of drawing/calculating layouts/shadows/fonts etc on every frame we can fast draw either a pre-rendered bitmap (SkImage) or a previously recorded set of drawing operations (SkPicture). Using caching properly is making DrawnUI to operate in retained mode.
Cached is invalidated manually or when some property changes, like Text property of a SkiaLabel .
In our app we have to invalidate manualy (call Update()) when we processed audio and we want to draw a changed visual EQ graphic. App canvas redraws only if something really changed, contrary to the usual SkiaSharp usage flow.
Speaking of drawn controls example, here is our BPM detection module constructor:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
public AudioMusicBPM()
{
UseCache = SkiaCacheType.Operations;
Children = new List<SkiaControl>
{
new SkiaLabel
{
FontSize = 140,
//MonoForDigits = "8", <-- this would make font act as mono, digits will take width of "8" and text will not "jump" when number changes, might useful for HUDs etc. We don't use this on purpose here to get a more vivid and less "toolish" look.
CharacterSpacing = 5.0,
Margin = new (2,16),
MaxLines = 1,
LineBreakMode = LineBreakMode.CharacterWrap,
UseCache = SkiaCacheType.Operations,
FontAttributes = FontAttributes.Bold,
FontFamily = AppFonts.Default,
TextColor = Colors.White,
HorizontalOptions = LayoutOptions.Center,
}.Assign(out _labelBpm),
new SkiaLabel
{
Text = "BPM",
Margin = new(0,150,0,0),
FontSize = 24,
FontFamily = AppFonts.Default,
TextColor = Colors.Gray,
HorizontalOptions = LayoutOptions.Center,
UseCache = SkiaCacheType.Operations,
}.Assign(out _labelBpmUnit),
new SkiaLabel
{
FontSize = 19,
Margin = new(0,180,0,0),
FontFamily = AppFonts.Default,
TextColor = Colors.LimeGreen,
HorizontalOptions = LayoutOptions.Center,
UseCache = SkiaCacheType.Operations,
}.Assign(out _labelConfidence),
new SkiaLabel
{
Margin = new Thickness(16,40),
Text = "Tap to reset BPM metering",
FontSize = 22,
FontFamily = AppFonts.Default,
TextColor = Colors.LightGray,
VerticalOptions = LayoutOptions.Start,
HorizontalOptions = LayoutOptions.Center,
UseCache = SkiaCacheType.Operations,
IsVisible = true,
}.Assign(out _labelNoSignal),
};
}
Using fluent extensions here. And let’s not forget about gestures:
1
2
3
4
5
6
7
8
9
public override ISkiaGestureListener ProcessGestures(SkiaGesturesParameters args, GestureEventProcessingInfo apply)
{
if (args.Type == TouchActionResult.Tapped)
{
Reset(); //reset our audio module to start analysing from scratch
return this; //this means "who consumed the gesture"
}
return base.ProcessGestures(args, apply); //would return null or one of the possible children if they consume anything
}
Accessing Canvas DIrectly
We can override the main painting method of any SkiaControl to access the drawing surface:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
protected override void Paint(DrawingContext ctx)
{
base.Paint(ctx); //background + changed children, like our labels etc, will be painted automatically inside
//we have total access to SkiaSharp canvas to draw EQ lines etc, all data we might need is:
var canvas = ctx.Context.Canvas; //SkCanvas
float scale = ctx.Scale; //density, how many pixels in one point
SKRect destination = this.DrawingRect; //in pixels, after measure/arrange
//an example of a usual SkiaSharp primitive:
canvas.DrawOval(destination.Width/2.0f, destination.Height/2.0f, 15 * scale, 11 * scale, somePaint); //if we use scale it will look same size on any device/platform
}
Shaders Everywhere
Okay now the most interesting for me part of SolTempo: an intensive use of shaders. Instead of the “usual” look, we go with SkiaSharp v3 SKSL wherever we can. This is also where the “single canvas” approach becomes useful, as we can make the whole UI elements tree be affected by shaders.
We already have been using shaders in Filters Camera and ShadersCarousel apps, those gave us the a base for a confident use.
Glass Backdrop
We are in 2026 so we couldn’t pass on using a liquid glass-like effect, that could give the app a modern look. Our buttons bar needed this badly:
Dynamic backdrop with glass effect. Hover effects for buttons on desktop.
An obvious choice was then to reuse it for main audio modules too, and this defined the final look of the app. This all was implemented as a GlassBackdropEffect (a small wrapper around SkiaShaderEffect), attachable to any control, it provides a lot of customizable properties like corner radius, emboss/refraction, edge glow, tint and much more. Shader runs when the parent SkiaBackdrop control redraws.
I took a MIT licenced https://github.com/bergice/liquidglass shader and deeply modified it, that resulted in a highly customisable visual effect:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
public class GlassBackdropEffect : SkiaShaderEffect
{
public GlassBackdropEffect()
{
ShaderSource = @"Shaders\glass.sksl"; //shipped inside `Resources/Raw` MAUI app folder
}
protected override SKRuntimeEffectUniforms CreateUniforms(SKRect destination)
{
var uniforms = base.CreateUniforms(destination);
var scale = Parent?.RenderingScale ?? 1f;
uniforms["iCornerRadius"] = CornerRadius * scale;
uniforms["iEmboss"] = Emboss;
uniforms["iDepth"] = Depth;
uniforms["iBlurStrength"] = BlurStrength;
uniforms["iOpacity"] = Opacity;
uniforms["iEdgeOpacity"] = EdgeOpacity;
uniforms["iEdgeGlow"] = EdgeGlow;
var c = Tint;
_uniformTint[0] = (float)c.Red; _uniformTint[1] = (float)c.Green;
_uniformTint[2] = (float)c.Blue; _uniformTint[3] = (float)c.Alpha;
uniforms["iTint"] = _uniformTint;
return uniforms;
}
Animated Popups
In the mood of extensively using shaders I added entrance/exit shaders for popups that show help and settings. no more standart scale/fade transforms. In short we attach an entrance shader to show and an exit shader to hide the control.
I created an AnimatedPopup class for that, and help and setting use it as base. I invite you to dig into the source code for a deeper look.
Transition for switching modules
This is a single-texture transition SKSL shader driven by a progress animator. At progress 0.0 we set the current control as shader texture source, at 0.5 (Midpoint) we set the second control as texture source to be used up to 1.0, and we run a progress animator built into a TransitionEffect.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
var fx = new TransitionEffect();
fx.Midpoint += (s, e) =>
{
ToggleVisualizerMode();
fx.AquiredBackground = false;
};
fx.Completed += (s, e) =>
{
_mainStack.VisualEffects.Remove(fx);
_mainStack.DisposeObject(fx);
};
_mainStack.VisualEffects.Add(fx);
fx.Play();
Transitionning to BPM module to detect music tempo.
Confetti
No shaders here! A simple “confetti helper” works well here, all is drawn with SkiaSharp primitives. Try sing a full octave to trigger them 😄!
Achievement Effect
When you sing a streak of correct notes for a double octave the app triggers encouraging golden rays effect, as seen in the video in the beginning of this article. For the “Perfect Streak” we run an animated fullscreen AchievementEffect SKSL shader (and remove it when done):
1
2
3
4
5
6
7
8
9
var fx = new AchievementEffect();
fx.Completed += (s, e) =>
{
_background.VisualEffects.Remove(fx);
_background.DisposeObject(fx);
};
_background.VisualEffects.Add(fx);
fx.Play();
Built-in Live Shader Editor
Writing SKSL shaders can be a trial-and-error process. To speed this up, I was using a built-in shader live editor that runs when the app is run on Windows. It opens when you press Settings button, i was attaching the shader to be edited via a shaderGlass variable:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
VisualEffects = new List<SkiaEffect>
{
new GlassBackdropEffect()
{
EdgeOpacity = 0.55f,
EdgeGlow = 0.95f,
Emboss = 9.2f,
BlurStrength = 1.0f,
Opacity = 0.9f,
Tint = Colors.Black.WithAlpha(0.33f),
CornerRadius = 24,
Depth = 1.66f
}.Assign(out shaderGlass) //for dev shader editor
}
which was used later like this:
1
2
3
4
5
6
7
private void TappedSettings()
{
_settingsPopup?.Show();
#if DEBUG && WINDOWS
OpenShaderEditor(shaderGlass);
#endif
}
This means you can tweak the SKSL code inside the app, hit Apply, and instantly see the liquid glass background or the popup transition change in real-time to your changed SKSL code, no need to restart the app for that. I used a similar workflow for Filters Camera app.
Additional Insights
Help popup content shipped as Markdown
App Help text required formatting, the Help popup loads its content from a Markdown file shipped inside the app package (Resources/Raw/Markdown/help.en.md). The popup just reads it at runtime once then sets a SkiaRichLabel property Text to markdown. Among other features this rather powerful control can parse and render markdown strings, including creating links.
Capping FPS on iOS (battery-friendly)
On iOS skia view is using Apple Metal for hardware accelerated rendering. This one can be very power consuming (and heat generating) when running at max fps. Since this is not a game but we still need to render constantly when sound comes at 48000 Hz rate on iPhone we capped fps:
1
2
3
#if IOS // spare battery because apple metal is draining much
Super.MaxFps = 30;
#endif
It is a small change which makes a difference for a “real-time” app that can run for hours if you practice solfeggio.
Ask For Rating
As the last step i quickly pluged Plugin.Maui.AppRating library, to ask people to rate the app in the store: only once and only if app was used for more than 1 hour. Was very easy to do, thanks to the plugin creator!
Final Thoughts
SolTempo being a compact playground for real-time audio analisys, visualization and SKSL shaders usage, we demonstrated that .NET MAUI has rather extended usage limits. With SkiaSharp and the drawn approach you can ship apps that look and feel very far from the usual.
SolTempo is fully open-source, Feel free to grab the code, experiment with the shader editor on Windows, and see what you can build.
App does not collect, store, or share personal data, audio analysis happens locally on your device and all data stays on it.
Links and Resources
- SolTempo - complete source code
- AppStore - install app on iOS
- GooglePlay - install app on Android
- DrawnUI for .NET MAUI - the canvas rendering engine
- SkiaSharp - the underlying 2D graphics library
- SKSL documentation - Skia Shading Language reference
- Plugin.Maui.AppRating - for a fast and easy way to ask users to rate the app
The author is available for consulting on drawn applications and custom controls for .NET MAUI. If you need help creating custom UI experiences, optimizing performance, or building entirely drawn apps, feel free to reach out.