Building a Real-time Audio Processing App with SKSL Shaders in .NET MAUI

Development insights for real-time audio pitch and BPM detection with SkiaCamera, along with using SKSL shaders.

Posted Mar 5, 2026 Updated Mar 10, 2026

By Nick Kovalsky

15 min read

Building a Real-time Audio Processing App with SKSL Shaders in .NET MAUI

Creating an Audio Processing App with SKSL Shaders

Recent enhancements shipped with DrawnUI SkiaCamera control brought features like real-time video and audio processing. We will talk about video frames processing in the next article, meanwhile let’s have some fun with the audio stream: we can work in audio-only monitoring mode.

For correctly singing a double octave you get a golden rays achievement SKSL shader running under two glass-effect shaders, Android.

For the purposes of this article I created an open-source .NET MAUI app for iOS, Mac Catalyst, Android, and Windows. It will serve as a playground for the app monitoring/processing feature of the SkiaCamera control. An additional challenge for us would be to bring out some neat SKSL shaders, similar to a liquid glass simulation and some more. SkiaSharp makes this all possible for .NET MAUI.

App will showcase the following cross-platform audio processing pipeline:

capture mic audio
apply transforms (Gain +5 in this app, it could be anything: voice changer, EQ, noise gate…),
analyze audio samples (notes / BPM)
render visuals from that state

For a better demonstrative effect app was published under the name of SolTempo in AppStore and GooglePlay.

App Features

Real-time note pitch detection for voice and instruments with tuning indicator: how sharp/flat you are relative to the nearest semitone
Multiple note notations: Letters, Solfeggio (fixed/movable), Cyrillic, Numbers
Optional semitones (C#, Eb, etc.) or “natural notes only” mode
BPM / tempo detection (40–260)
Audio settings: choose input device (or System Default)
Audio transform: will apply an optional Gain (+5) to audio samples
Visual achievements (“Full Octave” / “Perfect Streak”) with confetti and a fullscreen shader celebration

The Single Canvas Approach

App will be completely drawn on a single hardware-accelerated SkiaSharp-backed Canvas. The other native control we will use is the one presented via DisplayActionSheet - it’s cool to use this one to keep platform-native feel for users.

All the navigation, modals, and popups happen inside the canvas. To make the UI feel pleasant we use:

shader-based transitions when switching modules
shaders for entrance/exit of popups instead of usual scale/fade transforms
a dynamic liquid glass-like shader backdrop behind the main panel
a dynamic liquid glass-like shader backdrop behind the bottom icons menu panel
a constantly drawn audio equalizer at the bottom
a simple confetti helper when you hit a “Full Octave” streak
a neat animated shader for the “Perfect Streak” achievement

The UI itself is assembled in code (.NET HotReload-friendly), no XAML this time, and uses DrawnUI for layouts, gestures, shaders etc.

For example creating the glass panel behind the notes module will look like this:

  
new SkiaBackdrop()
{
	HorizontalOptions = LayoutOptions.Fill,
	VerticalOptions = LayoutOptions.Fill,
	Blur = 0,
	VisualEffects = new List<SkiaEffect>
	{
		new GlassBackdropEffect()
		{
			EdgeOpacity = 0.55f,
			EdgeGlow = 0.95f,
			Emboss = 9.2f,
			BlurStrength = 1.0f,
			Opacity = 0.9f,
			Tint = Colors.Black.WithAlpha(0.33f),
			CornerRadius = 24,
			Depth = 1.66f
		}
	}
}

The backdrop captures the background, a custom visual effect applies a shader to it. You can easily create your effects from scratch or subclassing some of the existing. The GlassBackdropEffect took a SkiaShaderEffect and wired up some custom properties on top for use with the glass.sksl shader shipped inside the Resources\Raw MAUI app folder. We would see it in more details later in this article.

You could also use XAML too, but today we will use code-behind, it works very well with .NET HotReload:

.NET HotReload in action: we are changing module card emboss.

Real-Time Audio with SkiaCamera

SkiaCamera can provide incoming audio buffers directly, and we can build a cross-platform audio pipeline on top of it. App continuously captures the audio, applies transforms if needed (like an optional +5 audio gain), and analyzes samples to detect pitch / compute BPM. Everything is processed on-device and then fed to UI visualizers.

Audio-only mode

SolTempo defines a SkiaCamera subclass that disables video and enables audio monitoring. I am including it in the UI tree as hidden, but normally for audio monitoring you could create and use it inside a ViewModel like you would use a service.

  
public partial class AudioRecorder : SkiaCamera
{
	public AudioRecorder()
	{
        // flags for permissions that will be required when turning control On.
		NeedPermissionsSet = NeedPermissions.Microphone;

		// turn on AUDIO recorder mode
		EnableAudioMonitoring = true;
		EnableAudioRecording = true;

		// turn off VIDEO
		EnableVideoPreview = false;
		EnableVideoRecording = false;
	}

	public float GainFactor { get; set; } = 5.0f;
	public bool UseGain { get; set; }

    // this is where you can hook whatever processing you want
	protected override AudioSample OnAudioSampleAvailable(AudioSample sample)
	{
		if (UseGain && sample.Data != null && sample.Data.Length > 1)
		{
			// Amplify PCM16 audio data in-place, no allocations
			AmplifyPcm16(sample.Data, GainFactor);
		}

		OnAudioSample?.Invoke(sample);
		return base.OnAudioSampleAvailable(sample);
	}
}

Notice that while in this exercise we do not record anything, processing hook OnAudioSampleAvailable would be also used to transform audio before it would go to real-time audio encoder for saving recorded audio.

Now that the sample is ready to be consumed:

  
//we hooked Recorder.OnAudioSample += OnAudioSample;
// now a sample that passed through processing comes in
private void OnAudioSample(AudioSample sample)
{
	// notes detector module
	if (_musicNotesWrapper.IsVisible)
		NotesModule.AddSample(sample);

	// our BPM module
	if (_musicBPMDetectorWrapper.IsVisible)
		_musicBPMDetector?.AddSample(sample);

	// EQ drawn on bottom
	if (_equalizer.IsVisible)
		_equalizer.AddSample(sample);
}

Audio analysis routines can be easily created with AI upon your needs, let us focus on presentation.

Rendering Modules

After the data received via AddSample was analyzed we need to paint our UI. We use DrawnUI for NET MAUI to be able to unleash the power of SkiaSharp here, it brings its own Canvas handlers, focused on UI rendering, fps-control and display sync, plus a comfortable WPF/MAUI-like layout system with gestures support etc.

App modules will use SkiaSharp in two main ways:

Using DrawnUI controls (SkiaLabel, SkiaShape, layouts) for everything that feels like UI
Painting directly on SKCanvas for drawings like waveforms, EQ shapes etc..

Drawn Controls

A DrawnUI control would be painted every frame if it and all of its parents are not cached or it’s cache is invalidated. Why caching? Instead of drawing/calculating layouts/shadows/fonts etc on every frame we can fast draw either a pre-rendered bitmap (SkImage) or a previously recorded set of drawing operations (SkPicture). Using caching properly is making DrawnUI to operate in retained mode.

Cached is invalidated manually or when some property changes, like Text property of a SkiaLabel .
In our app we have to invalidate manually (call Update()) when we processed audio and we want to draw a changed visual EQ graphic. App canvas redraws only if something really changed, contrary to the usual SkiaSharp usage flow.

Speaking of drawn controls example, here is our BPM detection module constructor:

  
public AudioMusicBPM()
{
    UseCache = SkiaCacheType.Operations;

    Children = new List<SkiaControl>
    {
        new SkiaLabel
        {
            FontSize = 140,
            //MonoForDigits = "8", <-- this would make font act as mono, digits will take width of "8" and text will not "jump" when number changes, might useful for HUDs etc. We don't use this on purpose here to get a more vivid and less "toolish" look.
            CharacterSpacing = 5.0,
            Margin = new (2,16),
            MaxLines = 1,
            LineBreakMode = LineBreakMode.CharacterWrap,
            UseCache = SkiaCacheType.Operations,
            FontAttributes = FontAttributes.Bold,
            FontFamily = AppFonts.Default,
            TextColor = Colors.White,
            HorizontalOptions = LayoutOptions.Center,
        }.Assign(out _labelBpm),

        new SkiaLabel
        {
            Text = "BPM",
            Margin = new(0,150,0,0),
            FontSize = 24,
            FontFamily = AppFonts.Default,
            TextColor = Colors.Gray,
            HorizontalOptions = LayoutOptions.Center,
            UseCache = SkiaCacheType.Operations,
        }.Assign(out _labelBpmUnit),

        new SkiaLabel
        {
            FontSize = 19,
            Margin = new(0,180,0,0),
            FontFamily = AppFonts.Default,
            TextColor = Colors.LimeGreen,
            HorizontalOptions = LayoutOptions.Center,
            UseCache = SkiaCacheType.Operations,
        }.Assign(out _labelConfidence),

        new SkiaLabel
        {
            Margin = new Thickness(16,40),
            Text = "Tap to reset BPM metering",
            FontSize = 22,
            FontFamily = AppFonts.Default,
            TextColor = Colors.LightGray,
            VerticalOptions = LayoutOptions.Start,
            HorizontalOptions = LayoutOptions.Center,
            UseCache = SkiaCacheType.Operations,
            IsVisible = true,
        }.Assign(out _labelNoSignal),

    };
}

Using fluent extensions here. And let’s not forget about gestures:

  
        public override ISkiaGestureListener ProcessGestures(SkiaGesturesParameters args, GestureEventProcessingInfo apply)
        {
            if (args.Type == TouchActionResult.Tapped)
            {
                Reset(); //reset our audio module to start analysing from scratch
                return this; //this means "who consumed the gesture"
            }
            return base.ProcessGestures(args, apply); //would return null or one of the possible children if they consume anything
        }

Accessing Canvas Directly

We can override the main painting method of any SkiaControl to access the drawing surface:

  
protected override void Paint(DrawingContext ctx)
    {
        base.Paint(ctx); //background + changed children, like our labels etc, will be painted automatically inside

        //we have total access to SkiaSharp canvas to draw EQ lines etc, all data we might need is:
        var canvas = ctx.Context.Canvas; //SkCanvas
        float scale = ctx.Scale; //density, how many pixels in one point
        SKRect destination = this.DrawingRect; //in pixels, after measure/arrange
            
        //an example of a usual SkiaSharp primitive:
        canvas.DrawOval(destination.Width/2.0f, destination.Height/2.0f, 15 * scale, 11 * scale, somePaint); //if we use scale it will look same size on any device/platform
    }

Shaders Everywhere

Maybe the most interesting part: an intensive use of shaders. Instead of the “usual” look, let’s go with SkiaSharp v3 SKSL shaders wherever we can. Here the “single canvas” approach becomes very useful, as we can affect all UI elements at once with our shaders.

Some previous shaders usecase were Filters Camera and ShadersCarousel demo apps, those gave us a base for a confident use now.

Glass Backdrop

We are in 2026 so we couldn’t pass on using a liquid glass-like effect, that could give the app a modern look. Our buttons bar needed this badly:

Dynamic backdrop with glass effect. Hover effects for buttons on desktop.

An obvious choice would be to reuse it for main audio modules too. Let’s create a GlassBackdropEffect (a small wrapper around SkiaShaderEffect), attachable to any control, it provides a lot of customizable properties like corner radius, emboss/refraction, edge glow, tint and much more. Shader runs when the parent SkiaBackdrop control redraws.

I found a nice MIT licenced https://github.com/bergice/liquidglass GLSL shader to be deeply modified, so it could serve for our needs for inside a highly customisable visual effect:

  
public class GlassBackdropEffect : SkiaShaderEffect
{
	public GlassBackdropEffect()
	{
		ShaderSource = @"Shaders\glass.sksl"; //shipped inside `Resources/Raw` MAUI app folder
	}

protected override SKRuntimeEffectUniforms CreateUniforms(SKRect destination)
{
     var uniforms = base.CreateUniforms(destination);

     var scale = Parent?.RenderingScale ?? 1f;
     uniforms["iCornerRadius"] = CornerRadius * scale;
     uniforms["iEmboss"] = Emboss;

     uniforms["iDepth"] = Depth;
     uniforms["iBlurStrength"] = BlurStrength;
     uniforms["iOpacity"] = Opacity;
     uniforms["iEdgeOpacity"] = EdgeOpacity;
     uniforms["iEdgeGlow"] = EdgeGlow;

     var c = Tint;
     _uniformTint[0] = (float)c.Red; _uniformTint[1] = (float)c.Green;
     _uniformTint[2] = (float)c.Blue; _uniformTint[3] = (float)c.Alpha;
     uniforms["iTint"] = _uniformTint;

     return uniforms;
 }

Animated Popups

Why not also use shaders for entrance/exit animations of popups? No more standard scale/fade transforms, we can attach an entrance shader to show and an exit shader to hide the control:

Created an AnimatedPopup class for that, and help and settings use it as base. I invite you to dig into the source code for a deeper look.

Transition for switching modules

This is a single-texture transition SKSL shader driven by a progress animator. At progress 0.0 we set the current control as shader texture source, at 0.5 (Midpoint) we set the second control as texture source to be used up to 1.0, and we run a progress animator built into a TransitionEffect.

  
var fx = new TransitionEffect();
fx.Midpoint += (s, e) =>
{
	ToggleVisualizerMode();
	fx.AquiredBackground = false;
};
fx.Completed += (s, e) =>
{
	_mainStack.VisualEffects.Remove(fx);
	_mainStack.DisposeObject(fx);
};

_mainStack.VisualEffects.Add(fx);
fx.Play();

Transitioning to BPM module to detect music tempo.

Confetti

No shaders here! A simple “confetti helper” works well, all is drawn with SkiaSharp primitives. After checking the source code, try sing a full octave to trigger them 😄!

Achievement Effect

When you sing a streak of correct notes for a double octave the app triggers encouraging golden rays effect, as seen in the video in the beginning of this article. For the “Perfect Streak” we run an animated fullscreen AchievementEffect SKSL shader (and remove it when done):

  
var fx = new AchievementEffect();
fx.Completed += (s, e) =>
{
	_background.VisualEffects.Remove(fx);
	_background.DisposeObject(fx);
};

_background.VisualEffects.Add(fx);
fx.Play();

Built-in Live Shader Editor

Writing SKSL shaders can be a trial-and-error process. To speed this up, we could use a built-in shader live editor that runs when the app is run on Windows. It opens when you press Settings button, we will attach the shader to be edited via a shaderGlass variable:

  
 VisualEffects = new List<SkiaEffect>
{
	new GlassBackdropEffect()
	{
		EdgeOpacity = 0.55f,
		EdgeGlow = 0.95f,
		Emboss = 9.2f,
		BlurStrength = 1.0f,
		Opacity = 0.9f,
		Tint = Colors.Black.WithAlpha(0.33f),
		CornerRadius = 24,
		Depth = 1.66f
	}.Assign(out shaderGlass) //for dev shader editor
}

which was used later like this:

  
    private void TappedSettings()
    {
        _settingsPopup?.Show();
#if DEBUG && WINDOWS
        OpenShaderEditor(shaderGlass);
#endif
    }

This means you can tweak the SKSL code inside the app, hit Apply, and instantly see the liquid glass background or the popup transition change in real-time to your changed SKSL code, no need to restart the app for that. I used a similar workflow for Filters Camera app.

Additional Insights

App Help text required formatting, the Help popup loads its content from a Markdown file shipped inside the app package (Resources/Raw/Markdown/help.en.md). The popup just reads it at runtime once then sets a SkiaRichLabel property Text to markdown. Among other features this rather powerful control can parse and render markdown strings, including creating links.

Capping FPS on iOS (battery-friendly)

On iOS skia view is using Apple Metal for hardware accelerated rendering. This one can be very power consuming (and heat generating) when running at max fps. Since this is not a game but we still need to render constantly when sound comes at 48000 Hz rate on iPhone we cap fps:

  
#if IOS // spare battery because apple metal is draining much
Super.MaxFps = 30;
#endif

It is a small change which makes a difference for long running apps.

Ask For Rating

As the last step we could quickly plug Plugin.Maui.AppRating library, to ask people to rate the app in the store: only once and only if app was used for more than 1 hour. Was very easy to do, thanks to the plugin creator!

Final Thoughts

We created a compact playground for real-time audio analysis, visualization and SKSL shaders usage, and demonstrated that .NET MAUI has rather extended usage limits. With SkiaSharp and the drawn approach we can ship apps that look and feel very far from the usual.

Please feel free to grab the code, maybe also experiment with the shader editor on Windows, and see what you can build.

Links and Resources

SolTempo - complete source code
AppStore - install app on iOS
GooglePlay - install app on Android
DrawnUI for .NET MAUI - the canvas rendering engine
SkiaSharp - the underlying 2D graphics library
SKSL documentation - Skia Shading Language reference
Plugin.Maui.AppRating - for a fast and easy way to ask users to rate the app

The author is available for consulting on drawn applications and custom controls for .NET MAUI. If you need help creating custom UI experiences, optimizing performance, or building entirely drawn apps, feel free to reach out.

MAUI, DrawnUI, Audio, Shaders

This post is licensed under CC BY 4.0 by the author.