Modern Music Visuals

October 20, 2021

Modern Music Visuals

Modern Music Visuals


Creating modern music visuals for performance has without a doubt come a long way in the past 20 years. Considering that things started with nothing but VHS tapes, video production has become a lot easier for VJs and video artists.


There are increasingly more and more VJing and music visualizer software out there with better accessibility and simpler workflows. VJs no longer need to know how to work with complex setups and tools and writing code is also becoming less of a requirement for video producers, thanks to modern music visualizer software. In this article, we will explore the amazing journey of music visuals from its beginning and look at what the future might possibly look like (including online music visualizers) as tools become better and more accessible.


Creative Coding 


Coding visuals is still one of the most common ways of making modern music visuals especially if audio reactive content is what you’re looking for. There are many tools available at the moment that make it easy to experiment with creative coding and expand your library of generative visuals.




Creative coding - shaders


Shaders have been around longer than most people expect but every VJ application out there still relies on them in some form or the other. Shaders are essentially small programs designed to run on your graphics card. They perform a given set of operations on each pixel of a video frame using the parallel processing power of the GPU. GLSL is now the most common and popular shader language and chances are every VJ software you have ever used is heavily based on this type of shader.


Shaders are here to stay for a long time and while tools and applications come and go, shaders will always be the building block of generative and modern music visuals in some shape or form and being familiar with them is a huge plus.




Creative coding - milkdrop


Milkdrop was one of the earliest and most revolutionary tools that completely changed how modern music visuals were made. You probably remember winamp’s built-in visualizer in Windows with its contantly shifting waves and colors. Truth is that at the time this tool was released there was nothing else like it. It featured advanced beat detection, audio processing and  a preset (scene) creation system that bred creativity.


Artists could create mindbending generative content and remix each other’s work to create completely unique visuals all in realtime and without any high end equipment.


Of course making visuals with Milkdrop was not easy and it needed a good understanding of creative coding and shaders but the results were definitely worth the wait. Nevertheless Milkdrop kickstarted the creation of a whole new generation of tools and is still being used by many music visualization applications today.




Creative coding - processing IDE


The Processing IDE remains a powerful modern tool for creative coders despite its limitations. This environment was made to make it easier for artists to learn to code visuals. It has a lot of built-in functionality that allows you to stay away from the rabbit hole that is creative coding. There are many libaries for sound processing, writing shaders, AI and even for simulations.


There are still shortcomings with Processing but it is a great tool to get started with if you are looking to get started with creative coding.




Creative coding - OpenFrameworks


OpenFrameworks is a more advanced counterpart of the Processing IDE. It has a lot of the same structure when it comes to creating applications and it also maintains the same modular approach. The advantage of using OpenFrameworks is that it has a lot more features and tools that allow artists to create better and more modern music visuals faster even using the most basic setup. There are modules for 3D modeling, advanced audio and video processing that focus on the power of the graphics card to create amazing music visuals.


If you are an intermediate or advanced creative coder, OpenFrameworks is the way to go.


Modern Music Visuals


Creative coding is amazing but it is not the way to go if you are a performing artist. If your main goal is to set up your equipment and load your software application and be ready to produce a live show, you definitely do not want to think about experimenting with code.


There is a need to distinguish between the role of a creative coder and performing video artist which is why the shader-based formats we talked about have been adapted to suite the need of a live performer a little better.


While these new formats are still incomplete and do not fully separate the content creation and performance aspect of making music visuals, they are definitely a step in the right direction.




Modern music visuals - ISF


ISF is the modern adaptation of shader visuals. It adds adaptability and flexibility to shaders and allows you to tune your visuals live without using any complicated tools or going too deep into coding.

With ISF you specify a custom user interface that the host application creates for you so that you can easily tune your shaders when performing live. The coding aspect of a shader is still there but things are simplified for the performer to only look at the finished visuals and tweak them as needed.


ISF is now one of the most popular formats for generative visuals with many VJ applications and music visualization platforms online.




Modern music visuals - FFGL


FFGL is essentially a more powerful but slower variant of ISF. It allows you to create even more complex visuals and provides features to interact with the user interface of an application even more closely than ISF. The difference is that this format requires more skill and time to create and modify visuals.


This can be an advantage or disadvantage depending on how you view it. If you want to be able to look inside the code that creates the visuals, then ISF is more likely a better option. But if you want to be completely free of tweaking code, then FFGL is the format for you.


It puts more focus on using a music visualization box that you can just plug and play without having to worry about what goes on inside it.


Formats similar to FFGL are essentially the standard formats for higher end video production applications and modern music visuals.




Modern music visuals - Nodes


Node-based formats like TouchDesigner and Vuo are the easiest and most common music visualization formats today. These formats enable performers to fully separate a performer from what goes on behinds the curtains of software applications. Nodes essentially do what ISF and FFGL aim to do more effectively.


With this format, you only need to be familiar with what a node does on a high level. After that, it is only a matter of designing a node graph that creates the music visuals you need.


Future Evolution


Looking at the progress of making live music visuals, you can see the obvious pattern. The process is slowly moving towards separating role of a content creator from the producer. This is exactly how music production tools evolved.


Power of Creation


In the early days of music production using computers, producers had to have a more in-depth understanding of how audio hardware and tools worked. Now, there are more VST plugins than ever which make music production so much simpler.


What is the secret? Utilizing the power of creation. The aim is to simplify the tools we use so that we could produce the final production as fast as possible. The video production and VJing world is slowly moving in the same direction and VJing tools are gradually becoming simpler and more accessible.




With video formats like ISF, FFGL and especially node-based applications, the idea is to create containers that VJs can just plug in and use on the fly. The next step in the evolution of modern music visualizers is to take the node-based format and simplify it even more so that the creator-performer overlap become completely redundant.


This is already being done by a lot of the major VJ applications and it makes for an exciting future to look forward to.

Related Posts

Oct 21, 2021

Real Time Audio Visualizers