Skip to content

06. Visualization Configuration

Jump to bottom
Jon McGuire edited this page Aug 6, 2023 · 23 revisions

The visualization configuration tells monkey-hi-hat what kind of data to send to the shaders, what kind of audio textures are used (if any), and where to find the vertex and shader files.

The [shader] Section

  • Description A line of text with the name of the visualization, and anything else you'd like to add such as the author's name or a URL (remember, no hashtags, that signifies the beginning of a comment). It's possible a future GUI control app will read and show these descriptions.

  • VisualizerTypeName This controls how data is fed to the shaders. Eventually plugins will be added, but currently only the following two built-in types are supported: VisualizerFragmentQuad and VisualizerVertexIntegerArray. See "Creating Visualizations" for more information about these.

  • VertexShaderFilename and FragmentShaderFilename are the names of the vert and frag shaders. They must be in the same directory as the .conf file.

  • BackgroundFloatRGB is the background color the buffer is cleared to before each frame is rendered. This is more important with VisualizerVertexIntegerArray because the output doesn't typically cover the entire screen, unlike VisualizerFragmentQuad which is full-coverage by definition. As the name suggests, the red, green, and blue color components must be normalized floats (values 0 to 1).

The [VisualizerVertexIntegerArray] Section

Visualizer Types can have their own custom settings. The section name matches the Visualizer Type Name.

Currently only VisualizerVertexIntegerArray has additional settings:

  • VertexIntegerCount defines the size and content of the integer array passed to the vertex shader. This must be a 32-bit integer value, so the visualizer can request one thousand, ten thousand, a million, or any other number of values. Typically, very high values aren't that useful, and the VertexShaderArt website limits users to a maximum value of 100,000.

  • ArrayDrawingMode defines how the output vertices are drawn. These are defined by OpenGL and include the following settings:

    • Points
    • Lines
    • LineStrip
    • LineLoop
    • Triangles
    • TriangleStrip
    • TriangleFan

The [AudioTextures] Section

If the visualizer is audio-responsive, this section lists the audio textures that will be used by the shaders.

The format of each entry is: texture_unit = uniform_name texture_type

OpenGL Texture Units are just a number -- start with zero and add one for each audio texture. The uniform name is how the texture is identified in the list of shader inputs. If you're using just one, probably "sound" is fine. VertexShader art defines several (volume, sound, and floatSound), and Shadertoy varies according to the input channel (iChannel0 through iChannel3). Finally, texture_type is the name of the eyecandy texture data that is required.

An example is: 0=sound AudioTextureWebAudioHistory

This assigns the uniform to TextureUnit zero, declares the uniform name as sound, and receives audio data processed by eyecandy's AudioTextureWebAudioHistory texture generator.

See "Understanding Audio Textures" for more details, but the currently available types are:

  • AudioTextureVolumeHistory
  • AudioTextureWaveHistory
  • AudioTextureFrequencyMagnitudeHistory
  • AudioTextureFrequencyDecibelHistory
  • AudioTextureShadertoy
  • AudioTexture4ChannelHistory
  • AudioTextureWebAudioHistory

The [AudioTextureMultipliers] section

This optional section allows you to assign a float multiplier to boost or cut the audio texture data, assuming the eyecandy type supports multipliers. For example, AudioTextureShadertoy actually processes two types of data in one texture. The PCM Wave data ignores the multiplier, but the multiplier is applied to the Web Audio data. All audio texture types default to a 1.0 multiplier.

The format is: texture_unit = multiplier

Examples

Here is a visualizer that reproduces the FFT Frequency Magnitude display from the eyecandy repository's History demo. This illustrates a few important points. This particular visualization needs an audio texture. The type of visualizer is similar to Shadertoy.com, which means most of the work is done in the fragment shader (or pixel shader, as Shadertoy often calls it). Thus, a default vertex shader is used, although that isn't required by any means; you can write custom vertex and fragment shaders, if you wish.

eyecandy_demo_history_freqmag.conf

[shader]
Description=Eyecandy Demo: FFT Frequency Magnitude History
VisualizerTypeName=VisualizerFragmentQuad

VertexShaderFilename=VisualizerFragmentQuad.vert
FragmentShaderFilename=eyecandy_demo_history.frag

[audiotextures]
0=sound AudioTextureFrequencyMagnitudeHistory

[audiotexturemultipliers]
0=5.0

VisualizerFragmentQuad.vert

#version 320 es

// Normally the primary processing for VisualizerFragmentQuad happens
// in the frag shader and this default vert shader always applies.

layout(location = 0) in vec3 vertices;
layout(location = 1) in vec2 vertexTexCoords;
out vec2 fragCoord;

void main(void)
{
    fragCoord = vertexTexCoords;
    gl_Position = vec4(vertices, 1.0);
}

eyecandy_demo_history.frag

#version 320 es
precision highp float;

in vec2 fragCoord;
uniform sampler2D sound;
out vec4 fragColor;

void main()
{
    fragColor = texture(sound, fragCoord);
}
Clone this wiki locally