Idea
The Shape of Sound: A Practice-Led Study in Creative Human-Computer Interaction
I want to make a simple but creative music production web app which will convert shapes and colours into sound. for example, synth properties controlled by properties of a shape (roundness, area, colour).
Format: loop maker - see splice beatmaker
This would be more vertical, and fill the screen.
Each occupied cell indicates a sound, and the beat will play from left to right.
The user will be able to select a BPM.
They will be able to drag and drop preset shapes (circle, triangle, square - these will correspond to sine waves, saw wave and square wave). from there, the user can customise each individual shape with the following properties:
- roundness (the more round, the closer to a sine wave it is)
- fuzziness (find some quality of a synth sound which corresponds to how fuzzy something is)
- colour (pitch?)
- more to come
Visual properties
| Visual property | Initial idea | Expansion | Research angle |
|---|---|---|---|
| Shape type | Circle / Square / Triangle / Right-angle triangle Sine / Square / Triangle / Sawtooth | Fundamental building blocks of synthesis. | |
| Roundness | More round = closer to a sine wave | Could morph the waveform itself | Timbre and harmonic content - how does the perception of “roundness” map to “smoothness” in sound? |
| Fuzziness | Distortion effect or filter resonance | Explore textual qualities in both visual and auditory domains (synesthesia) | |
| Colour (Hue) | Pitch (0-360 mapped to musical scale) | Select scale (start with just C Major) | Relation between colour and musical notes - Scriabin’s clavier à lumières |
| Colour (Lightness) | Volume (amplitude) | ||
| Colour (Saturation) | Low pass filter cut-off (Saturated = bright, clear, Desaturated = muted, duller sound) | Textural link - how does visual vibrancy translate to sonic brightness? | |
| Size / area | Sustain (ADSR envelope) | ||
| Rotation | Panning (L/R) |
Colour by chromatic scale:
| Note | Colour |
|---|---|
| C | red |
| C# | violet |
| D | yellow |
| D# | dark blue (slightly purply) “steel” |
| E | light gray |
| F | dark red |
| F# | light blue |
| G | orange |
| G# | pink |
| A | green |
| A# | dark grey |
| B | blue (#0000FF) |
| ![[scriabin keyboard.png | 250]] |
Tech stack
| Component | Technology | Justification |
|---|---|---|
| Frontend framework | Next.js (React) | I have years of experience with react and have been working as a Junior Software Developer using primarily React for 7 months. |
| Graphics / canvas | Three.js | For rendering shapes and allowing transitions between shape properties for a smooth responsive feel. |
| Audio engine | Tone.js | A framework built on the native Web Audio API. Professional standard for web-based audio applications. Has a sequencer, synths and basic effects already, allowing me to focus on mapping shape properties to manipulate synths and effects. |
| State management | Zustand | Will be used for BPM, playback state, individual shape properties, tracks, etc. These will all be updated regularly and reflected in the UI, so state management is needed. |
| UI & styling | Tailwind CSS & Framer Motion | Tailwind provides utility classes for styling quick elements. With the whole website likely having only 1 page, there is no need for individual “home.css”, “profile.css”, “settings.css” stylesheets. Tailwind makes styling a single page much more elegant. I will use framer motion to create animations for drag-and-drop and UI interactions. |
| Deployment | Vercel | The most optimal platform for hosting Next.js applications. Very seamless deployment through GitHub, and I have been using it for several years |
Tone.js
808 example
const osc = new Tone.Oscillator().toDestination();
// start at "C4" to give the impression of impact
osc.frequency.value = "C4";
// ramp down to "C2" almost immediately
osc.frequency.rampTo("C2", 0.05);
// 6 seconds
osc.start().stop("+6")