Machine learning Week 8 Marko Řeháček rehacek@mail.muni.cz GENERATIVE DESIGN PROGRAMMING → The numerous eyes tend to follow a singular point almost like a frenzy. This piece was designed to generate a sense of how herd mentality functions. It shows how a singular point can garner extreme attention and judgement from observers. Herd mentality is a state where people can be influenced by their peers to adopt certain behaviours on a largely emotional, rather than rational, basis. INTERACTIVE APP MACHINE LEARNING IN ARTS Magnetic vision Krishnokoli Roychakraborty, 2020 Video, p5 editor GENERATIVE DESIGN PROGRAMMING → In book ‘Nature of Code’, Schiffman explains how fractals can be interpreted as the visual depictions of tree branches, lightning bolts or mountains. This unique visualisation along with the concept of social afforestation was the motivation behind this piece. Like the idea of social afforestation which requires active human participation for forestry this piece’s interactivity lies in proximity of the viewer and the screen. INTERACTIVE APP MACHINE LEARNING IN ARTS Blooming binary trees Krishnokoli Roychakraborty, 2020 Video, p5 editor Check out other examples from Krishnokoli GENERATIVE DESIGN PROGRAMMING → INTERACTIVE APP Infinite Herbarium Caroline Rothwell Using similar technology to that which powers Google Lens, participants are invited to create a plant ‘morph’. Two plants are identified. The visual characteristics of those plants are fed into a ML model that has been trained to generate mutating plant images through exposure to scientific illustration data, made available in the open source Biodiversity Heritage Library. https://infiniteherbarium.withgoogle.com/ MACHINE LEARNING IN ARTS GENERATIVE DESIGN PROGRAMMING → INTERACTIVE APP Scroobly Google & bit.studio, 2020 https://www.scroobly.com/ MACHINE LEARNING IN ARTS GENERATIVE DESIGN PROGRAMMING → INTERACTIVE APP Interplay Mode Google https://experiments.withgoogle.com/interplay-mode/view/ MACHINE LEARNING IN ARTS People watch videos everyday to learn new things. But what if you could do more than just watch? What if there could be interplay between you and the video creator? These prototypes let you interact right alongside the video and practice what you just saw. GENERATIVE DESIGN PROGRAMMING → INTERACTIVE APP, AUDIO Lots of other cool examples on music: https://codepen.io/teropa/pens/public MACHINE LEARNING IN ARTS Robot Neil’s Bubble Bath Tero Parviainen (Teropa), 2020 GENERATIVE DESIGN PROGRAMMING → Scribbling Speech Xinyue Yang, 2018 xinyue.de/scribbling-speech Language and images are closely intertwined: We think in pictures and we explain facts as spatial constellations. What if the spoken word could be transformed into dynamic visual worlds in real time? Speech input, machine learning and recurrent neural networks for image generation allow to computer generate complex imaginary worlds that follow the narrator and thus create complex animations controlled by linguistic structures. (Bc. thesis) MACHINE LEARNING IN ARTS GENERATIVE DESIGN PROGRAMMING Other examples Let’s read a story – The project takes the corpus of Aesop fables and investigates the possibility of exploring the connections between different characters and ideas from the original fables in a new and fun way using machine learning language models. AR body filters Quick, draw! – Google GENERATIVE DESIGN PROGRAMMING → www.generativedesign.cz/archive/unstopeppable All good things have an end. Thankfully, computer-generated Peppa Pig cartoons are not a good thing, and also have no end. They make just enough sense to be uncanny, and just enough nonsense to be on the edge between 'dark' and 'hilarious'. And what is more, you cannot stop them — they're unstoPEPPAble. A text-generating neural network (trained on the whole of the Internet), a state-of-the-art set of TTS voices (meant to help the visually impaired) and a bunch of video and image editing tools are misused with one goal in mind: destroying your childhood. VIDEO unstoPEPPAble Jan Pokorný, 2021 GENERATIVE DESIGN PROGRAMMING Basic terminology ML – machine learning model – machine learning program training pre-trained models transfer-learning GENERATIVE DESIGN PROGRAMMING https://ml5js.org/ MACHINE LEARNING IN ARTS GENERATIVE DESIGN PROGRAMMINGML5.JS GENERATIVE DESIGN PROGRAMMING Getting output from webcam: https://editor.p5js.org/mrehacek/sketches/3_nNWmIIe PoseNet: detecting person (skeleton tracking) https://editor.p5js.org/mrehacek/sketches/RIP1tEMY2 MobileNet: detect objects in images https://editor.p5js.org/mrehacek/sketches/xISNXPZ5L Example sketches MACHINE LEARNING IN ARTS GENERATIVE DESIGN PROGRAMMING Teachable Machine (transfer learning – create own model) https://teachablemachine.withgoogle.com/ MACHINE LEARNING IN ARTS GENERATIVE DESIGN PROGRAMMING All resources MACHINE LEARNING IN ARTS 1. ml5.js – library for p5.js to start playing with ML 2. Experiments with Google examples 3. Coding Train: Beginner's Guide to Machine Learning in JavaScript with ml5.js – video series about all 4. ML x Art examples 5. ML4A complex examples 6. Awesome-AI-art github – list of all software, people, etc. regarding AI in arts 7. NVIDIA examples 8. RunwayML – software with more “complex” models