How children inspired my painting robots
Seeing how simple my young children's art was, I wondered if I could teach their creative process to a painting robot. My attempts, failures and successes led to a 15-year journey of artistic discovery that gave me profound insights into my own mind, including how my creative process worked.
The artist Paul Klee once described the creative process as a feedback loop. The painter makes a mark then steps back to evaluate the mark before making the next mark. This is done repeatedly as the effect of each stroke provides feedback for how and where to apply the next stroke. Shapes, ideas, and themes emerge as the complexity of the painting increases. These abstract concepts are in turn recognized by the artist and influences the direction of the artwork. According to Klee, it is in this exact manner that an artist creates something from nothing. The creative process is a self-referential response to the act of creation.
The creative process Klee describes is innate to humans and appears at an early age. Anyone who has been around children has observed it. As soon as a child realizes that crayons can be used to make marks, they will begin drawing on just about anything they can reach. Having four children there is evidence of this all over their toys, as well as on most of my walls and furniture.
In a more specific example, I have seen my children using creative feedback loops as early as two years old. Place a blank piece of paper in front of almost any child, and they will begin making marks. As areas of the page fill, they will begin drawing in the empty areas. This is perhaps the most basic example of the process described by Klee. Marks are made, observed, and used to direct where future marks are made.
My favourite part of waking up is walking into my studio and seeing what my robots have been painting all night. | © Pindar Van Arman
Step 1) Find a blank area in the composition.
Step 2) Put marks in that area.
Step 3) If blank areas remains, go back to Step 1.
Step 4) Once all the blank areas are filled, stop.
As I watched my children develop, I noticed that many of the processes behind their early creations had simple algorithms behind them. They sometimes chose the colours based on nothing more that the proximity of a crayon to their hand. Once they learned a basic shape, they would fill every blank space on a page with it. Even though their techniques were elementary, I had little doubt that they were being creative, and this led me to a realization that has shaped the last fifteen years of my art.
I realized that if I could teach a painting robot some of these same rudimentary techniques, the machines would be creative. To argue that a robot using the exact same creative process as a child was not creative, would be like arguing that children were not creative, when their creations most certainly were. To be clear and head off any hate mail, of which I receive a fair amount, I am not asserting that machines can make art. That is a different subject entirely. I am merely focused on creativity.
"Corinne": An artwork created in a collaboration between a robot and my 5 year old daughter | © Pindar Van Arman
Replicating simple creative algorithms became the focus of my early painting robots. I taught them the horror vacui algorithm described earlier. Seeing how my kids would colour even though they only had a couple crayons, I wrote algorithms for the robots to paint with a limited palette. Over the years these algorithms became more complex and I soon found myself implementing routines that I used as a seasoned painter. I taught the machines to recognize balanced compositions, increase contrast, and paint with complementary colours, among others. While things began simple, the robots were soon using their artificial intelligence to paint portraits with the sophistication of a high school student.
It was about five years ago that I discovered machine learning and added them to my robot’s repertoire. I taught them an algorithm known as Style Transfer so that they could replicate artistic styles. I added Generative Adversarial Networks so that they could imagine unique faces. They were soon using more than two dozen creative algorithms of varying sophistication to complete each painting.
This collection of algorithms led me to my most recent artistic discovery. I realized that while my machines were not intelligent, they did in fact have something that resembled a creative mind. Using feedback loops, the robots were continually taking a picture of the canvas, analysing it, and asking themselves how to complete the next set of brushstrokes. This approach closely resembled a popular theory of mind proposed by Marvin Minsky in his seminal book “Society of Minds.” While it remains a theory, Minsky proposed that our minds are not a single super intelligence, but rather a collection of smaller intelligent agents that surface as needed and compete to solve problems. Painting portraits is a problem just like any other, and Minsky’s approach is effective at solving it. With each feedback loop, the robots asked their AI how to best solve the problem of making the portrait more beautiful. In each artwork, their society of creative minds competed for control of the brush as they completed the painting one brushstroke at a time.
What began as an inquiry into the nature of my children’s creativity, has grown into 15-year project of self-discovery. By trying to teach naïve computers my own artistic process, I have gained insights into myself, as well as the nature of my own mind.
My art is not the paintings created by my AI. The paintings are merely a record of the progress that I am making towards my ultimate goal, which is the complete deconstruction and codification of my creative process. Oddly, I feel no closer to an understanding than I when I began fifteen years ago. Despite this I look forward to continuing this journey and following it wherever it leads.
Learn more about Pindar Van Arman's views on the future of creative AI here.
The creative process Klee describes is innate to humans and appears at an early age. Anyone who has been around children has observed it. As soon as a child realizes that crayons can be used to make marks, they will begin drawing on just about anything they can reach. Having four children there is evidence of this all over their toys, as well as on most of my walls and furniture.
In a more specific example, I have seen my children using creative feedback loops as early as two years old. Place a blank piece of paper in front of almost any child, and they will begin making marks. As areas of the page fill, they will begin drawing in the empty areas. This is perhaps the most basic example of the process described by Klee. Marks are made, observed, and used to direct where future marks are made.
My favourite part of waking up is walking into my studio and seeing what my robots have been painting all night. | © Pindar Van Arman
The urge to fill empty space
While this is an unsophisticated act, it is creative, and even has a name in the artistic lexicon. The urge to fill empty space with detail is known as horror vacui. Composition is created by finding an empty space and filling it with something, anything. This is repeated in a loop until marks fill the page. It is such a simple pattern that one can think of it as an algorithm.Step 1) Find a blank area in the composition.
Step 2) Put marks in that area.
Step 3) If blank areas remains, go back to Step 1.
Step 4) Once all the blank areas are filled, stop.
As I watched my children develop, I noticed that many of the processes behind their early creations had simple algorithms behind them. They sometimes chose the colours based on nothing more that the proximity of a crayon to their hand. Once they learned a basic shape, they would fill every blank space on a page with it. Even though their techniques were elementary, I had little doubt that they were being creative, and this led me to a realization that has shaped the last fifteen years of my art.
I realized that if I could teach a painting robot some of these same rudimentary techniques, the machines would be creative. To argue that a robot using the exact same creative process as a child was not creative, would be like arguing that children were not creative, when their creations most certainly were. To be clear and head off any hate mail, of which I receive a fair amount, I am not asserting that machines can make art. That is a different subject entirely. I am merely focused on creativity.
"Corinne": An artwork created in a collaboration between a robot and my 5 year old daughter | © Pindar Van Arman
Adding to the skillset
Replicating simple creative algorithms became the focus of my early painting robots. I taught them the horror vacui algorithm described earlier. Seeing how my kids would colour even though they only had a couple crayons, I wrote algorithms for the robots to paint with a limited palette. Over the years these algorithms became more complex and I soon found myself implementing routines that I used as a seasoned painter. I taught the machines to recognize balanced compositions, increase contrast, and paint with complementary colours, among others. While things began simple, the robots were soon using their artificial intelligence to paint portraits with the sophistication of a high school student.
It was about five years ago that I discovered machine learning and added them to my robot’s repertoire. I taught them an algorithm known as Style Transfer so that they could replicate artistic styles. I added Generative Adversarial Networks so that they could imagine unique faces. They were soon using more than two dozen creative algorithms of varying sophistication to complete each painting.
This collection of algorithms led me to my most recent artistic discovery. I realized that while my machines were not intelligent, they did in fact have something that resembled a creative mind. Using feedback loops, the robots were continually taking a picture of the canvas, analysing it, and asking themselves how to complete the next set of brushstrokes. This approach closely resembled a popular theory of mind proposed by Marvin Minsky in his seminal book “Society of Minds.” While it remains a theory, Minsky proposed that our minds are not a single super intelligence, but rather a collection of smaller intelligent agents that surface as needed and compete to solve problems. Painting portraits is a problem just like any other, and Minsky’s approach is effective at solving it. With each feedback loop, the robots asked their AI how to best solve the problem of making the portrait more beautiful. In each artwork, their society of creative minds competed for control of the brush as they completed the painting one brushstroke at a time.
What began as an inquiry into the nature of my children’s creativity, has grown into 15-year project of self-discovery. By trying to teach naïve computers my own artistic process, I have gained insights into myself, as well as the nature of my own mind.
My art is not the paintings created by my AI. The paintings are merely a record of the progress that I am making towards my ultimate goal, which is the complete deconstruction and codification of my creative process. Oddly, I feel no closer to an understanding than I when I began fifteen years ago. Despite this I look forward to continuing this journey and following it wherever it leads.
Learn more about Pindar Van Arman's views on the future of creative AI here.