Art and A.I. — A Vision of Where It’s Going

The Github Archive Program is meant to preserve software, and so it only makes sense that I would use A.I. in the process of creating the artwork for it.

I’ve been playing with A.I. artistically for years now. The following is as best of an account I can give of what the process is like, what’s new about it, and where I think it’s going.

Context, Philosophy, and an Imaginary Helicopter

As a lifelong student of human nature, I also treat the journey (as I do everything) as an invitation to observe the evolution in my own relationships with body, mind, spirit, and machines.

When working on a piece, I often experience little flashes of surrealism — this sort of awareness, that some small code/art decision I’m dealing with on my laptop, viewed through another lens, is a real-life instance of a philosophical flashpoint that humanity has fiercely debated for ages.

Trying to answer the philosophical questions, I usually feel like a monkey trying to fly a helicopter built out of leaves and sticks, for which the operating manual and engineering diagrams are left as an exercise to the pilot.

That is to say, a lot of the popularized philosophy and the actual technology are in completely different languages. The translation is often poor or nonexistent — a lot of the philosophy was born before we had technical knowledge to base it on.

Helicopter operating diagram (Original artwork: Kandinsky)

The dialogues I read about A.I. and art often center around questions such as:

  • Is A.I. art really art?
  • Can a machine be creative?
  • Will A.I. make artists obsolete?

I’ve come to view a lot of these questions as bordering on koans — they’re not koans by any strict definition, but the more you work with A.I. creatively, the more you realize that a lot of how these questions are phrased is structurally disconnected from the underlying reality of how A.I. and art actually work. At least the way I see it.

The remaining points are a loose set of axioms that I’ve found to be true in my explorations. It is also a vision of what I believe A.I. art has the potential to evolve into.

I’ll also add that I use the term “A.I.” more broadly than many computer scientists are probably comfortable with —you can safely substitute “neural network” in place of “A.I.” in most places in this article where I am describing my own experience. As an artist I don’t like drawing strict lines about these things — everything is a canvas. It all works together.

  1. The Paintbrush Thinketh. Sort of.

This is, to me, the crux of what’s new.

Compared to something like traditional painting — in which the medium is a tool and canvas used to directly transpose an artist’s vision — A.I. as a paintbrush has a mind of its’ own, and that’s a major part of the process. When I am painting with neural networks, I am constantly dancing with how my neural networks (I like the phrase “thinking paintbrushes”) will interpret and transform the imagery that I give it.

To extend the painting metaphor — it’s like whispering carefully worded suggestions into a microphone connected to a guitarist’s distortion pedals, into a computer mind guiding a maelstrom of etheric hands that are painting my picture, usually without having any idea what they’re doing.

I operate by cleverly experimenting and feeding and tuning my artistic medium — not by directly controlling it. What happens if I tell my machines to paint a bear but give them a picture of a unicorn? Let’s find out!

To be clear — the paintbrush is not “thinking” in the human sense of thinking. But it’s doing things with my input that are so complex that it is beyond my ability to deterministically create the outcome. Some of that is in the A.I.’s court. As the technology advances, the complexity and sophistication of this “thinking” grows.

2. The Thinking Paintbrush Itself is an Artistic Medium

It’s important to understand that when I tell an A.I., “paint me a picture of a dolphin doing a backflip over a boat” (side note — we’re not quite there yet), the actual algorithm that is creating that image is still a product and a reflection of human consciousness, expressed through a lot of research and mathematical experimentation. It can be tweaked, tuned, trained, manipulated, re-weighted, and adjusted in an infinity of ways. The form that I am using it in is merely one manifestation of that infinite space.

In other words, as the artist, you have the power to alter how your paintbrush thinks to the degree that you have the ability to modify and tune it. And to the degree of creativity with which you train it, and combine it with other tools — thinking or nonthinking. This tuning and training and combining is, in and of itself, a creative process. Code is art. Infrastructure becomes art.

Nowadays, when I’m walking around and I see an interesting pattern in some flowers, or some pretty colored rocks, or an object with an interesting texture, I pull out my phone and take a picture, because I know I can feed it to my A.I. paintbrush and I might be able to make something interesting with it later. Sometimes I’ll arrange objects or Photoshop things a little based on what I know I can tune a neural network to grab onto.

Succulents. I saw these walking through Berkeley and had to snap a photo.
Plums
Succulents, painted with plums
Butterfly and DNA background letters, painted with succulents. The grayish tint in a lot of the background is from the neural network feeling mathematically ambiguous about how to map the succulent texture onto the featureless white background behind the letters. It’s alright, buddy. We all have those days.

Sometimes, I just take random objects and see what I can get the A.I. to transform them into.

Approached in this way, A.I. tools can add nearly effortless magical augmentations to “default reality”.

A piece of fluff that I found at Burning Man
This is what happens when you bring things home from Burning Man

3. The Thinking Paintbrush is only A.I. Art’s “Hello World”.

Up until now, I’ve stuck to static visual art as an example art form, but when we remember that the thinking paintbrush is an art in and of itself, we open the question of what other forms of art we can create — or what other technical or nontechnical mediums we can extend our paintbrushes into. The landscape is more or less limitless.

Especially because…

4. Human — A.I. Art is Way More Interesting than Pure A.I. Art.

This is where the future is.

A.I. is not some omnipotent overlord magic bullet singularity simulation cat video robot deity. It’s a bunch of computers doing math.

There are things it’s great at. There are things it produces that defy human logic in uncanny, brilliant, sometimes disturbing ways. There are also things that it’s terrible at, and there are also many places in art where human presence simply can not be substituted by a machine without killing the art. The most potent magic is in knowing what those territories are, and then in getting creative in how you structure your relationship with them as an artist, or as a collective of artists.

In my own art — I often will spend time feeding increasingly strange inputs to A.I. to see how it surprises me in what it puts out (and to pick up on the subtleties of how it processes them). Sometimes this will give me ideas or perspectives, that later develop into art pieces. Sometimes the only useful thing the A.I. did was gave me an idea.

Sometimes I need a couple days to get my mind past something disturbing that my A.I. created, because it was a computer doing math that didn’t know what it was creating.

My approach to visual A.I. art is hybrid. I seldom work in the realm of pure generative art — often my pieces will involve me arranging in-person aesthetics, taking a photo, running it through multiple neural networks, and then combining the outputs in photoshop. And then maybe giving that to a neural network to see what comes out of it. Ad infinitum, until I’m satisfied with it, or until my flow loses coherence and I declare it an experiment.

Driftwood, flower, peacock feather
Sky deity. The neural effects were light on this one, but they were enough to turn a piece of driftwood into something that looks it’s nursing some feels about not being in a Studio Ghibli film.
Before A.I. effects. I did this photoshoot with a variety of objects, intentionally chosen because I’d experimented with photographing them and running them through neural nets before, and I knew I could make them come out interesting. In my right hand is a ball of messily coiled black aluminum wire, in my left is a faux vine and the same little ball of magic Burning Man fluff from before.
After. The intention behind this particular photoshoot was an exploration into Jung’s Magician archetype, and the words “Spirit and Glitch”.

When you wander from the world of visual art and into the world of performance, this relationship between artists and machines becomes much more interesting. The A.I. can become part of the performance (generative video, responsive/aware environment) — or, it can become a backbone coordinating artists and/or machines.

There are artists in some of my communities who are beginning to create alternate reality “films”, storylines that break the traditional “fourth wall” in theater, and that then play out, with audience participation, in real time on platforms like Facebook and Instagram. This gets especially interesting with things like deepfakes in the mix. It’s like flashmobs, online.

You can use A.I. in an installation art piece to “read” a room, like a person or an animal would intuitively notice things. Are people excited? Are people asleep? Are people dancing? Are people being loud? And then you program the art (say, a room-wide ecosystem of LED flowers and plants) to act and vibe accordingly. You can literally make whole rooms and spaces feel like they’re alive and breathing if you do it right. And then that influences how people feel in the space. It’s pure magic.

I think a lot of people tend to think of “A.I.” as a whole brain — the practical reality is that what we call “A.I.” today is more like a scattering of thin slices of a few layers of neurons that serve some specific purpose, such as interpreting lines and shapes in front of your eyes, or gauging sentiment in someone’s voice. A major component of creating great interactive/installation A.I. art is knowing how to use these layers in elegant ways, and not overdoing it.

There is immense creative potential here. It also goes without saying that there is immense destructive potential, wielded carelessly.

The instances in which I’ve seen people respond well to this and enjoy it have generally been when the experience is being created by someone they know and trust, when there is a high degree of care for the audience’s feelings and sense of safety, and when there is a clear gate through which they can consent to the experience and through which they can easily leave it.

(I’m intentionally staying far from politics, but consider how much A.I. is on the backend of political/internet advertising right now, ask whether anything in the above paragraph is true of that situation, and consider how the whole ecosystem makes people feel)

5. Large Scale Corporate Technical Landscapes Are Going To Change.

A.I. is going to dramatically transform corporate creative industry.

In the near term future, I think that it will radically alter graphic design and film production. A.I. is already producing tools that replace hours of human Photoshop labor with seconds of GPU time. Give it a little time, and a few large-scale iterations, and it will radically alter job markets, marketable vs obsolete skills, and company/team structures for art and film production.

It will also enable smaller players to produce things that would have been out of resource range for them before. Options open up to creative people when manual processes that used to take weeks turn into automatic processes that take minutes, with maybe a light editing pass before and after.

In the further term future, this trend continues as far as the tools can reliably replicate manual human effort for faster and cheaper. The tools and the human workflows also co-evolve, in much less predictable ways. There’s a lot of discussion to be had about this — it’s an enormous topic, for another post.

6. Traditional Art is Not Going Anywhere.

My appreciation for live, “organic” art — witnessing live music, viewing (or creating) a painting, seeing and participating in theater (I especially love immersive theater), dancing — has actually grown much deeper from spending time in A.I. land.

There is a quality to completely human art, that not only can’t be replicated by A.I. — it is inherently destroyed in the attempt. The more time you spend around machines, the more their absence becomes fresh air. You become acutely aware of how nourishing it is to take in direct human expression.

It’s not a matter of technique — it’s a matter of knowing that the art I am experiencing is a distilled essence of someone’s human experience. Especially if I know that person. There is something special and uniquely human about that space that is not the purview of algorithms to replace.

When I examine the fears that I have around A.I. replacing human artists, the reason that those fears exist is that I love human art and I don’t want something else to replace it. I’m an A.I. artist, and *I* don’t want A.I. to replace human artists. I don’t think anyone does. Knowing that brings me some degree of comfort.

And with that, I will sharpen my point about A.I. and humans a bit:

7. The A.I. Does Not Replace the Artist.

and…

8. In a Supporting Role, A.I. Opens Portals to New Worlds.

The point of art isn’t to be technically impressive — the point of art is to make us feel things. To cut through the noise of a senseless, numb world. To connect us. To comfort the disturbed, and to disturb the comfortable. To transcend worlds and bridge conflicting realities through currents of sameness that run deeper than our differences. To share the experience of being human with each other.

To share the experience of being human with each other.

No matter how good it is — I don’t trust a machine to share the experience of being human with me, because no machine has ever had the experience of being a human.

(The experience of analyzing emotions and facial expressions in millions of hours of human footage is the experience of analyzing millions of hours of footage, not the experience of being human.)

On some very deep level, I only trust a human to share that experience. That human should wield whatever tools best enable them to do that. For some, that tool may be a paintbrush. Or their voice. For others, a fleet of machines creating a mind-blowing orchestra of shapeshifting color and narrative around me.

Life is performance art. Technology is the assistant and sometimes the medium.

The danger in technically sophisticated forms of expression, is that the moment that I feel like the machines are telling the story instead of the artist, my openness to the experience vanishes. I sense abdication. I stop caring.

Worse, if I feel like the A.I. is being used to manipulate me, that openness turns to hostility.

9. Trust with A.I. is Everything.

For all of its’ power, adding A.I. adds a surprising fragility to art that leverages it, in that it is often tempting to misuse it in a way that compromises your audience’s trust — whether by intent, or through simply getting carried away in the novelty and forgetting that you are telling a story to a human being. I have done this plenty of times without intending to. It never feels good.

The power of that technical complexity, wielded skillfully, is that you can tell deeper stories to more people in more immersive, complex, awe-inspiring ways. If your art is compelling, your audience will love your worlds and stories and weave them into their own consciousness. Awe is a powerful thing.

This is an honor, and to some degree this sharing and evolution of stories is why art exists, but with A.I. in the mix it is especially important to understand that the more powerful your creative medium, the more imperative it is to be truthful in your expression. The more important it is to give your audiences stories and experiences that serve them. Your creations will echo through peoples’ thoughts, feelings, and lived experiences. Don’t feed your audience garbage. Nourish them.

10. It is always our responsibility to create in ways that respect our own humanity, and the humanity of our audience.

Our art is a reflection of our nature.

Our technical prowess means that we are dancing ever more consequentially with our own patterns — whether or not we are aware of them. When we invest in our mastery of our tools and ignore our relationship with our internal worlds, we invite the unprocessed lessons of our nature to show up at scale.

Our growing technical abilities give us more power than ever to share those lessons as we learn them. They also give us more responsibility than ever to remember that we are human, and that those who consume our art are human.

However impressive our means — whether paint, spoken word, immersive theater, or A.I. orchestra — at the end of the day, we are still sitting around the campfire, telling each other stories.

Artist, Communityist, Systems Thinker, Multi-Competency Engineer alexmakijokela.com

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store