Text to Art: Using NLP to Create Art

10/03/2022
blog

From finance and marketing to higher education, AI has taken over the entire world. The world of creators and artists is no different. If someone told you a few years ago that you can convert text to art, you would probably call them mental and laugh off the idea. Today, however, it is a reality that has opened doors for tons of benefits for the artist community.

The idea has become immensely popular due to the appearance of NFTs in the market. NFTs have started an era of digital artwork, allowing the artists to tokenize and sell their work to a quickly growing marketplace. For instance, Art AI has recently introduced Eponym, a software that allows text to art conversion, and direct creation of NFTs.

Since digital artwork is the new tomorrow, we have decided to explore the concepts of converting text to art by using NLP (Natural Language Processing). So, let’s begin, shall we?

What is Natural Language Processing (NLP)?

Natural Language Processing is a branch of Artificial Intelligence (AI) that allows computers and machines to understand texts and spoken words. NLP allows machines to understand the actual meaning of the data, including sarcasm, sentiment, and intent.

With NLP, humans can communicate with the computers in their intrinsic language, and by using statistical, machine learning, and deep learning models, NLP interprets the command and produces the results.

While the concept may sound too complex, you will be surprised to find out that you have encountered NLP several times in your life, in fact, in your day today. Did you use voice-operated GPS while going to work today? Or did you need Google Translate to find out what your colleague was telling you in his native language?  From virtual agents, like Siri and Alexa, to customer service chat boxes that pop up when you visit your favourite websites, NLP is everywhere!

Why is NLP being used to Convert Text to Art?

Artists and creators are known to be unique personalities that have god-gifted methods of expressing themselves. On the other hand, AI is a highly logical concept with no space for emotion whatsoever. Combining the both might seem illogical, yet it is a concept being widely accepted due to the advent of NFTs.

Don’t know about NFTs? Visit Vidalgo, where we talk about NFTs, their market growth, trends in this field and much more!

Click on the animation for more options

NFTs have converted artwork into a source of money. And like any other money-generating source, people are racing to double it. While traditional artwork might require days to create, an NLP-based artwork can take minutes, if not seconds. It has also opened doors for non-artists to enjoy the OpenSea marketplace. You no longer need to be a professional to earn from NFTs; you just have to have a thought and voila! –you have your art.

But What about Creativity?

Many people see the use of AI in creation as a threat to the creators. While the artist community is still an underdeveloped and under-appreciated part of the world, there should be no doubt about the fact that, like any other community, they need to keep up with the world.

According to Forbes, “it is becoming harder and harder to deny that artificial intelligence is capable of creativity.” Yet there is no doubt that this creativity stems from the creativity of a human being himself.

NLP-based artwork can never take away an artist’s job because an artist is an initiator for it. His creativity is the only thing such software can display. Without his vision, NLP-based art is nothing. Furthermore, a human is needed to remove racism, hate or inappropriate elements from the artwork that logic-based software may generate.

DALL-E –A NLP Project by OpenAI

DALL-E is an extension of GPT-3, which was a language model that created human-like text. The results of GPT-3 make text written by humans and machines indistinguishable. Working in the same line, OpenAI introduced DALL-E, which can not only generate an original image but can also extend an image towards the bottom if commanded. Just like GPT-3, DALL-E is a transformer. A transformer uses the attention mechanism, giving different levels of importance to different parts of inputs.

The word DALL-E is the scientists’ way of giving tribute to the Spanish artist Salvador Dali and Pixar’s beloved 2008 robot, WALL-E.  From the looks of it, DALL-E can interpret commands that are far from reality, for instance, “an arm-chair in the shape of an avocado.”

Text to Art: avocado-shaped arm chair
Image Courtesy: Open AI

The fact that this AI can combine entirely unrelated textual commands and convert them into an art piece can make anyone dumb-founded, and that’s what scientists at OpenAI feel like. Since the field is quite unexplored, their creation has several limitations. The software creates a different image or piece of digital art every time the user rephrases the same command. However, the breakthrough is big enough for anyone to ponder too much on the limitations for now.

How does DALL-E convert text to art?

DALL-E requires user input in the form of text and an image input from a program called CLIP. DALL-E divides the input into tokens. For a human being, each letter of the alphabet is a token. For DALL-E, there are 256 BPE-encoded tokens and 1024 image tokens. When a stream of input goes into the program, it divides it into a maximum of 1028 tokens. It then tries to generate the details present in the maximum number of tokens individually.

In this way, the program tries its best to create the maximum number of details mentioned in the input caption. However, it does not always interpret the text correctly; for instance, it cannot create shapes such as a pentagon. Also, it has trouble creating a larger object sitting on a smaller one or correctly understanding the words “standing left off”, “standing below”, and of the likes.

Similarly, the program can create word strings. However, the longer the string, the lesser will be its accuracy. The program also has trouble distinguishing between colours, but don’t we all?

The Potential of NLP and AI Artwork on the Future

With DALL-E and its successors, artists of every kind will be able to fare better. They can save their time and supplies. Corrections to their artworks will be easier, and brainstorming will be a piece of cake.

Fashion designers no longer need to rely on digital or pencil sketches, and they can just tell the program how many flares the skirt should have. Logo designers no longer need to put an effort to place their logos on several cards and shop fronts to check how they look, and they can tell the program to create their design with multiple backgrounds.  

Similarly, game designers can create their otherworld characters and see exactly what kind of beak fits in the place of a nose. The possibilities are endless! So, are you ready to explore them?

Visit our blog for more information about machine learning and deep learning in animation and artwork.

Other Posts

blog
Best Lottie Players, Plugins and Tools
30/04/2022
Lottie has revolutionized the world of animation. To give a very basic definition, Lottie is a file format specific for animations, much like GIF. Due to Lottie animations, users can now display high-quality animations without any problems, such as pixelation, large file size, or slow webpage loading speeds. A single Lottie file is many times […]
blog
Infographics Trends and Animated Infographics in 2022
23/04/2022
With the rise of social media usage, visual content is becoming more and more popular than mere texts. From informational videos to sending memes to your friends and family, visual content is what people love nowadays. A popular type of visual content is infographics- packed with information, made to be attractive to the audience. Want […]
blog
7 Best Lottie Marketplaces To Help With Your Projects
16/04/2022
The Internet has brought the world closer, and sharing information has become extremely easy in this day and age. Today, there are numerous methods that let you share information, from videos to text and infographics to images. One such method is the use of animations which gives lets you provide information in a highly expressive […]

Will artificial intelligence (AI) take over the world of animation?

23/11/2021
blog

Modern technology has become an irreplaceable pillar in our society. With phones, tablets, and laptops always within an arm’s reach, it would be silly to not take advantage of such helpful tools. Technology is used in every aspect of modern life but the one field it has yet to take over is the arts. AI animation has become a popular stepping stone into the field due to the apprehension amongst other art forms. 

As technology advances more and more by the minute, we should not forget that with advancement also comes the potential to uplift and create. Technology is a tool, not a threat, so as long as we embrace the advancements, I believe we can use this technology as a shortcut to lead to more opportunity for artists and animators. 

That leads us to the question; Will Artificial Intelligence take over animation? As a career? No. As a tool? Absolutely. Let’s talk about it.

What is AI?

What comes to mind when I say, “artificial intelligence”?  You might be thinking of sophisticated robots or those text generated scripts that circulate twitter from time to time. And while those are certainly forms of AI, in reality this technology is much more present in our everyday lives. Every time you perform a simple google search, you’re interacting with AI. 

How about speech recognizing software such as Alexa or Siri? You’re having a conversation with AI. Every time you scroll through recommendations on Netflix, you’re looking through a custom panel created for you by AI animation. Even when you’re playing the ‘computer’ in a quick round of chess to kill time, you’re in a simulated game experience run by AI. It can literally be a part of nearly every moment.

So what is “AI” really?

AI or artificial intelligence is any machine that can demonstrate “cognitive” function that mimics the natural intelligence displayed by animals – humans included.

 Although it sounds like a product from science fiction fanfare of the 1950s, the idea of artificial intelligence has been around since ancient times. Ancient Greek mythology tells the story of Talos, a giant bronze human-like figure who protected Crete from possible invaders.  In 1818, Mary Shelley wrote a novel conceiving of creating life and intelligence through electricity, Frankenstein

The ethics of the topic have also been up for debate for just as long. The most popular argument has been that of an artificial overthrow. Where AI could potentially become so powerful that humans would not be able to stop it from achieving its goals. A scary enough thought to become the plot of dozens of hollywood horror films. It’s even created “Man vs Machine” as its own conflict category in literature. 

And our fears have also found themselves in our real world.  Many debate today the ethics of having AI in our homes, helping us perform basic tasks, and helping surveillance.

Artificial Intelligence in Art

Despite these ethical concerns, playing a fictional opponent is a good way to practice your end game. Similarly, asking Siri to set a timer serves as the perfect assistant. So why are people still so uncomfortable with the idea of AI in fields such as the arts? 

Fear of the unknown. From an early age, humans are taught to fear what’s different from us which is why we grew up learning phrases like “stranger danger.” When something as heartfelt and soul-filled as art gets threatened by cold soulless technology, it causes people to panic. Will robots replace artists? Will the already shrinking art fields become more sparse? If not for the artists, who will record the human condition? It can be a scary consideration for those that don’t trust that humans can work with the technology. 

Here’s the catch; fear is the roadblock of progress. Instead of seeing AI art as competition we need to see it as a teammate. AI animation refers to any work assisted by AI. Instead of being seen as a creation it’s more of a collaboration between man and machine. When speaking specifically about art created by AI, even that can’t be seen as original work.

The computers most often rely on machine learning, a specific type of AI that involves feeding computers countless examples of something until they learn the repeating patterns which results in them being able to create their own similar result. Hence, mimicked work.

Instead of seeing AI art as competition we need to see it as a teammate.

Avidor, Vidalgo

AI Animation brings a huge value to artists

To be truly original, one has to create, led by their own experiences. Without a heart behind it, I don’t believe that art created by AI can be considered true art. Art created by AI, although interesting in theory, relies too heavily on examples to be original.

It’s important to use this technology to our advantage. When it comes to AI animation,  it would be silly not to acknowledge the strengths and use them to streamline tedious, repetitive work loads. Most animation companies and studios have been using this technology and developing their own over the past decade.

Specific AI art software can be created to animate a character’s hair to such success that an audience can see individual realistic strands. A feat that would not be possible with production schedules being as strict as they are if it wasn’t for the help of AI animation.

Just take our own tools at Vidalgo.  By no means does Vidalgo take the creator out of the equation.  We simply give the creator the support of AI so they can scale effectively and focus most of their energy on the creation itself.

Final Thoughts

One thing certain about today’s climate is the uncertainty of tomorrow. It’s scary to watch the rapid acceleration of technology without knowing where it’s future is going. If we’ve learned anything about technological advancement, it’s that it can lead to massive job loss.

That leads us back to the main question at hand; Will AI take over animation? 

In my own opinion as a humble artist, I don’t believe it will. It’s an age old question, “Will our creations come to replace us?” I don’t think so. To be human is to live and create. All art comes from a place of personal experience or inspiration. A machine, no matter how hard it tries, can only mimic the original art we give it to study. Yes, art can be patterninistic and repetitive but again, something human inspired those patterns to be where they’re placed. 

There is nothing in the world that can replace the voice of a person with real emotions and lived experience. Each artist is unique and technology won’t be replacing that, only improving it.

Other Posts

blog
Best Lottie Players, Plugins and Tools
30/04/2022
Lottie has revolutionized the world of animation. To give a very basic definition, Lottie is a file format specific for animations, much like GIF. Due to Lottie animations, users can now display high-quality animations without any problems, such as pixelation, large file size, or slow webpage loading speeds. A single Lottie file is many times […]
blog
Infographics Trends and Animated Infographics in 2022
23/04/2022
With the rise of social media usage, visual content is becoming more and more popular than mere texts. From informational videos to sending memes to your friends and family, visual content is what people love nowadays. A popular type of visual content is infographics- packed with information, made to be attractive to the audience. Want […]
blog
7 Best Lottie Marketplaces To Help With Your Projects
16/04/2022
The Internet has brought the world closer, and sharing information has become extremely easy in this day and age. Today, there are numerous methods that let you share information, from videos to text and infographics to images. One such method is the use of animations which gives lets you provide information in a highly expressive […]