The Princess' Butterflies - Using an AI Image to Video Generator to Animate Art That Started as a 23 Year Old Rough Pen Sketch

The Princess' Butterflies - Image to Video with PixVerse AI

Back in October of 2023 I used Prome AI's Sketch Rendering feature to color a 23 year old rough pen sketch I did of a princess watching butterflies, sitting next to a waterfall in the forest. You can read more about that in my previous article, The Princess' Butterflies - Using an AI Art Generator to Color and Enhance a 23 Year Old Rough Pen Sketch

The three images I used to create my final image
(bottom right). Notice how closely the AI has followed
my line art sketch (top left) but in the first image
(top right) has misinterpreted the waterfall as some kind
of large tree stump.

Prome's render of the artwork was pretty good but I did end up creating a composite image myself from two of the best renders, to get my final image.

Since then AI Image to Video has come a long way, particularly when it comes to creating animation from artwork and maintaining the art style for the whole generation.

AI still does much better with turning photographic images to video but I found an AI site, PixVerse, that seems more focused on animated styles. Initially I used the site to animate my cartoon TET avatar, which you can read about in my Animation and Video life blog.

Since PixVerse did such a good job at maintaining my art style, by not actually applying any of their art style models to the image (which are all very distinct animation styles like Anime), I thought it might be worth a try running my Princess AI image through to see what happened.

Overall I was quite impressed. I generated four different videos of nine seconds each, getting something reasonably coherent each time.

I did have a few issues like butterflies morphing into a single butterfly, butterflies randomly appearing out of nowhere, and the usual deformed hands (but only if you really look because it tended to happen if the hand moved fast into position).

PixVerse created this 'happy accident' when it randomly
generated a magical circle after misinterpreting my prompt.

One generation I got a happy accident where I attempted to get the butterflies to fly in a more circular formation. Instead the AI interpreted the word circle by adding a magical light circle (kind of like a spell had been cast) appearing around the butterflies.

I decided to take my four generations and edit the best bits together into a single video clip, adding sound effects, music, and even the princess whispering some dialogue. Which was an after thought. It just seemed odd to have all the sound design and the princess, who is clearly speaking, not having any audio.

Although PixVerse does have a lip sync feature that I could have used, I ran out of paid credits, and didn't want to purchase more because I didn't have any immediate need for them after this project - which, in itself, is just a test project. So I just tried to come up with some dialogue using ElevenLabs AI voices that approximated with her lip movement.

You can also generate sound, speech, and even extend the video right in PixVerse but again, I didn't want to purchase the extra credits on a test animation.

The resulting video is below. I tried to edit it so that if you loop it, it's not too jarring, though obviously the dialogue becomes a bit redundant after the second play through. I hope you like it. Feel free to leave a comment to let me know what you think.

Comments

New Cat Art Collaboration: TET's Cats Paintings and OpenArt AI Model Workspace (Photobooth)

TET's Cats AI generated art trained on my own art style. Way back in the early 2000's I started painting stylized cat artworks to illustrate some cat themed poems I'd written, that I exhibited and sold online in an exhibition titled 'Sleeping Cats' in 2004. You can see all these early works in my Flickr Album . Many are also available to buy as prints in my RedBubble Store . Leading on from that I began selling my paintings on ebay where the cat themed works were almost guaranteed to sell over any other subject I painted. As a result I became some what known for my cat art to the point where people would commission me to create images of their own pet cats in my cartoony style. Flash forward a decade (almost two at this point) and I haven't painted any cat themed art in years. To be honest I haven't done any traditional painting at all in years. In the last couple of years AI image generators have really caught my attention. Specifically that they are able t...

TET Artwork Diary: Nansen The Cabin Boy's Cat of the Belgian Sailing Ship, RV Belgica, Trapped in an Antarctica Winter, 1897

Belgica in the ice. 1897. Photo: Adrien de Gerlache Y ou wouldn't ordinarily think of cats and penguins together in the same image but that's what I found myself creating when thinking of an idea for my next painting. I have painted penguins before as part of my Australian Animals series from the early 2000's, and I have a couple of cartoon penguins (that you can see in a very old post, Designing Formal Penguins ) I created for a Yeti skateboard graphic, so, I didn't just randomly think, 'penguins'. Recently I had a revelation about my art, in that I don't simply paint pictures of cats, I paint 'relationships'. Almost all of my paintings of cats feature two or more cats, or a cat with something, such as a butterfly, bird, fish, goat... and why not a penguin? However, I didn't want to just paint some random cat with some random penguin. I wondered if there was ever any real cats in history that may have a reason to meet a penguin? Turns out, on...

"Ya Mum Reads it..."

You'l find my own Photography in this blog too. Quite a while back I did a survey that tells you what kind of audience credibility your blog has. I'm not sure what the score was but I do remember the commentary which boiled down to "Your mother reads your blog regularly but you probably don't have much wider appeal". Fuck you survey! (Hi Mum, sorry for cursing! I do appreciate you reading, really I do!) I don't know how many regular readers I have, if any, but I do know, on average this blog receives more than 4000 page hits per month, much of it coming from search engines like Google. Perhaps of some interest is my new current top search entry point of all time, people looking for Bert and Ernie, who I wrote some commentary about when it was officially announced that the Sesame Street Characters were not gay . Which is quite a departure from Krippin Virus from a few years back when the movie I am Legend came out. The thing about this blog is t...

Review: ArtHelper - The All-In-One AI Writing + Marketing Assistant for Artists - 'ChatGPT for Artists'

ArtHelper prides itself on being all 'human-made' art. T he idea of an AI, trained specifically on art business marketing, that can not only offer advice on marketing your work, but also assist with creating all the content too, is certainly appealing. Especially to those of us who would rather spend more time creating our art than trying to sell it. ArtHelper does just that whilst attempting to be your 'home' on the internet. A destination for your profile and portfolio, a marketplace for your art, and a directory of artists as well, with one distinction - all the art must be human made. Which, for you AI artists, doesn't count the prompt for AI generated art - because the idea, according to ArtHelper's creators, isn't the art. Which is a fair point, in terms of promoting art 'made by a human', but can get kind of murky when you understand that not all AI art is generated from a single prompt... and 'found object art' isn't actually ...

Movie Review: Spider-Man: Brand New Day (2026) No Spoilers

T his is the first time in quite a while we've seen a live action Spider-Man that is, essentially, a fully formed, grown up superhero, fighting crime on his own terms rather than looking to mentors to guide him. In Spider-Man: Brand New Day it's four years after the events of Spider-Man: No Way Home . No one knows who Peter Parker is but Spider-Man has become a local legend, helping to keep New York free of crime. Personally, Peter is still struggling with not reconnecting with his former friends, whose lives are again put at risk when a new foe crosses his path. If you're a fan of Tom Holland's Spider-Man films, and the MCU in general this film should not only not disappoint, you may even consider it to be one of the best MCU films you've seen in a while. I really enjoyed all the quick cameos by small time Spider-Man villains even though I don't know many of them. Their inclusion really started to broaden the scope of the character, despite this installment ...

Boom Crash Opera Born Classic But Not Again

Boom Crash Opera are an Australian Band that reached the peak of their popularity in the mid to late nineteen eighties. They are a band that I knew about at that time but was never really excited by until they released their ill fated double album Born and Born Again in 1995 (Album cover pictured). At the time of its release I was very much into emerging Australian musical acts and was also looking out for new sounds that were different and had kind of a futuristic/electronic sound. Artists that I was buying at the time included; Swoop , Nine Inch Nails and Pop Will Eat Its Self . As well as a really interesting release by David Bowie, the concept album, Outside . Born was a fairly radical departure for Boom Crash Opera (BCO). The first single, Gimme , was often compared to the sounds of Gary Glitter, particularly his single, Rock n Roll part 2 , because of the pounding drum loops. Watch the video below. My favorite single from the album is dissemble which probably went now...

Embarrassing Yourself with Doof Doof Music

Ever since Spinal Tap popularized the notion of turning an amp up to eleven audio systems have been rising to the challenge of louder, more awesome sound. In particular, car audio systems have embraced the notion of the bigger the noise the better the sound must surely be. What I'd like to know is, if these sound systems are so good, why do they make all music sound the same... Doof, Doof, Doof? You've all heard them. Pimped up cars, with audio systems seemingly worth more than gold, volume blasting way past eleven. Sound waves booming long before you make a positive visual on the car... Doof, Doof, Doof. It has to be the audio system right? Surely all these people can't be playing the same tune? Doof, Doof, Doof. Could it be that the more money spent on a car sound system the less musical range it will actually play? Maybe these people can't afford to buy music after purchasing the audio system so they all play the same demo track? Doof, doof, doof. In my own car I hav...

TET.Life (The Extraordinary Tourist Life)

Search This Blog