So Sad – Rein (AI-Directed Music Video)
Project Overview
A client approached me to produce a music video for a collaboration between three artists. Each artist lived in a different state, making a traditional video shoot impractical due to travel costs and scheduling. The project also had a limited production budget and an aggressive three-day deadline.
After evaluating the project's constraints, I determined that an AI-assisted production pipeline would provide the highest production value while remaining within budget and delivering on time. Rather than using AI simply to generate visuals, I treated it as a production tool within a carefully planned creative workflow.
My Process
1. Evaluating the Production Constraints
Before creating anything, I evaluated the project's limitations.
Challenges
Three artists located in different states
No budget for travel or production crews
No professional filming locations
Limited budget
Three-day turnaround
Wanted a cinematic music video rather than a lyric video
I also considered what the concept required.
The client wanted each artist to exist within a completely different period in history. Accomplishing this traditionally would have required:
Horses
Western towns
Vintage vehicles
Period wardrobe
Historical props
Multiple filming locations
Drone operators
Camera crews
Set design
Producing this conventionally would have been well outside the available budget and timeline. AI made these environments achievable without sacrificing the creative vision.
2. Creative Direction & Concept Development
I began by writing a creative brief instead of jumping directly into image generation.
The concept was to have each artist appear in a different era of history, giving every verse its own distinct visual identity while maintaining a cohesive narrative.
The historical themes included:
American West
1930s
1980s
Having a strong concept first ensured that every creative decision—from wardrobe to environments—supported the overall story.
3. Storyboarding the Entire Song
Rather than generating random visuals and hoping they fit, I broke the song down line by line.
For every lyric, I identified:
What the audience should see
The emotional tone
Camera composition
Character performance
Environmental details
Scene transitions
This became the storyboard that drove the entire production.
Planning everything beforehand dramatically reduced unnecessary generations later and kept the visual storytelling synchronized with the music.
4. Gathering Visual Assets
The artists did not have professional photography available.
Instead, I sourced the highest-quality images from their Instagram profiles and gathered multiple references of each artist.
These included:
Front view
Side profile
Three-quarter profile
Different facial expressions
Hair references
Additional photos whenever available
I also researched historical reference material for each era, including:
Clothing
Architecture
Vehicles
Streets
Lighting
Props
Color palettes
Photography styles
This research established visual consistency before any AI generation began.
5. Preparing Character Reference Images
The Instagram photos were not high enough quality to be used directly for generation.
Using ChatGPT image generation, I created higher-quality reference images of each artist while preserving their likeness.
Rather than generating a single portrait, I intentionally created multiple viewing angles.
This included:
Front
Side
Three-quarter
Rear
Close-up facial references
Having multiple reference angles significantly reduced identity drift and hallucinations during later image and video generation.
It essentially gave the AI a much stronger understanding of what each artist looked like.
6. Costume Design
After establishing consistent character references, I generated historically accurate wardrobe for each artist.
The clothing needed to:
Match the assigned era
Reflect each artist's personality
Preserve recognizable facial identity
Feel believable within each environment
This step established consistency before moving into scene generation.
7. Generating Key Scene Images
Next, I generated the hero images for every planned shot.
Each scene considered:
Lyrics
Camera framing
Lighting
Composition
Mood
Environment
Character performance
Wardrobe
Historical accuracy
Rather than thinking of these as standalone images, I treated them as keyframes that would eventually become moving shots.
8. AI Video Generation
Once the keyframes were complete, I imported them into Grok to generate video sequences.
Each prompt described:
Character movement
Camera movement
Environmental motion
Atmospheric effects
Performance direction
Scene pacing
Examples included:
Walking through town
Riding horses
Crowd movement
Wind interaction
Slow cinematic push-ins
Tracking shots
Performance animation
Very rarely does AI generate the perfect result on the first attempt.
Each shot required multiple iterations.
My workflow consisted of:
Generate → Review → Identify Problems → Rewrite Prompt → Regenerate
This iterative process continued until the generated clip matched the intended creative direction.
9. Maintaining Character Continuity
One of the biggest challenges in AI filmmaking is maintaining consistency between scenes.
Without guidance, AI often changes:
Facial structure
Hairstyles
Clothing
Backgrounds
Camera position
Character proportions
To solve this, I frequently captured the final frame of one completed scene and used it as the starting image for the following scene.
This created visual continuity and dramatically reduced unexpected changes between shots.
This technique helped make the finished video feel much more like traditionally filmed footage rather than disconnected AI-generated clips.
10. Quality Control
Throughout production, I continuously reviewed every generated asset for consistency.
I checked for:
Facial accuracy
Historical accuracy
Clothing consistency
Prompt artifacts
Extra limbs
Distorted anatomy
Lighting consistency
Scene continuity
Composition
Emotional tone
Whenever something felt off, I regenerated or revised the prompts before moving forward.
Quality control was continuous rather than something saved until the end.
11. Post Production
Once all scenes were complete, I assembled the final music video inside Final Cut Pro.
Post-production included:
Editing
Scene timing
Music synchronization
Shot selection
Transitions
Color adjustments
Minor cleanup
Correcting AI artifacts when necessary
Most corrections were small, but Final Cut allowed me to polish anything AI didn't execute perfectly.
Challenges
The biggest challenge wasn't editing the video.
The biggest challenge was directing the AI.
Generating quality AI content requires significantly more than writing prompts.
It involves:
Creative direction
Storyboarding
Reference gathering
Prompt engineering
Visual research
Character design
Iterative testing
Continuity management
Quality assurance
Production planning
The AI became another production tool that required direction and refinement, much like directing a film crew.
Results
The complete production—from concept development through final delivery—took approximately 3–4 hours.
The client received a cinematic music video that would have been extremely expensive to produce traditionally.
The AI workflow allowed me to:
Deliver within a three-day deadline
Eliminate travel for three artists
Create historically accurate worlds
Produce cinematic camera movements
Generate production value that exceeded the available budget
Maintain visual consistency across multiple historical settings
Lessons Learned
This project reinforced several best practices for AI-assisted creative production:
Strong planning produces better AI outputs than relying on prompting alone.
High-quality reference images dramatically improve character consistency.
Storyboarding before generation reduces unnecessary iterations.
AI works best when guided with clear creative direction rather than vague instructions.
Iterative refinement is an essential part of the workflow.
Continuity management is one of the most important skills in AI filmmaking.
Human judgment remains critical throughout the production process, from concept development to final quality control.
Key Takeaway
This project demonstrates my ability to design and direct an end-to-end AI production workflow—not simply generate content with AI. I combined creative strategy, visual storytelling, prompt engineering, asset preparation, continuity planning, iterative refinement, and professional post-production to deliver a cinematic music video under significant budget and time constraints.
Rather than replacing the creative process, AI accelerated execution while allowing me to focus on the decisions that matter most: concept development, storytelling, art direction, and overall production quality.