691
likes
30
comments
daily view 0
monthly view 0

Share

Live Analytics

Comments 0

End-to-End (small) Vision Language Model Fine-tuning Tutorial | On DGX Spark Analytics Table

Income Estimates for End-to-End (small) Vision Language Model Fine-tuning Tutorial | On DGX Spark

Based on this YouTube video's total view count of 24.0K views and industry-standard rates, the estimated total earning is $17 - $48 through ad revenue. Historical data is not yet available to calculate daily, weekly, or monthly averages.

About End-to-End (small) Vision Language Model Fine-tuning Tutorial | On DGX Spark

Explore End-to-End (small) Vision Language Model Fine-tuning Tutorial | On DGX Spark with 24,049 views, 691 likes, and 30 comments. Experience the impact of this video content that has captured audience attention.

In this video we fine-tune Hugging Face's SmolVLM2-500M Vision Language Model do structured data extraction from images. Because the SmolVLM2-500M model is is quite small in world of LLMs/VLMs, we're able to do all of the training locally on a NVIDIA DGX Spark (see here for more: https://nvda.ws/4iQXZU4). The code should also run in Google Colab. If you have any issues, please let me know in a comment. Links: Google Colab Notebook - https://colab.research.google.com/drive/1yOwjCGZSq2jB4YLF0O0na2rEuqB6Nh7m?usp=sharing GitHub - https://github.com/mrdbourke/learn-huggingface/blob/main/notebooks/hugging_face_vlm_fine_tune_tutorial.ipynb Learn Hugging Face Book Version - https://www.learnhuggingface.com/notebooks/hugging_face_vlm_fine_tune_tutorial Dataset - https://huggingface.co/datasets/mrdbourke/FoodExtract-1k-Vision Base model (SmolVLM2-500M) - https://huggingface.co/HuggingFaceTB/SmolVLM2-500M-Video-Instruct Demo - https://huggingface.co/spaces/mrdbourke/FoodExtract-Vision-v1 Livestreams (where I build this project from scratch): Part 1: Creating a VLM dataset - https://www.youtube.com/live/cZVU559BLLM?si=jyI9pWXxkmXMO9qq Part 2: Fine-tuning a VLM with LoRA and QLoRA and getting many errors (mostly my fault) - https://www.youtube.com/live/Lgcp9hBqWEM?si=gA_7exPeqIwcxRYj Part 3: Switching from using LoRA and QLoRA (we’ll do these in a future video) to fine-tuning a smaller model (SmolVLM2) successfully, uploading it to the Hugging Face Hub and then creating an publishing a demo - https://youtube.com/live/cZVU559BLLM?feature=share Courses I teach: Learn AI/ML (beginner-friendly course) - https://dbourke.link/ZTMMLcourse Learn Hugging Face - https://dbourke.link/ZTMHuggingFace Learn TensorFlow - https://dbourke.link/ZTMTFcourse Learn PyTorch - https://dbourke.link/ZTMPyTorch Connect elsewhere: Download Nutrify (my startup) - https://apple.co/4ahM7Wc My website - https://www.mrdbourke.com X/Twitter - https://www.twitter.com/mrdbourke LinkedIn - www.linkedin.com/in/mrdbourke Get email updates on my work - https://dbourke.link/newsletter Read my novel Charlie Walks - https://www.charliewalks.com Timestamps: 00:00:00 - Introduction 00:02:19 - What is a VLM? 00:03:45 - Why fine-tune your own model? 00:06:05 - LLM fine-tuning mindset 00:06:51 - Case study: Nutrify 00:09:16 - Case study: Invoice Extractor 00:11:06 - Ingredients and tools we're going to use 00:12:16 - Exploring the FoodExtract-1k-Vision dataset 00:15:52 - My setup 00:16:13 - Dataset formatting for VLMs 00:16:54 - Dataset Creation for VLMs 00:17:20 - Getting a model to fine-tune 00:18:13 - Our task overview (structured data extraction from images) 00:20:11 - What we're going to end up with 00:22:38 - Code Starts 00:23:31 - Viewing a single data sample 00:29:08 - Splitting our data into train and test sets 00:34:25 - Inspecting our model's architecture 00:40:03 - Reading the recipe of the SmolDocling paper 00:45:29 - Freezing the vision encoder in our model 00:47:34 - Discussing batch sizes 00:49:06 - Setting up SFTConfig 00:52:03 - Training our model with SFTTrainer 00:54:11 - Model training starts 00:54:19 - Model training finishes 00:56:13 - Inspecting our model's loss curves 00:57:10 - Uploading our trained model to Hugging Face 00:58:19 - Model uploading to Hugging Face begins 00:58:26 - Model uploading finishes 00:59:38 - Comparing the base model to the fine-tuned model 01:01:06 - Viewing our fine-tuned model's first predictions 01:03:35 - Creating a demo with Gradio 01:06:46 - Uploading our demo to the Hugging Face Hub 01:07:35 - Trying out our demo 01:08:27 - What's next and extensions

About YouTube Real-Time View Count

With SocialCounts.org’s view counter, track your YouTube video’s live view count and YouTube likes count in real time with fast, reliable updates.

Watch every YouTube video live view count rise with our real-time YouTube views tracker—built for accuracy and minimal delay.

Follow YouTube real time views as they happen, using our dedicated view counter for YouTube videos.

Get up-to-date live view count on YouTube and see real-time growth with SocialCounts.org’s smart tracking tools.

Embed Widget

Parameters:

  • fullscreen=true - Fullscreen counter
  • graph=true - Live graph chart
  • counter=0/1/2 - Select counter (0=likes, 1=views, 2=comments)
URL

Click to copy the embed URL to your clipboard