17
likes
0
comments
daily view 0
monthly view 0

Share

Live Analytics

Comments 0

LLMLingua: Compressing Prompts for Accelerated Inference of LLMs Analytics Table

Income Estimates for LLMLingua: Compressing Prompts for Accelerated Inference of LLMs

Based on this YouTube video's total view count of 771 views and industry-standard rates, the estimated total earning is $1 - $2 through ad revenue. Historical data is not yet available to calculate daily, weekly, or monthly averages.

About LLMLingua: Compressing Prompts for Accelerated Inference of LLMs

Explore LLMLingua: Compressing Prompts for Accelerated Inference of LLMs with 771 views, 17 likes, and 0 comments. Experience the impact of this video content that has captured audience attention.

LLMs have been applied in various applications due to their astonishing capabilities. With advancements in technologies such as chain-of-thought (CoT) prompting and in-context learning (ICL), the prompts fed to LLMs are becoming increasingly lengthy, even exceeding tens of thousands of tokens. To accelerate model inference and reduce cost, LLMLingua is a coarse-to-fine prompt compression method that involves a budget controller to maintain semantic integrity under high compression ratios, a token-level iterative compression algorithm to better model the interdependence between compressed contents, and an instruction tuning based method for distribution alignment between language models. Experiments and analysis are done over four datasets from different scenarios, i.e., GSM8K, BBH, ShareGPT, and Arxiv-March23. They show that LLMLingua yields state-of-the-art performance and allows for up to 20x compression with little performance loss. In this video, I talk about the following: How does LLMLingua compress prompts? How does LLMLingua perform? Case study on GSM8K math reasoning prompt compression. Can we recover original prompts compressed by LLMLingua? For more details, please look at https://llmlingua.com/ and https://arxiv.org/pdf/2310.05736 Jiang, Huiqiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. "LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models." In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 13358-13376. 2023.

About YouTube Real-Time View Count

With SocialCounts.org’s view counter, track your YouTube video’s live view count and YouTube likes count in real time with fast, reliable updates.

Watch every YouTube video live view count rise with our real-time YouTube views tracker—built for accuracy and minimal delay.

Follow YouTube real time views as they happen, using our dedicated view counter for YouTube videos.

Get up-to-date live view count on YouTube and see real-time growth with SocialCounts.org’s smart tracking tools.

Embed Widget

Parameters:

  • fullscreen=true - Fullscreen counter
  • graph=true - Live graph chart
  • counter=0/1/2 - Select counter (0=likes, 1=views, 2=comments)
URL

Click to copy the embed URL to your clipboard