Bedste Igor Melnyk-podcasts (2024)

1
[QA] Recite, Reconstruct, Recollect: Memorization in LMs as a Multifaceted Phenomenon 7:42

7h ago7:42

7:42

Memorization in language models is complex and influenced by various factors. A taxonomy approach helps understand and predict memorization patterns. https://arxiv.org/abs//2406.17746 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id169247…

1
Recite, Reconstruct, Recollect: Memorization in LMs as a Multifaceted Phenomenon 12:44

7h ago12:44

12:44

Memorization in language models is complex and influenced by various factors. A taxonomy approach helps understand and predict memorization patterns. https://arxiv.org/abs//2406.17746 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id169247…

1
[QA] Adam-mini: Use Fewer Learning Rates To Gain More 7:57

1d ago7:57

7:57

Adam-mini optimizer reduces memory footprint by using average learning rates within parameter blocks, achieving performance comparable to AdamW with significantly less memory. https://arxiv.org/abs//2406.16793 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/pod…

1
Adam-mini: Use Fewer Learning Rates To Gain More 13:47

1d ago13:47

13:47

Adam-mini optimizer reduces memory footprint by using average learning rates within parameter blocks, achieving performance comparable to AdamW with significantly less memory. https://arxiv.org/abs//2406.16793 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/pod…

1
[QA] Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMs 11:02

1d ago11:02

11:02

SEPs offer a cost-effective method for detecting hallucinations in Large Language Models by approximating semantic entropy from hidden states, improving efficiency and generalization. https://arxiv.org/abs//2406.15927 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.co…

1
Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMs 14:49

1d ago14:49

14:49

SEPs offer a cost-effective method for detecting hallucinations in Large Language Models by approximating semantic entropy from hidden states, improving efficiency and generalization. https://arxiv.org/abs//2406.15927 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.co…

1
[QA] Evaluating Numerical Reasoning in Text-to-Image Models 11:41

2d ago11:41

11:41

Text-to-image models struggle with numerical reasoning tasks, showing limitations in generating exact numbers, understanding quantifiers, zero, and advanced concepts. GECKONUM benchmark is introduced for evaluation. https://arxiv.org/abs//2406.14774 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Pod…

1
Evaluating Numerical Reasoning in Text-to-Image Models 13:10

2d ago13:10

13:10

Text-to-image models struggle with numerical reasoning tasks, showing limitations in generating exact numbers, understanding quantifiers, zero, and advanced concepts. GECKONUM benchmark is introduced for evaluation. https://arxiv.org/abs//2406.14774 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Pod…

1
[QA] Advantage Alignment Algorithms 8:25

2d ago8:25

8:25

The paper introduces Advantage Alignment, an algorithm for opponent shaping in AI agents to find socially beneficial equilibria efficiently, proving its effectiveness in various social dilemmas. https://arxiv.org/abs//2406.14662 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcas…

1
Advantage Alignment Algorithms 11:51

2d ago11:51

11:51

The paper introduces Advantage Alignment, an algorithm for opponent shaping in AI agents to find socially beneficial equilibria efficiently, proving its effectiveness in various social dilemmas. https://arxiv.org/abs//2406.14662 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcas…

1
[QA] Transcendence: Generative Models Can Outperform The Experts That Train Them 9:23

3d ago9:23

9:23

Generative models can surpass human performance when trained on data generated by humans, demonstrated by a chess-playing transformer model achieving better performance than human players. https://arxiv.org/abs//2406.11741 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.app…

1
Transcendence: Generative Models Can Outperform The Experts That Train Them 15:59

3d ago15:59

15:59

Generative models can surpass human performance when trained on data generated by humans, demonstrated by a chess-playing transformer model achieving better performance than human players. https://arxiv.org/abs//2406.11741 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.app…

1
[QA] Refusal in Language Models Is Mediated by a Single Direction 7:44

3d ago7:44

7:44

Study explores refusal behavior in chat models, identifying a one-dimensional subspace mediating refusal. Proposes a method to disable refusal while preserving other capabilities, highlighting safety fine-tuning limitations. https://arxiv.org/abs//2406.11717 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers …

1
Refusal in Language Models Is Mediated by a Single Direction 18:40

3d ago18:40

18:40

Study explores refusal behavior in chat models, identifying a one-dimensional subspace mediating refusal. Proposes a method to disable refusal while preserving other capabilities, highlighting safety fine-tuning limitations. https://arxiv.org/abs//2406.11717 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers …

1
[QA] Instruction Pre-Training: Language Models are Supervised Multitask Learners 11:26

4d ago11:26

11:26

The paper introduces Instruction Pre-Training, a framework for supervised multitask pre-training of language models using instruction-response pairs, showing improved generalization and performance. https://arxiv.org/abs//2406.14491 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://po…

1
Instruction Pre-Training: Language Models are Supervised Multitask Learners 12:59

4d ago12:59

12:59

The paper introduces Instruction Pre-Training, a framework for supervised multitask pre-training of language models using instruction-response pairs, showing improved generalization and performance. https://arxiv.org/abs//2406.14491 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://po…

1
[QA] Can Long-Context Language Models Subsume Retrieval, RAG, SQL, and More? 11:15

4d ago11:15

11:15

Long-context language models (LCLMs) show promise in revolutionizing tasks without external tools, as demonstrated by LOFT benchmark's evaluation of LCLMs' performance in complex contexts. https://arxiv.org/abs//2406.13121 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.app…

1
Can Long-Context Language Models Subsume Retrieval, RAG, SQL, and More? 17:14

4d ago17:14

17:14

Long-context language models (LCLMs) show promise in revolutionizing tasks without external tools, as demonstrated by LOFT benchmark's evaluation of LCLMs' performance in complex contexts. https://arxiv.org/abs//2406.13121 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.app…

1
[QA] RL on Incorrect Synthetic Data Scales the Efficiency of LLM Math Reasoning by Eight-Fold 10:29

5d ago10:29

10:29

https://arxiv.org/abs//2406.14532 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016 Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers --- Support this podcast: https://podcasters.spotify.com/pod/show/arxiv-papers/supp…

1
RL on Incorrect Synthetic Data Scales the Efficiency of LLM Math Reasoning by Eight-Fold 13:02

5d ago13:02

13:02

https://arxiv.org/abs//2406.14532 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016 Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers --- Support this podcast: https://podcasters.spotify.com/pod/show/arxiv-papers/supp…

1
[QA] Consistency Models Made Easy 8:31

5d ago8:31

8:31

Proposes Easy Consistency Tuning (ECT) for training consistency models, improving efficiency significantly. Achieves high quality results on CIFAR-10 in just 1 hour on a single GPU. https://arxiv.org/abs//2406.14548 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/…

1
Consistency Models Made Easy 19:39

5d ago19:39

19:39

Proposes Easy Consistency Tuning (ECT) for training consistency models, improving efficiency significantly. Achieves high quality results on CIFAR-10 in just 1 hour on a single GPU. https://arxiv.org/abs//2406.14548 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/…

1
[QA] What Are the Odds? Language Models Are Capable of Probabilistic Reasoning 14:25

7d ago14:25

14:25

Paper evaluates language models' probabilistic reasoning abilities using statistical distributions. Three tasks assessed with different contextual inputs. Models can infer distributions with real-world context and simplified assumptions. https://arxiv.org/abs//2406.12830 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@…

1
What Are the Odds? Language Models Are Capable of Probabilistic Reasoning 10:43

7d ago10:43

10:43

Paper evaluates language models' probabilistic reasoning abilities using statistical distributions. Three tasks assessed with different contextual inputs. Models can infer distributions with real-world context and simplified assumptions. https://arxiv.org/abs//2406.12830 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@…

1
[QA] Adversarial Attacks on Multimodal Agents 8:27

7d ago8:27

8:27

The paper explores safety risks posed by multimodal agents and demonstrates attacks using adversarial text strings to manipulate VLMs, with varying success rates based on different models. https://arxiv.org/abs//2406.12814 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.app…

1
Adversarial Attacks on Multimodal Agents 16:08

7d ago16:08

16:08

The paper explores safety risks posed by multimodal agents and demonstrates attacks using adversarial text strings to manipulate VLMs, with varying success rates based on different models. https://arxiv.org/abs//2406.12814 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.app…

1
[QA] Can Go AIs be adversarially robust? 7:44

7d ago7:44

7:44

The paper explores defenses to improve KataGo's performance against adversarial attacks in Go, finding some defenses effective but none able to withstand adaptive attacks. https://arxiv.org/abs//2406.12843 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast…

1
Can Go AIs be adversarially robust? 12:59

7d ago12:59

12:59

The paper explores defenses to improve KataGo's performance against adversarial attacks in Go, finding some defenses effective but none able to withstand adaptive attacks. https://arxiv.org/abs//2406.12843 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast…

1
[QA] Autoregressive Image Generation without Vector Quantization 10:24

8d ago10:24

10:24

Proposing a diffusion-based approach for autoregressive modeling in continuous-valued space, eliminating the need for discrete tokens and achieving strong results in image generation. https://arxiv.org/abs//2406.11838 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.co…

1
Autoregressive Image Generation without Vector Quantization 9:16

8d ago9:16

9:16

Proposing a diffusion-based approach for autoregressive modeling in continuous-valued space, eliminating the need for discrete tokens and achieving strong results in image generation. https://arxiv.org/abs//2406.11838 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.co…

1
[QA] Measuring memorization in RLHF for code completion 9:39

8d ago9:39

9:39

https://arxiv.org/abs//2406.11715 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016 Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers --- Support this podcast: https://podcasters.spotify.com/pod/show/arxiv-papers/supp…

1
Measuring memorization in RLHF for code completion 16:38

8d ago16:38

16:38

https://arxiv.org/abs//2406.11715 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016 Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers --- Support this podcast: https://podcasters.spotify.com/pod/show/arxiv-papers/supp…

1
[QA] Bootstrapping Language Models with DPO Implicit Rewards 8:41

9d ago8:41

8:41

The paper introduces DICE, a method for aligning large language models using implicit rewards from DPO. DICE outperforms Gemini Pro on AlpacaEval 2 with 8B parameters and no external feedback. https://arxiv.org/abs//2406.09760 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts…

1
Bootstrapping Language Models with DPO Implicit Rewards 16:02

9d ago16:02

16:02

The paper introduces DICE, a method for aligning large language models using implicit rewards from DPO. DICE outperforms Gemini Pro on AlpacaEval 2 with 8B parameters and no external feedback. https://arxiv.org/abs//2406.09760 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts…

1
[QA] Ad Auctions for LLMs via Retrieval Augmented Generation 10:03

9d ago10:03

10:03

Novel auction mechanisms for ad allocation and pricing in large language models (LLMs) are proposed, maximizing social welfare and ensuring fairness. Empirical evaluation supports the approach's feasibility and effectiveness. https://arxiv.org/abs//2406.09459 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers…

1
Ad Auctions for LLMs via Retrieval Augmented Generation 14:18

9d ago14:18

14:18

Novel auction mechanisms for ad allocation and pricing in large language models (LLMs) are proposed, maximizing social welfare and ensuring fairness. Empirical evaluation supports the approach's feasibility and effectiveness. https://arxiv.org/abs//2406.09459 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers…

1
[QA] An Empirical Study of Mamba-based Language Models 10:36

10d ago10:36

10:36

Mamba models challenge Transformers at larger scales, with Mamba-2-Hybrid surpassing Transformers on various tasks, showing potential for efficient token generation. https://arxiv.org/abs//2406.07887 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv…

1
An Empirical Study of Mamba-based Language Models 28:32

10d ago28:32

28:32

Mamba models challenge Transformers at larger scales, with Mamba-2-Hybrid surpassing Transformers on various tasks, showing potential for efficient token generation. https://arxiv.org/abs//2406.07887 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv…

1
[QA] Unpacking DPO and PPO: Disentangling Best Practices for Learning from Preference Feedback 7:22

10d ago7:22

7:22

Preference-based learning for language models is crucial for enhancing generation quality. This study explores key components' impact and suggests strategies for effective learning. https://arxiv.org/abs//2406.09279 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/…

1
Unpacking DPO and PPO: Disentangling Best Practices for Learning from Preference Feedback 9:29

10d ago9:29

9:29

Preference-based learning for language models is crucial for enhancing generation quality. This study explores key components' impact and suggests strategies for effective learning. https://arxiv.org/abs//2406.09279 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/…

1
[QA] What If We Recaption Billions of Web Images with LLaMA-3? 10:11

11d ago10:11

10:11

The paper introduces Recap-DataComp-1B, an enhanced dataset created using LLaMA-3-8B to improve vision-language model training, showing benefits in performance across various tasks. https://arxiv.org/abs//2406.08478 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/…

1
What If We Recaption Billions of Web Images with LLaMA-3? 12:27

11d ago12:27

12:27

The paper introduces Recap-DataComp-1B, an enhanced dataset created using LLaMA-3-8B to improve vision-language model training, showing benefits in performance across various tasks. https://arxiv.org/abs//2406.08478 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/…

1
[QA] SAMBA: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling 9:34

11d ago9:34

9:34

SAMBA is a hybrid model combining Mamba and Sliding Window Attention for efficient sequence modeling with infinite context length, outperforming existing models. https://arxiv.org/abs//2406.07522 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-pap…

1
SAMBA: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling 13:02

11d ago13:02

13:02

SAMBA is a hybrid model combining Mamba and Sliding Window Attention for efficient sequence modeling with infinite context length, outperforming existing models. https://arxiv.org/abs//2406.07522 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-pap…

1
[QA] Why Warmup the Learning Rate? Underlying Mechanisms and Improvements 6:44

12d ago6:44

6:44

The paper explores the benefits of warmup in deep learning, showing how it improves performance by allowing networks to handle larger learning rates and suggesting alternative initialization methods. https://arxiv.org/abs//2406.09405 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://p…

1
Why Warmup the Learning Rate? Underlying Mechanisms and Improvements 21:40

12d ago21:40

21:40

The paper explores the benefits of warmup in deep learning, showing how it improves performance by allowing networks to handle larger learning rates and suggesting alternative initialization methods. https://arxiv.org/abs//2406.09405 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://p…

1
[QA] An Image is Worth More Than 1616 Patches: Exploring Transformers on Individual Pixels 9:36

12d ago9:36

9:36

Vanilla Transformers can achieve high performance in computer vision by treating individual pixels as tokens, challenging the necessity of locality bias in modern architectures. https://arxiv.org/abs//2406.09415 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/p…

1
An Image is Worth More Than 1616 Patches: Exploring Transformers on Individual Pixels 12:30

12d ago12:30

12:30

Vanilla Transformers can achieve high performance in computer vision by treating individual pixels as tokens, challenging the necessity of locality bias in modern architectures. https://arxiv.org/abs//2406.09415 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/p…

1
[QA] Large Language Models Must Be Taught to Know What They Don't Know 10:14

13d ago10:14

10:14

Prompting alone is insufficient for reliable uncertainty estimation in large language models. Fine-tuning on a small dataset of correct and incorrect answers can provide better calibration with low computational cost. https://arxiv.org/abs//2406.08391 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple P…

1
Large Language Models Must Be Taught to Know What They Don't Know 14:43

13d ago14:43

14:43

Prompting alone is insufficient for reliable uncertainty estimation in large language models. Fine-tuning on a small dataset of correct and incorrect answers can provide better calibration with low computational cost. https://arxiv.org/abs//2406.08391 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple P…

Podcasts der er værd at lytte til

Igor Melnyk Podcasts

Podcasts der er værd at lytte til

Hurtig referencevejledning