Subscribe on your fav podcast app: https://www.vscodepodcast.com/subscribe

In this episode of the VS Code Insiders podcast, James sits down with Julia to explore the fast-changing world of AI models inside VS Code. They dive into how developers can evaluate models, the role of fine-tuning, and why speed versus complexity matters. From Raptor Mini to Sonnet, this conversation reveals the behind-the-scenes work shaping your coding experience today.

Raptor Mini: https://github.blog/changelog/2025-11-10-raptor-mini-is-rolling-out-in-public-preview-for-github-copilot/

Chapters:
00:00 Welcome & introductions
00:51 Diving into models and evaluations
02:25 Choosing models: gut feel vs. science
04:47 Why models feel different week to week
07:17 Fine-tuning explained & checkpoints
09:49 The story behind Raptor Mini
12:21 When to use mini vs. larger models
15:12 Context, complexity, and developer workflows
16:47 Understanding evaluations (evals)
19:35 Benchmarks & VS Code’s own eval suite
21:15 How developers can give feedback
23:34 Closing thoughts & subscribe reminder

Subscribe on your favorite podcast app:
- Apple Podcasts: https://podcasts.apple.com/us/podcast/id1833924784
- Spotify: https://open.spotify.com/show/3S2fExHkmbfQwwYw4a56yQ
- Amazon Music: https://music.amazon.com/podcasts/bd5f1efc-cdae-49c8-8ec7-c9b48b00ce46/vs-code-insiders-podcast
- Castbox: https://castbox.fm/channel/id6720052?country=us
- Overcast: https://overcast.fm/itunes1833924784/vs-code-insiders-podcast
- Pocket Casts: https://pca.st/itunes/1833924784
- and more: https://www.vscodepodcast.com/subscribe

Follow VS Code:
* X: https://x.com/code
* Bluesky: https://bsky.app/profile/vscode.dev
* YouTube: https://youtube.com/code
* LinkedIn: https://www.linkedin.com/showcase/104107263
* GitHub: https://github.com/microsoft/vscode

#vscode #ai #agents

Visual Studio Code is a lightweight and powerful code editor developed by Microsoft, offering built-in support for multiple programming languages, debugging, and extensions. From this resource developers can learn about VSCode's features, extensions, and customization options to enhance their coding productivity and workflow.

Visual Studio Code

VS Code PM Julia Kasper discusses model selection, evaluation strategies, and the new Raptor Mini model. She explains how VS Code continuously improves AI coding assistants through custom prompts, fine-tuning on developer workflows, and both online and offline evaluations. Raptor Mini is a fine-tuned GPT-4o mini optimized for speed and repetitive tasks, offering zero token cost. The team uses VC Bench (an in-house evaluation suite) alongside public benchmarks to measure model performance, token usage, and tool calling behavior specific to the IDE environment.

Models, Evals, and Raptor Mini with Julia Kasper