Home
Artists
Search Recent Random
Posts
Search DMs Popular Hash Lookup Tags Random
Importer
Import FAQ
Register
Partychan Matrix ThePornDude
Home Artists Posts Import Register
The Offical Matrix Groupchat is online! >>CLICK HERE<<

Sébastien Bubeck - Exclusive Interview (Patreon)

Published:
2023-12-17 19:42:06
Edited:
2023-12-18 08:57:02
Imported:

Downloads

Content

An exclusive interview with Microsoft's Sébastien Bubeck, co-author of Sparks of AGI and the Phi series of models. From practical tips to AGI timelines, AI on a phone to whether models can reason, it was a great chat and it even ended on a powerful note regarding the future of scaling language models.

Exclusive Interview +

Bubeck: https://en.m.wikipedia.org/wiki/S%C3%A9bastien_Bubeck

Phi-2: https://www.microsoft.com/en-us/research/blog/phi-2-the-surprising-power-of-small-language-models/

Textbooks are All You Need: https://www.microsoft.com/en-us/research/publication/textbooks-are-all-you-need/

Microsoft Prompting: https://www.microsoft.com/en-us/research/blog/steering-at-the-frontier-extending-the-power-of-prompting/

MedPrompt: https://www.microsoft.com/en-us/research/blog/the-power-of-prompting/

H100s and B100s: https://www.tomshardware.com/tech-industry/nvidia-ai-and-hpc-gpu-sales-reportedly-approached-half-a-million-units-in-q3-thanks-to-meta-facebook

https://www.digitimes.com/news/a20231215PD209/microsoft-nvidia-amd-h100-b100-ai-it+ce-it-components-peripherals-llm.html

AI Explained Phi-2: https://youtu.be/nPgs8THgbuI?si=MgiUS1HikYSR0IJi

TinyGSM (paper he had just published): https://arxiv.org/abs/2312.09241

Comments

Christo Wilken

Thank you, good interview! Add around 18:43 you say something about a “metprompt thing that Microsoft did”. I couldn’t quite make it out. Could you clarify what your meant by that?

AIExplained
>>124758069

Thanks Christo! It's MedPrompt: https://www.microsoft.com/en-us/research/blog/the-power-of-prompting/, plus this: https://www.microsoft.com/en-us/research/blog/steering-at-the-frontier-extending-the-power-of-prompting/

Younesse Kaddar

Thanks for the interview! In case others were wondering, the SLM + verifier paper he mentioned: https://arxiv.org/abs/2312.09241

GGuy

I find it interesting that the evidence hints that synthetic data could assist in runaway progress, at least in the short term. Yet he was reluctant to speculate on "AGI" dates. 🤔

Shaun McDonogh

Fascinating work. Thank you for this.

Michal Babula

Great interview - thank you :) Based on my experience, I can achieve significant improvement in LMM's output by providing high-quality data for retrieval (which I prepare myself; instead of a full 200-page book, I create a 20-page draft containing the most important information). I totally agree with what Sebastian said.

Christian Nofziger

Loved this one. Really incredible how you’re pushing the conversations forward Phillip

Jonathan Kirk

All of this is incredibly fascinating, but quite terrifying. I just finished my first semester at college, but with all the rampant progress, I am unsure what to do. How does someone make a decision about the future in today's age? Also I absolutely love your videos, as it is so far the clearest and most objective un-clickbait information I can find all wrapped up into a digestible format👍

Daniel Schönbohm

I love the direction in which he is working! The performance seems quite interesting. Are there any good performance tests of Phi-2 for code generation that someone could share? I'm fascinated by the idea of using SLMs to quickly generate less than 150 lines of code from text with less than 2K tokens, plus technologies like etched.ai (mentioned in the latest AI Explained YT vid).

Michal Babula
>>124758069

Christo, I created prompt based on MedPrompt methodology. I tested it on the most challanging diagnostic cases from NEJM magazine. The performence was amazing - more accurate that most physicians that participated into challanges. In Microsoft"s publications you do not find the examples of used prompts. Below this is my version: Task: Diagnose the disease based on symptoms from provided [patient history] . Recall and select examples of similar cases you were trained on. Now, based on the selected examples, describe your reasoning process. Step 1: Identify commonalities in symptoms among the selected cases. Step 2: Consider possible diagnoses for these symptoms. Step 3: Draw a conclusion about the most likely diagnosis. Potential Diagnoses: D1, D2, D3. Shuffle these diagnoses in different orders and create a reasoning path for each order. Finally, determine the most consistent diagnosis across these paths. [patient history] = '''...'''

Andrew Walker

Have you ever done a deep dive on the concept of toxicity and bias itself? Whenever it’s discussed it always occurs to me that it’s not at all trivial to be objective of what is bias and what is just data. Is the only way to meet one observers notion of non-bias alignment with all of their own biases?

Shawn Fumo
>>124834293

Maybe one thing to take solace in is that in a lot of ways we’re all in the same boat. As a senior dev, I have an advantage in the very near term in that maybe there will be less interns and juniors. But at the same time, not all older people will do that, and it also empowers people who are starting out to learn and do more. And if my job is totally automated, tons of others will be too by then. Maybe we do UBI, though I have to think there is a very rough transition. I don’t know your major but if it is dev focused, I think I’d try to learn as much ML as you can (Karpathy’s zero to hero vids are a great dev intro) vs making websites. But yeah, I feel like a crazy person sometimes. If you aren’t following all of what is going on, it hard to have any idea of the scope. Not just LLMs but everything else in images, video, voice, robotics.

Shawn Fumo
>>124834293

Also, while easier said than done, a perspective shift can help a little. I’ve been doing meditation for a while now and between Buddhism and other sources (there’s a recent interesting book called 4000 Weeks which presents as a time management book but really more about philosophy), a big source of stress is trying to control the future. AI makes it more obvious, but don’t need that for the future to be uncertain. Could have a car accident tomorrow and be paralyzed. Could meet the person of your dreams and move with them to another state. Doesn’t mean we shouldn’t plan at all, but often we need to let go a little too and be ready to pivot as opportunities come up.