So I've been testing Gemini 4...

It finally happened. Gemini 4 Argon is It finally happened. Gemini 4 Argon is here. Before we get started, I got a here. Before we get started, I got a here. Before we get started, I got a quick question. How long do you think it quick question. How long do you think it quick question. How long do you think it has been since we got a normal-sized has been since we got a normal-sized has been since we got a normal-sized large Gemini model release, like the Pro large Gemini model release, like the Pro large Gemini model release, like the Pro line? How many months do you think it line? How many months do you think it line? How many months do you think it has been? The gap between a model has been? The gap between a model has been? The gap between a model release like 5 6 Soul and 6 Astra was 1 release like 5 6 Soul and 6 Astra was 1 release like 5 6 Soul and 6 Astra was 1 to 2 months. The gap between 3 1 Pro and to 2 months. The gap between 3 1 Pro and to 2 months. The gap between 3 1 Pro and Gemini 4 Argon is 8 months. It's been Gemini 4 Argon is 8 months. It's been Gemini 4 Argon is 8 months. It's been since February. We have not gotten since February. We have not gotten since February. We have not gotten anything other than flash models from anything other than flash models from anything other than flash models from Google. And now, after teasing Gemini 4 Google. And now, after teasing Gemini 4 Google. And now, after teasing Gemini 4 Argon over a week ago, they have finally Argon over a week ago, they have finally Argon over a week ago, they have finally put it out for all of us to use. With a put it out for all of us to use. With a put it out for all of us to use. With a drop like this, there's a lot we have to drop like this, there's a lot we have to drop like this, there's a lot we have to cover, from how it's priced to how good cover, from how it's priced to how good cover, from how it's priced to how good it is at front end, to all the 3D it is at front end, to all the 3D it is at front end, to all the 3D capabilities, the weird things that make capabilities, the weird things that make capabilities, the weird things that make Gemini models Gemini models, the unique Gemini models Gemini models, the unique Gemini models Gemini models, the unique capabilities in those models, and of capabilities in those models, and of capabilities in those models, and of course, the comparison with recent course, the comparison with recent course, the comparison with recent releases like Opus 5.5, GPT 6 1 Soul, releases like Opus 5.5, GPT 6 1 Soul, releases like Opus 5.5, GPT 6 1 Soul, and Muse Wonderful, which if you've seen and Muse Wonderful, which if you've seen and Muse Wonderful, which if you've seen my video on that, I am quite enjoying. A my video on that, I am quite enjoying. A my video on that, I am quite enjoying. A handful of my friends at Google actually handful of my friends at Google actually handful of my friends at Google actually stuck their necks out a bit to get me stuck their necks out a bit to get me stuck their necks out a bit to get me early access to this model. I'm very early access to this model. I'm very early access to this model. I'm very thankful for them, and I sincerely hope thankful for them, and I sincerely hope thankful for them, and I sincerely hope they don't get in trouble for what I'm they don't get in trouble for what I'm they don't get in trouble for what I'm going to say. There are things that are going to say. There are things that are going to say. There are things that are good about this model, and there are good about this model, and there are good about this model, and there are things that are less good about this things that are less good about this things that are less good about this model. Normally, this is where I'd make model. Normally, this is where I'd make model. Normally, this is where I'd make a joke about how I'm not a paid shill a joke about how I'm not a paid shill a joke about how I'm not a paid shill and the company isn't paying me, but in and the company isn't paying me, but in and the company isn't paying me, but in this particular case, I can't because this particular case, I can't because this particular case, I can't because Google does pay me quite a bit because Google does pay me quite a bit because Google does pay me quite a bit because I'm a YouTuber, as you might know, I'm a YouTuber, as you might know, I'm a YouTuber, as you might know, watching this video likely on YouTube.

watching this video likely on YouTube. watching this video likely on YouTube. All of that said, they are not paying me All of that said, they are not paying me All of that said, they are not paying me for my coverage for anti-gravity and for my coverage for anti-gravity and for my coverage for anti-gravity and Gemini stuff. In fact, my previous Gemini stuff. In fact, my previous Gemini stuff. In fact, my previous Gemini videos almost got me demonetized Gemini videos almost got me demonetized Gemini videos almost got me demonetized on YouTube, so this comes at a cost to on YouTube, so this comes at a cost to on YouTube, so this comes at a cost to me, if anything, which is why I hope you me, if anything, which is why I hope you me, if anything, which is why I hope you can forgive us for a quick break for can forgive us for a quick break for can forgive us for a quick break for today's sponsor. I got an interesting today's sponsor. I got an interesting today's sponsor. I got an interesting question for you. You want people like question for you. You want people like question for you. You want people like me to use your product because if you me to use your product because if you me to use your product because if you do, you probably shouldn't do a normal do, you probably shouldn't do a normal do, you probably shouldn't do a normal sign-in flow because I'm not going to be sign-in flow because I'm not going to be sign-in flow because I'm not going to be clicking those buttons myself nowadays. clicking those buttons myself nowadays. clicking those buttons myself nowadays. I do most of my work through my agents I do most of my work through my agents I do most of my work through my agents now, which means any platform that my now, which means any platform that my now, which means any platform that my agents choose needs to be usable by the agents choose needs to be usable by the agents choose needs to be usable by the agent. In particular, it needs to be agent. In particular, it needs to be agent. In particular, it needs to be able to be signed up for by your agents. able to be signed up for by your agents. able to be signed up for by your agents. Today's sponsor is WorkOS, and that Today's sponsor is WorkOS, and that Today's sponsor is WorkOS, and that might confuse you because they are a might confuse you because they are a might confuse you because they are a platform for user signing in, right? platform for user signing in, right? platform for user signing in, right? Well, they are. They are the best way to Well, they are. They are the best way to Well, they are. They are the best way to have humans and businesses signing into have humans and businesses signing into have humans and businesses signing into your app. So, if you're trying to get your app. So, if you're trying to get your app. So, if you're trying to get enterprise customers, then you should enterprise customers, then you should enterprise customers, then you should probably be using WorkOS. That's why probably be using WorkOS. That's why probably be using WorkOS. That's why everyone from OpenAI and Anthropic to everyone from OpenAI and Anthropic to everyone from OpenAI and Anthropic to Fowl, Cursor, Bolt, Vanta, Carta, and Fowl, Cursor, Bolt, Vanta, Carta, and Fowl, Cursor, Bolt, Vanta, Carta, and more are all using them. But, I want to more are all using them. But, I want to more are all using them. But, I want to talk about AuthMD because this is a talk about AuthMD because this is a talk about AuthMD because this is a super, super cool standard. The goal of super, super cool standard. The goal of super, super cool standard. The goal of AuthMD is to make it easier for agents AuthMD is to make it easier for agents AuthMD is to make it easier for agents to sign up for your apps. It's a new to sign up for your apps. It's a new to sign up for your apps. It's a new standard that's been designed by WorkOS standard that's been designed by WorkOS standard that's been designed by WorkOS in partnership with companies like Fire in partnership with companies like Fire in partnership with companies like Fire Crawl and Cloudflare. And the results Crawl and Cloudflare. And the results Crawl and Cloudflare. And the results speak for themselves. There's already a speak for themselves. There's already a speak for themselves. There's already a ton of companies adopting these ton of companies adopting these ton of companies adopting these standards like Neon, Monday, Parallel, standards like Neon, Monday, Parallel, standards like Neon, Monday, Parallel, and more. What makes it so cool is that and more. What makes it so cool is that and more. What makes it so cool is that agents can now register on behalf of a agents can now register on behalf of a agents can now register on behalf of a user. So, if you're trying to get user. So, if you're trying to get user. So, if you're trying to get business customers and you want their business customers and you want their business customers and you want their agents signing up, this is a great place agents signing up, this is a great place agents signing up, this is a great place to start. Get Auth Right for all your to start. Get Auth Right for all your to start. Get Auth Right for all your users at swyd.link/workos.

users at swyd.link/workos. users at swyd.link/workos. Now that that's out of the way, let's Now that that's out of the way, let's Now that that's out of the way, let's dive into the newest model from Google, dive into the newest model from Google, dive into the newest model from Google, Gemini 4 Argon. Flash banging warning Gemini 4 Argon. Flash banging warning Gemini 4 Argon. Flash banging warning for their website because it is not dark for their website because it is not dark for their website because it is not dark mode for some reason. Argon is now mode for some reason. Argon is now mode for some reason. Argon is now globally available today for developers globally available today for developers globally available today for developers and enterprises via the API and studio and enterprises via the API and studio and enterprises via the API and studio and Gemini platform, Google AI Ultra and Gemini platform, Google AI Ultra and Gemini platform, Google AI Ultra subscribers and anti-gravity, and Google subscribers and anti-gravity, and Google subscribers and anti-gravity, and Google AI Ultra and Pro subs in the Gemini app AI Ultra and Pro subs in the Gemini app AI Ultra and Pro subs in the Gemini app and AI mode in search. This is the God, and AI mode in search. This is the God, and AI mode in search. This is the God, I did not realize how bad Google's like I did not realize how bad Google's like I did not realize how bad Google's like products line had gotten until this. products line had gotten until this. products line had gotten until this. It's a mess. Model excels at deep work It's a mess. Model excels at deep work It's a mess. Model excels at deep work in complex real-world tasks across in complex real-world tasks across in complex real-world tasks across domains, including coding, finance, domains, including coding, finance, domains, including coding, finance, legal, cybersecurity, and science. Its legal, cybersecurity, and science. Its legal, cybersecurity, and science. Its depth and precision across multi-step depth and precision across multi-step depth and precision across multi-step tasks lead Argon to outperform other tasks lead Argon to outperform other tasks lead Argon to outperform other frontier models on benchmarks across frontier models on benchmarks across frontier models on benchmarks across domains, ranking number one on the VALS domains, ranking number one on the VALS domains, ranking number one on the VALS index, which measures model performance index, which measures model performance index, which measures model performance across critical economic sectors like across critical economic sectors like across critical economic sectors like finance, coding, legal, and tax work. finance, coding, legal, and tax work. finance, coding, legal, and tax work. And yes, indeed, Gemini 4 Argon does And yes, indeed, Gemini 4 Argon does And yes, indeed, Gemini 4 Argon does appear to be one of the best values appear to be one of the best values appear to be one of the best values here. Somehow they have a cost for here. Somehow they have a cost for here. Somehow they have a cost for wonderful, wonderful, wonderful, though, which is strange cuz I was never though, which is strange cuz I was never though, which is strange cuz I was never given token pricing. Okay, they're given token pricing. Okay, they're given token pricing. Okay, they're saying it's $4.00 in and $20.00 out.

saying it's $4.00 in and $20.00 out. saying it's $4.00 in and $20.00 out. That's wrong. We don't have a cost for That's wrong. We don't have a cost for That's wrong. We don't have a cost for wonderful, and it won't be the same wonderful, and it won't be the same wonderful, and it won't be the same price as standard wonder, obviously. So, price as standard wonder, obviously. So, price as standard wonder, obviously. So, uh that makes me trust this a lot less. uh that makes me trust this a lot less. uh that makes me trust this a lot less. I'll be real. Regardless, I have my own I'll be real. Regardless, I have my own I'll be real. Regardless, I have my own benches. I've I've run a lot of benches benches. I've I've run a lot of benches benches. I've I've run a lot of benches on this model. I'll start with some on this model. I'll start with some on this model. I'll start with some benches that aren't mine. With Slot benches that aren't mine. With Slot benches that aren't mine. With Slot Politics, I already shared the numbers Politics, I already shared the numbers Politics, I already shared the numbers for Gemini here. You can see that its for Gemini here. You can see that its for Gemini here. You can see that its cost is not actually that great when you cost is not actually that great when you cost is not actually that great when you factor in how many tokens it uses on factor in how many tokens it uses on factor in how many tokens it uses on something like the Artificial Analysis something like the Artificial Analysis something like the Artificial Analysis Intelligence Index, but its score is Intelligence Index, but its score is Intelligence Index, but its score is putting it above Astra and also above 61 putting it above Astra and also above 61 putting it above Astra and also above 61 Soul, but meaningfully lower than Opus Soul, but meaningfully lower than Opus Soul, but meaningfully lower than Opus 5.5 on high as well as obviously X high 5.5 on high as well as obviously X high 5.5 on high as well as obviously X high and max. And it is also more expensive and max. And it is also more expensive and max. And it is also more expensive than Opus 5.5 on high despite getting a than Opus 5.5 on high despite getting a than Opus 5.5 on high despite getting a lower score. Of course, Artificial lower score. Of course, Artificial lower score. Of course, Artificial Analysis Intelligence Index is not a Analysis Intelligence Index is not a Analysis Intelligence Index is not a perfect measure of model capabilities. perfect measure of model capabilities. perfect measure of model capabilities. It is not the best index for things like It is not the best index for things like It is not the best index for things like code, but it is a good general idea of code, but it is a good general idea of code, but it is a good general idea of where intelligence sets. I chose to show where intelligence sets. I chose to show where intelligence sets. I chose to show you these numbers first because I'm you these numbers first because I'm you these numbers first because I'm scared people's brains fall out whenever scared people's brains fall out whenever scared people's brains fall out whenever I show them the model price. Because if I show them the model price. Because if I show them the model price. Because if you see this $2 per mil in and $10 per you see this $2 per mil in and $10 per you see this $2 per mil in and $10 per mil out, your immediate assumption's mil out, your immediate assumption's mil out, your immediate assumption's going to be this model is cheap when you going to be this model is cheap when you going to be this model is cheap when you compare it to things like the 5.25 rates compare it to things like the 5.25 rates compare it to things like the 5.25 rates for Opus 5.5. This seems like a really for Opus 5.5. This seems like a really for Opus 5.5. This seems like a really good deal. It's not. It's a combination good deal. It's not. It's a combination good deal. It's not. It's a combination of reasons. The first one is that this of reasons. The first one is that this of reasons. The first one is that this model is it's token hungry. It's not model is it's token hungry. It's not model is it's token hungry. It's not particularly efficient, which is part of particularly efficient, which is part of particularly efficient, which is part of what puts it in the higher price range what puts it in the higher price range what puts it in the higher price range like we just saw on Artificial Analysis.

like we just saw on Artificial Analysis. like we just saw on Artificial Analysis. The other really, really egregious The other really, really egregious The other really, really egregious problem is that it sucks at hitting problem is that it sucks at hitting problem is that it sucks at hitting cash. I have had the worst rate of cash cash. I have had the worst rate of cash cash. I have had the worst rate of cash reads from any model I've used recently reads from any model I've used recently reads from any model I've used recently with this release. I'm talking like 70 with this release. I'm talking like 70 with this release. I'm talking like 70 to 80% cash hit versus other models to 80% cash hit versus other models to 80% cash hit versus other models where I see 99% plus. It means that that where I see 99% plus. It means that that where I see 99% plus. It means that that 95% discount on cash reads is not going 95% discount on cash reads is not going 95% discount on cash reads is not going to happen as often and you're going to to happen as often and you're going to to happen as often and you're going to actually eat a lot of input cost on this actually eat a lot of input cost on this actually eat a lot of input cost on this model. This might be a bug in model. This might be a bug in model. This might be a bug in Anti-Gravity either in how it's actually Anti-Gravity either in how it's actually Anti-Gravity either in how it's actually sending things or how it's reporting sending things or how it's reporting sending things or how it's reporting data to me. It is slightly possible this data to me. It is slightly possible this data to me. It is slightly possible this is a bug in the MCP implementation that is a bug in the MCP implementation that is a bug in the MCP implementation that we use in T3 Code because you will never we use in T3 Code because you will never we use in T3 Code because you will never catch me dead in their awful CLI, but I catch me dead in their awful CLI, but I catch me dead in their awful CLI, but I am not particularly confident in any of am not particularly confident in any of am not particularly confident in any of this and I am not interested in doing this and I am not interested in doing this and I am not interested in doing more work to figure it out because more work to figure it out because more work to figure it out because Google should be the ones doing this, Google should be the ones doing this, Google should be the ones doing this, It's me. They have a section about using It's me. They have a section about using It's me. They have a section about using it to solve complex engineering tasks it to solve complex engineering tasks it to solve complex engineering tasks saying that uh anti-gravity has helped saying that uh anti-gravity has helped saying that uh anti-gravity has helped teams increase agentic code submissions teams increase agentic code submissions teams increase agentic code submissions by more than 35% while reducing code by more than 35% while reducing code by more than 35% while reducing code rollbacks by nearly a third. rollbacks by nearly a third. rollbacks by nearly a third. Those are not great numbers considering Those are not great numbers considering Those are not great numbers considering that what they were using before was that what they were using before was that what they were using before was also not particularly good. The more also not particularly good. The more also not particularly good. The more interesting thing and I won't air out interesting thing and I won't air out interesting thing and I won't air out who this was, but I was talking to a who this was, but I was talking to a who this was, but I was talking to a Google employee in not one of the ones Google employee in not one of the ones Google employee in not one of the ones that Google knows I talked to, so don't that Google knows I talked to, so don't that Google knows I talked to, so don't go after the people who normally work go after the people who normally work go after the people who normally work with me. The way they described Argon to with me. The way they described Argon to with me. The way they described Argon to me wasn't it's a revolution or this me wasn't it's a revolution or this me wasn't it's a revolution or this model has changed how we build. The way model has changed how we build. The way model has changed how we build. The way it was described to me is I find myself it was described to me is I find myself it was described to me is I find myself reaching for Opus 5 less. Not 5.5, Opus reaching for Opus 5 less. Not 5.5, Opus reaching for Opus 5 less. Not 5.5, Opus 5. The one that I notoriously panned 5. The one that I notoriously panned 5. The one that I notoriously panned after realizing I was wrong in my first

after realizing I was wrong in my first after realizing I was wrong in my first video. It is what it is. Let's see what video. It is what it is. Let's see what video. It is what it is. Let's see what this can do at least according to this can do at least according to this can do at least according to Google. Apparently they use it to Google. Apparently they use it to Google. Apparently they use it to untangle browser feature code in Chrome. untangle browser feature code in Chrome. untangle browser feature code in Chrome. Piping multiple clean code changes at a Piping multiple clean code changes at a Piping multiple clean code changes at a time, Argon helped the team finish an time, Argon helped the team finish an time, Argon helped the team finish an architectural refactor across hundreds architectural refactor across hundreds architectural refactor across hundreds of files completing months of work in of files completing months of work in of files completing months of work in weeks. This feels like it was written in weeks. This feels like it was written in weeks. This feels like it was written in 2024. It optimized a production Python 2024. It optimized a production Python 2024. It optimized a production Python code base in just a few hours. The code base in just a few hours. The code base in just a few hours. The Here they have some fun demos of the Here they have some fun demos of the Here they have some fun demos of the cool 3D stuff they've been doing with cool 3D stuff they've been doing with cool 3D stuff they've been doing with it. it. it. God, why is it making sounds? That's God, why is it making sounds? That's God, why is it making sounds? That's horrible. horrible. horrible. This isn't a generated video. This is This isn't a generated video. This is This isn't a generated video. This is generated 3D models that they then went generated 3D models that they then went generated 3D models that they then went through with some type of like software through with some type of like software through with some type of like software rotoscope type thing to show the models rotoscope type thing to show the models rotoscope type thing to show the models in video form. in video form. in video form. Very cool. Animations are still a bit Very cool. Animations are still a bit Very cool. Animations are still a bit cringe. cringe. cringe. This YouTube player embed is broken and This YouTube player embed is broken and This YouTube player embed is broken and the overlay won't disappear. the overlay won't disappear. the overlay won't disappear. Classic Google-isms. I I've joked for a Classic Google-isms. I I've joked for a Classic Google-isms. I I've joked for a bit now, you'll know when Google has a bit now, you'll know when Google has a bit now, you'll know when Google has a good model when their software finally good model when their software finally good model when their software finally starts working. They got a quote from starts working. They got a quote from starts working. They got a quote from someone at Deloitte. Early impressions someone at Deloitte. Early impressions someone at Deloitte. Early impressions from Gemini 4 are very strong. For from Gemini 4 are very strong. For from Gemini 4 are very strong. For long-running tasks across these long-running tasks across these long-running tasks across these workflows, it completed successfully workflows, it completed successfully workflows, it completed successfully 100% of the time in our e-vals, 100% of the time in our e-vals, 100% of the time in our e-vals, maintaining context and running through maintaining context and running through maintaining context and running through multiple tools across impressive spans.

multiple tools across impressive spans. multiple tools across impressive spans. Instead of the reasoning delivers Instead of the reasoning delivers Instead of the reasoning delivers significantly higher, clearer, yada significantly higher, clearer, yada significantly higher, clearer, yada yada, the text clips. yada, the text clips. yada, the text clips. >> [sighs] >> [sighs] >> [sighs] >> Google being Google. I shouldn't be as >> Google being Google. I shouldn't be as >> Google being Google. I shouldn't be as annoyed as I am by this completed annoyed as I am by this completed annoyed as I am by this completed successfully 100% of the time in our successfully 100% of the time in our successfully 100% of the time in our evals thing because this depends evals thing because this depends evals thing because this depends entirely on what's being evaluated. For entirely on what's being evaluated. For entirely on what's being evaluated. For example, I have an eval that Gemini example, I have an eval that Gemini example, I have an eval that Gemini completed 100% of the time. It's completed 100% of the time. It's completed 100% of the time. It's Skatebench. I still actually can't Skatebench. I still actually can't Skatebench. I still actually can't believe this. I think it's cool as hell. believe this. I think it's cool as hell. believe this. I think it's cool as hell. Gemini for Argon got a perfect score on Gemini for Argon got a perfect score on Gemini for Argon got a perfect score on my private version of my benchmark for my private version of my benchmark for my private version of my benchmark for seeing how well models name skateboard seeing how well models name skateboard seeing how well models name skateboard tricks. For reference, Opus 5.5 on Max tricks. For reference, Opus 5.5 on Max tricks. For reference, Opus 5.5 on Max gets a 79%. This bench has proven to be gets a 79%. This bench has proven to be gets a 79%. This bench has proven to be weirdly difficult for most models to do. weirdly difficult for most models to do. weirdly difficult for most models to do. In particular, some of the tricks that In particular, some of the tricks that In particular, some of the tricks that are like switch and fakie stance cuz are like switch and fakie stance cuz are like switch and fakie stance cuz skateboarding is a whole grammar around skateboarding is a whole grammar around skateboarding is a whole grammar around the way you're standing on the board. the way you're standing on the board. the way you're standing on the board. Google has always had a weird lead here Google has always had a weird lead here Google has always had a weird lead here with models like 3.8 flash getting a 96% with models like 3.8 flash getting a 96% with models like 3.8 flash getting a 96% but seeing the first perfect score was but seeing the first perfect score was but seeing the first perfect score was kind of trippy for me. Especially when I kind of trippy for me. Especially when I kind of trippy for me. Especially when I looked into it more and saw just how looked into it more and saw just how looked into it more and saw just how many errors it was hitting when it ran many errors it was hitting when it ran many errors it was hitting when it ran because the API was quite unstable because the API was quite unstable because the API was quite unstable during my early testing. I don't know if during my early testing. I don't know if during my early testing. I don't know if that's improved since but I was getting that's improved since but I was getting that's improved since but I was getting pretty intense error rates and I had to pretty intense error rates and I had to pretty intense error rates and I had to overhaul a bunch of my benchmark just to overhaul a bunch of my benchmark just to overhaul a bunch of my benchmark just to get around the fact that the endpoints get around the fact that the endpoints get around the fact that the endpoints error as often as they did. It was also error as often as they did. It was also error as often as they did. It was also relatively impressive in terms of its relatively impressive in terms of its relatively impressive in terms of its token utilization in the bench like it token utilization in the bench like it token utilization in the bench like it got answers fast and didn't overthink got answers fast and didn't overthink got answers fast and didn't overthink which is rare for a Gemini model and which is rare for a Gemini model and which is rare for a Gemini model and nice to see. So, yeah, if your task is nice to see. So, yeah, if your task is nice to see. So, yeah, if your task is somehow reminiscent to naming skateboard somehow reminiscent to naming skateboard somehow reminiscent to naming skateboard tricks, there's a decent chance you're tricks, there's a decent chance you're tricks, there's a decent chance you're going to have a good experience with going to have a good experience with going to have a good experience with Argon both in its like speed, token Argon both in its like speed, token Argon both in its like speed, token efficiency, but also its accuracy in

efficiency, but also its accuracy in efficiency, but also its accuracy in doing the thing. But again, when you say doing the thing. But again, when you say doing the thing. But again, when you say a model has 100% success in our evals, a model has 100% success in our evals, a model has 100% success in our evals, the eval and its relation to the work the eval and its relation to the work the eval and its relation to the work you're doing matters and I don't think a you're doing matters and I don't think a you're doing matters and I don't think a lot of people are making a living naming lot of people are making a living naming lot of people are making a living naming skateboard tricks. I wish they were. I skateboard tricks. I wish they were. I skateboard tricks. I wish they were. I wish I could have that be my job but uh wish I could have that be my job but uh wish I could have that be my job but uh sadly it is not. They also say that the sadly it is not. They also say that the sadly it is not. They also say that the model has advanced reasoning and model has advanced reasoning and model has advanced reasoning and multimodal capabilities while multimodal capabilities while multimodal capabilities while maintaining strict grounding, yada yada maintaining strict grounding, yada yada maintaining strict grounding, yada yada yada. It's good at looking at documents, yada. It's good at looking at documents, yada. It's good at looking at documents, PDFs, videos, audio, things like that. I PDFs, videos, audio, things like that. I PDFs, videos, audio, things like that. I have historically seen a meaningful lead have historically seen a meaningful lead have historically seen a meaningful lead in Gemini models' ability to like do in Gemini models' ability to like do in Gemini models' ability to like do things with vision. Like giving it a things with vision. Like giving it a things with vision. Like giving it a picture of a motherboard and asking it picture of a motherboard and asking it picture of a motherboard and asking it to label the parts and draw that in like to label the parts and draw that in like to label the parts and draw that in like Python on top of it. The Google models Python on top of it. The Google models Python on top of it. The Google models have historically been very impressive have historically been very impressive have historically been very impressive in this space. I did not benchmark any in this space. I did not benchmark any in this space. I did not benchmark any of that because I was just using it for of that because I was just using it for of that because I was just using it for code. But if you have complex vision code. But if you have complex vision code. But if you have complex vision tasks, it is definitely worth running tasks, it is definitely worth running tasks, it is definitely worth running them against Gemini 4 just to see if you them against Gemini 4 just to see if you them against Gemini 4 just to see if you have a real benefit. You might be have a real benefit. You might be have a real benefit. You might be surprised. Do note though, as we start surprised. Do note though, as we start surprised. Do note though, as we start to complain more about Googleisms, that to complain more about Googleisms, that to complain more about Googleisms, that the current price of $2 per mil in and the current price of $2 per mil in and the current price of $2 per mil in and $10 per mil out is an introductory $10 per mil out is an introductory $10 per mil out is an introductory price. This is Google's new favorite price. This is Google's new favorite price. This is Google's new favorite thing to do when they drop a model. They thing to do when they drop a model. They thing to do when they drop a model. They charge a reasonable price up until a charge a reasonable price up until a charge a reasonable price up until a certain date, and then they double the certain date, and then they double the certain date, and then they double the price after. So this model's actually price after. So this model's actually price after. So this model's actually going to be $4 per mil in and $20 per going to be $4 per mil in and $20 per going to be $4 per mil in and $20 per mil out starting December 1st. It's mil out starting December 1st. It's mil out starting December 1st. It's almost like they don't want you to start almost like they don't want you to start almost like they don't want you to start building heavily on top of the model.

building heavily on top of the model. building heavily on top of the model. It's It's rough. They've been doing this It's It's rough. They've been doing this It's It's rough. They've been doing this for most of their model drops recently for most of their model drops recently for most of their model drops recently and it makes pricing weird. I I don't and it makes pricing weird. I I don't and it makes pricing weird. I I don't think it's as bad as what Meta did by think it's as bad as what Meta did by think it's as bad as what Meta did by not putting out a price at all for not putting out a price at all for not putting out a price at all for wonderful and only having it in the sub, wonderful and only having it in the sub, wonderful and only having it in the sub, but I still hate this. Just give us a but I still hate this. Just give us a but I still hate this. Just give us a price that you can actually serve it at. price that you can actually serve it at. price that you can actually serve it at. Lower the price in the future. Don't Lower the price in the future. Don't Lower the price in the future. Don't raise it. The only lab that regularly raise it. The only lab that regularly raise it. The only lab that regularly raises prices for models is our friends raises prices for models is our friends raises prices for models is our friends over at Google. And then they tell us to over at Google. And then they tell us to over at Google. And then they tell us to get started. So I guess we should. get started. So I guess we should. get started. So I guess we should. First, we'll start with one of the First, we'll start with one of the First, we'll start with one of the things Google models have historically things Google models have historically things Google models have historically been pretty strong at. The thing is of been pretty strong at. The thing is of been pretty strong at. The thing is of course front end. I have used Google course front end. I have used Google course front end. I have used Google models for front end stuff pretty often models for front end stuff pretty often models for front end stuff pretty often when they are the new thing. when they are the new thing. when they are the new thing. Specifically when 3 Pro and 3 1 Pro Specifically when 3 Pro and 3 1 Pro Specifically when 3 Pro and 3 1 Pro dropped, I found a lot of good uses. I dropped, I found a lot of good uses. I dropped, I found a lot of good uses. I built a lot of the UI and UX around built a lot of the UI and UX around built a lot of the UI and UX around Lawn, my alternative to Frame.io for Lawn, my alternative to Frame.io for Lawn, my alternative to Frame.io for video stuff. It's what we actually use video stuff. It's what we actually use video stuff. It's what we actually use for all of our video management for my for all of our video management for my for all of our video management for my company. Every one of my YouTube videos company. Every one of my YouTube videos company. Every one of my YouTube videos gets uploaded on this platform and gets uploaded on this platform and gets uploaded on this platform and managed by my team on Lawn, and a lot of managed by my team on Lawn, and a lot of managed by my team on Lawn, and a lot of the UI and UX here was built by Gemini 3 the UI and UX here was built by Gemini 3 the UI and UX here was built by Gemini 3 1 Pro. I obviously did a lot more work 1 Pro. I obviously did a lot more work 1 Pro. I obviously did a lot more work on top of that using the Opus models at on top of that using the Opus models at on top of that using the Opus models at the time, but I found that the back and the time, but I found that the back and the time, but I found that the back and forth between those models allowed me to forth between those models allowed me to forth between those models allowed me to make actually unique-ish homepages at make actually unique-ish homepages at make actually unique-ish homepages at the time. So, let's see how that the time. So, let's see how that the time. So, let's see how that capability has carried through. I think capability has carried through. I think capability has carried through. I think this is quite good. I don't necessarily this is quite good. I don't necessarily this is quite good. I don't necessarily think a homepage should have this much think a homepage should have this much think a homepage should have this much text on it, text on it, text on it, but it is very nice-looking. I would not but it is very nice-looking. I would not but it is very nice-looking. I would not have guessed that this was AI-generated have guessed that this was AI-generated have guessed that this was AI-generated if I just stumbled on a page like this if I just stumbled on a page like this if I just stumbled on a page like this like browsing the web. It also made a like browsing the web. It also made a like browsing the web. It also made a nice little interactive section here nice little interactive section here nice little interactive section here where you can do things that is actually where you can do things that is actually where you can do things that is actually functional and not cringe. Like, this is

functional and not cringe. Like, this is functional and not cringe. Like, this is decent. Believe me, I would talk a lot decent. Believe me, I would talk a lot decent. Believe me, I would talk a lot of crap if I could. This is fine. Hop of crap if I could. This is fine. Hop of crap if I could. This is fine. Hop into the other designs. That one had a into the other designs. That one had a into the other designs. That one had a bit of like font popping, that might bit of like font popping, that might bit of like font popping, that might just be the way that WhichAI is set up. just be the way that WhichAI is set up. just be the way that WhichAI is set up. By the way, WhichAI created by our By the way, WhichAI created by our By the way, WhichAI created by our friend Dara, really cool kid. Check it friend Dara, really cool kid. Check it friend Dara, really cool kid. Check it out whichai.dev. Very useful to see how out whichai.dev. Very useful to see how out whichai.dev. Very useful to see how the front-end capabilities vary across the front-end capabilities vary across the front-end capabilities vary across different models. All of the versions different models. All of the versions different models. All of the versions I'm showing now are using the official I'm showing now are using the official I'm showing now are using the official Anthropic design skill. So, if you Anthropic design skill. So, if you Anthropic design skill. So, if you aren't getting good designs out of the aren't getting good designs out of the aren't getting good designs out of the model, definitely try turning that on model, definitely try turning that on model, definitely try turning that on and see if you have better results. If I and see if you have better results. If I and see if you have better results. If I turn it off, you get a lot more cringe, turn it off, you get a lot more cringe, turn it off, you get a lot more cringe, a lot more of this like unnecessary text a lot more of this like unnecessary text a lot more of this like unnecessary text everywhere, a lot of fonts that are kind everywhere, a lot of fonts that are kind everywhere, a lot of fonts that are kind of gross, just not the best quality. of gross, just not the best quality. of gross, just not the best quality. Like, all of them have that like awful Like, all of them have that like awful Like, all of them have that like awful top nav, like every single one of them top nav, like every single one of them top nav, like every single one of them does. God, this retro one is so cringe. does. God, this retro one is so cringe. does. God, this retro one is so cringe. And this is without any skills on. So, And this is without any skills on. So, And this is without any skills on. So, if you are using this for design, highly if you are using this for design, highly if you are using this for design, highly recommend Anthropic design skill. I recommend Anthropic design skill. I recommend Anthropic design skill. I don't recommend the taste skill though. don't recommend the taste skill though. don't recommend the taste skill though. That gets it back to where it was with That gets it back to where it was with That gets it back to where it was with the weird cringe top navs and as well as the weird cringe top navs and as well as the weird cringe top navs and as well as like the really consistent gross layout.

like the really consistent gross layout. like the really consistent gross layout. The design skill seems to be a good The design skill seems to be a good The design skill seems to be a good balance that actually forces the model balance that actually forces the model balance that actually forces the model to make things that look decent and to make things that look decent and to make things that look decent and unique. So, for front-end, seems pretty unique. So, for front-end, seems pretty unique. So, for front-end, seems pretty good. Is it state-of-the-art? I still good. Is it state-of-the-art? I still good. Is it state-of-the-art? I still think Fable 5.1 is the best I've seen. think Fable 5.1 is the best I've seen. think Fable 5.1 is the best I've seen. Opus has its moments, but Fable overall Opus has its moments, but Fable overall Opus has its moments, but Fable overall makes the coolest, makes the coolest, makes the coolest, most unique front-ends and like most unique front-ends and like most unique front-ends and like homepages, marketing sites, that type of homepages, marketing sites, that type of homepages, marketing sites, that type of thing. thing. thing. I still don't think any model is good at I still don't think any model is good at I still don't think any model is good at just like doing things right the first just like doing things right the first just like doing things right the first try, and I prefer ones where I can and try, and I prefer ones where I can and try, and I prefer ones where I can and back and forth. For example, with back and forth. For example, with back and forth. For example, with Slapolitics, I used OpenAI models for Slapolitics, I used OpenAI models for Slapolitics, I used OpenAI models for this entire site. I did everything with this entire site. I did everything with this entire site. I did everything with Astra on ultrafast. Not because Astra on ultrafast. Not because Astra on ultrafast. Not because ultrafast is magically better at design, ultrafast is magically better at design, ultrafast is magically better at design, all OpenAI models kind of suck at all OpenAI models kind of suck at all OpenAI models kind of suck at design, but since I was using ultrafast, design, but since I was using ultrafast, design, but since I was using ultrafast, I could tell it what I wanted and ask I could tell it what I wanted and ask I could tell it what I wanted and ask for different versions and go back and for different versions and go back and for different versions and go back and forth until I got it just how I wanted forth until I got it just how I wanted forth until I got it just how I wanted without having to wait constantly without having to wait constantly without having to wait constantly throughout. So, I think that speed is throughout. So, I think that speed is throughout. So, I think that speed is really important when you're iterating really important when you're iterating really important when you're iterating on real UX, not like a marketing site, on real UX, not like a marketing site, on real UX, not like a marketing site, but like the buttons people press on the but like the buttons people press on the but like the buttons people press on the website and how they interface with it. website and how they interface with it. website and how they interface with it. Google models are fast, but that doesn't Google models are fast, but that doesn't Google models are fast, but that doesn't necessarily mean they get work done necessarily mean they get work done necessarily mean they get work done quickly. Anecdotally, having tested this quickly. Anecdotally, having tested this quickly. Anecdotally, having tested this a bunch myself, it's not a super fast a bunch myself, it's not a super fast a bunch myself, it's not a super fast model. It might generate tokens quickly, model. It might generate tokens quickly, model. It might generate tokens quickly, but it doesn't solve problems fast.

but it doesn't solve problems fast. but it doesn't solve problems fast. Next, we should talk a bit about the Next, we should talk a bit about the Next, we should talk a bit about the unique capabilities of Google models. unique capabilities of Google models. unique capabilities of Google models. This is strange because every lab kind This is strange because every lab kind This is strange because every lab kind of has something like this. For example, of has something like this. For example, of has something like this. For example, with OpenAI models, you can generate with OpenAI models, you can generate with OpenAI models, you can generate images and it will use their API to call images and it will use their API to call images and it will use their API to call the image gen tools and models that they the image gen tools and models that they the image gen tools and models that they have, where with Claude models, you have, where with Claude models, you have, where with Claude models, you can't do that. You have to call some can't do that. You have to call some can't do that. You have to call some other external service. Grok models have other external service. Grok models have other external service. Grok models have the ability to do cool things with the the ability to do cool things with the the ability to do cool things with the Twitter API. Like if I want to get good Twitter API. Like if I want to get good Twitter API. Like if I want to get good data out of Twitter, I can't do it data out of Twitter, I can't do it data out of Twitter, I can't do it through the API, it's way too expensive. through the API, it's way too expensive. through the API, it's way too expensive. So, I'll just tell Codex or Claude to So, I'll just tell Codex or Claude to So, I'll just tell Codex or Claude to call Grok, and by calling Grok, it gets call Grok, and by calling Grok, it gets call Grok, and by calling Grok, it gets access to the XAI APIs. A lot of access to the XAI APIs. A lot of access to the XAI APIs. A lot of different models have these types of different models have these types of different models have these types of capabilities. If they have access to capabilities. If they have access to capabilities. If they have access to unique data, you can get it through that unique data, you can get it through that unique data, you can get it through that model. So, why am I bringing this up? model. So, why am I bringing this up? model. So, why am I bringing this up? Gemini models have some unique Gemini models have some unique Gemini models have some unique capabilities, many of which are very capabilities, many of which are very capabilities, many of which are very useful to me. Why are they useful to me? useful to me. Why are they useful to me? useful to me. Why are they useful to me? I'm a YouTuber. That means I have to do I'm a YouTuber. That means I have to do I'm a YouTuber. That means I have to do a lot of stuff on YouTube, and Google a lot of stuff on YouTube, and Google a lot of stuff on YouTube, and Google gives very generous access to the gives very generous access to the gives very generous access to the YouTube API and capabilities to the YouTube API and capabilities to the YouTube API and capabilities to the Gemini models. If you want to get a Gemini models. If you want to get a Gemini models. If you want to get a transcript from a YouTube video, the transcript from a YouTube video, the transcript from a YouTube video, the Gemini API is one of the best ways to do Gemini API is one of the best ways to do Gemini API is one of the best ways to do it right now. And we've been using the it right now. And we've been using the it right now. And we've been using the Google models heavily for all of those Google models heavily for all of those Google models heavily for all of those types of things throughout my types of things throughout my types of things throughout my businesses. So, I will be using this businesses. So, I will be using this businesses. So, I will be using this model a lot. Not because I necessarily model a lot. Not because I necessarily model a lot. Not because I necessarily want to or I'm choosing to cuz it's so want to or I'm choosing to cuz it's so want to or I'm choosing to cuz it's so uniquely intelligent and such a good uniquely intelligent and such a good uniquely intelligent and such a good value. Just because Google has unique value. Just because Google has unique value. Just because Google has unique capabilities that they're providing capabilities that they're providing capabilities that they're providing their models through their APIs, through their models through their APIs, through their models through their APIs, through their data centers, through the things their data centers, through the things their data centers, through the things that they uniquely have access to. That that they uniquely have access to. That that they uniquely have access to. That said, I've actually found the Gemini said, I've actually found the Gemini said, I've actually found the Gemini models not that good at most Google models not that good at most Google models not that good at most Google services. I have a much better services. I have a much better services. I have a much better experience dealing with my email inbox experience dealing with my email inbox experience dealing with my email inbox through tools like Grok and Chat GPT

through tools like Grok and Chat GPT through tools like Grok and Chat GPT than I do with anything Gemini. In fact, than I do with anything Gemini. In fact, than I do with anything Gemini. In fact, every time the Gemini button appears in every time the Gemini button appears in every time the Gemini button appears in some app, it it scares me and is often some app, it it scares me and is often some app, it it scares me and is often not functional or just outright cringe. not functional or just outright cringe. not functional or just outright cringe. I think if you're a YouTuber or you work I think if you're a YouTuber or you work I think if you're a YouTuber or you work with YouTubers, Gemini will be a very with YouTubers, Gemini will be a very with YouTubers, Gemini will be a very useful thing to have in your pipeline. useful thing to have in your pipeline. useful thing to have in your pipeline. If you're doing a lot of unique vision If you're doing a lot of unique vision If you're doing a lot of unique vision work, historically the Gemini models work, historically the Gemini models work, historically the Gemini models have been good at that as well, so worth have been good at that as well, so worth have been good at that as well, so worth benchmarking. I don't think there is as benchmarking. I don't think there is as benchmarking. I don't think there is as much here as there used to be though, much here as there used to be though, much here as there used to be though, cuz a lot of the things that Google's cuz a lot of the things that Google's cuz a lot of the things that Google's models were uniquely good at have been models were uniquely good at have been models were uniquely good at have been maintained and achieved by others maintained and achieved by others maintained and achieved by others externally. There is some secret sauce externally. There is some secret sauce externally. There is some secret sauce in this model. There's something that in this model. There's something that in this model. There's something that makes it different. You can see it in makes it different. You can see it in makes it different. You can see it in SkateBench. It's the only model to get a SkateBench. It's the only model to get a SkateBench. It's the only model to get a perfect score and the only models to perfect score and the only models to perfect score and the only models to break the 90 range are Gemini and break the 90 range are Gemini and break the 90 range are Gemini and somehow Muse wonderful. Everything else somehow Muse wonderful. Everything else somehow Muse wonderful. Everything else is in the 80s, 70s, even 50s and 60s. is in the 80s, 70s, even 50s and 60s. is in the 80s, 70s, even 50s and 60s. So, there might be more secret hidden So, there might be more secret hidden So, there might be more secret hidden capabilities I'm not seeing in the capabilities I'm not seeing in the capabilities I'm not seeing in the model. I've heard people say that model. I've heard people say that model. I've heard people say that they're really good for they're really good for they're really good for internationalization, localization, internationalization, localization, internationalization, localization, multilingual stuff. I haven't seen too multilingual stuff. I haven't seen too multilingual stuff. I haven't seen too much of this unique capability in my own much of this unique capability in my own much of this unique capability in my own testing. I just wanted to call it out testing. I just wanted to call it out testing. I just wanted to call it out because it's not that common nowadays.

because it's not that common nowadays. because it's not that common nowadays. Usually, since all the models are just Usually, since all the models are just Usually, since all the models are just throw all the data in pre-training and throw all the data in pre-training and throw all the data in pre-training and then RL it on code, we don't get as much then RL it on code, we don't get as much then RL it on code, we don't get as much unique capability in things anymore, but unique capability in things anymore, but unique capability in things anymore, but Google still has some and I thought it Google still has some and I thought it Google still has some and I thought it was worth noting. Now, we get into the was worth noting. Now, we get into the was worth noting. Now, we get into the meat, the things that y'all are most meat, the things that y'all are most meat, the things that y'all are most hyped about, things like the real hyped about, things like the real hyped about, things like the real day-to-day code work, the things I hate day-to-day code work, the things I hate day-to-day code work, the things I hate about Gemini itself and the Gemini about Gemini itself and the Gemini about Gemini itself and the Gemini ecosystem, Opus, Soul, and Muse and how ecosystem, Opus, Soul, and Muse and how ecosystem, Opus, Soul, and Muse and how they compare, and of course, the thing they compare, and of course, the thing they compare, and of course, the thing everyone is hyped about, weirdly so, everyone is hyped about, weirdly so, everyone is hyped about, weirdly so, Fish Slap, my game that I make with Fish Slap, my game that I make with Fish Slap, my game that I make with every model to see how good they are at every model to see how good they are at every model to see how good they are at 3D. So, uh without further ado, I think 3D. So, uh without further ado, I think 3D. So, uh without further ado, I think it's time for fish slop. And no, the it's time for fish slop. And no, the it's time for fish slop. And no, the video is not encoding incorrectly. It video is not encoding incorrectly. It video is not encoding incorrectly. It did actually screw up the top of the did actually screw up the top of the did actually screw up the top of the water that badly. The UI is uh water that badly. The UI is uh water that badly. The UI is uh it's Gemini, to say the least. It did it's Gemini, to say the least. It did it's Gemini, to say the least. It did make the whole game loop where there's make the whole game loop where there's make the whole game loop where there's five tanks to get through, which is five tanks to get through, which is five tanks to get through, which is solid. Submarine models, solid. Submarine models, solid. Submarine models, not particularly great. It's not as bad not particularly great. It's not as bad not particularly great. It's not as bad as things like Muse and Grok, but it's as things like Muse and Grok, but it's as things like Muse and Grok, but it's not great. Considering how crazy all the not great. Considering how crazy all the not great. Considering how crazy all the 3D demos I saw other people posting 3D demos I saw other people posting 3D demos I saw other people posting were, were, were, I expected a lot more. I gave it the I expected a lot more. I gave it the I expected a lot more. I gave it the exact same prompt, exact same setup, exact same prompt, exact same setup, exact same prompt, exact same setup, exact same tools, and everything as the exact same tools, and everything as the exact same tools, and everything as the others had.

others had. others had. It does animation decently well. Like, It does animation decently well. Like, It does animation decently well. Like, the way the fish moves is reasonable. the way the fish moves is reasonable. the way the fish moves is reasonable. The eyes are pointed in the right The eyes are pointed in the right The eyes are pointed in the right direction, which might not sound like direction, which might not sound like direction, which might not sound like much, but just a few months ago that was much, but just a few months ago that was much, but just a few months ago that was a rare skill. a rare skill. a rare skill. But, there are some uniquely But, there are some uniquely But, there are some uniquely Google-esque things here. Google-esque things here. Google-esque things here. Let me show you one of the funnier Let me show you one of the funnier Let me show you one of the funnier examples. examples. examples. So, games have to have physics and So, games have to have physics and So, games have to have physics and mechanics. My game has a mechanic where mechanics. My game has a mechanic where mechanics. My game has a mechanic where the fish eat the food. the fish eat the food. the fish eat the food. There's a problem. There's a problem. There's a problem. The food can fall to the bottom of the The food can fall to the bottom of the The food can fall to the bottom of the tank, and when it does, it doesn't tank, and when it does, it doesn't tank, and when it does, it doesn't expire. expire. expire. The fish can go to the bottom of the The fish can go to the bottom of the The fish can go to the bottom of the tank to eat it, but they can't figure tank to eat it, but they can't figure tank to eat it, but they can't figure out how. out how. out how. So, the fish will come over cuz they're So, the fish will come over cuz they're So, the fish will come over cuz they're hungry. They'll try to eat the food hungry. They'll try to eat the food hungry. They'll try to eat the food that's on the floor, so they can't, and that's on the floor, so they can't, and that's on the floor, so they can't, and then they'll just sit there staring at then they'll just sit there staring at then they'll just sit there staring at the food trying to get to it until they the food trying to get to it until they the food trying to get to it until they die. It wouldn't be too bad if I could die. It wouldn't be too bad if I could die. It wouldn't be too bad if I could just like feed them more, but the game just like feed them more, but the game just like feed them more, but the game has a mechanic where you can only put so has a mechanic where you can only put so has a mechanic where you can only put so much food in the tank at a time, and it much food in the tank at a time, and it much food in the tank at a time, and it forgot to make these expire. forgot to make these expire. forgot to make these expire. So, it will actually just sit here So, it will actually just sit here So, it will actually just sit here forever. I also just realized the sound forever. I also just realized the sound forever. I also just realized the sound is not routing, so let me fix that is not routing, so let me fix that is not routing, so let me fix that quick. There we go. Sorry for the lack quick. There we go. Sorry for the lack quick. There we go. Sorry for the lack of sound for the first bit. As I said, of sound for the first bit. As I said, of sound for the first bit. As I said, you weren't missing out on too much.

you weren't missing out on too much. you weren't missing out on too much. It's not particularly great-sounding It's not particularly great-sounding It's not particularly great-sounding game. game. game. But, this version was bad enough and But, this version was bad enough and But, this version was bad enough and broken enough that I wanted to give it broken enough that I wanted to give it broken enough that I wanted to give it one more shot. one more shot. one more shot. So, I did. I told about some of the So, I did. I told about some of the So, I did. I told about some of the things that were broken, like the things that were broken, like the things that were broken, like the shimmering on the ceiling of the tank, shimmering on the ceiling of the tank, shimmering on the ceiling of the tank, the food mechanic being broken where the the food mechanic being broken where the the food mechanic being broken where the food wouldn't expire and they couldn't food wouldn't expire and they couldn't food wouldn't expire and they couldn't get at the bottom, and that I was get at the bottom, and that I was get at the bottom, and that I was disappointed in the graphics and I disappointed in the graphics and I disappointed in the graphics and I wanted it to improve. So, it ran for a wanted it to improve. So, it ran for a wanted it to improve. So, it ran for a while. It came back already looking a while. It came back already looking a while. It came back already looking a hell of a lot better. hell of a lot better. hell of a lot better. Didn't touch the model for the sub, Didn't touch the model for the sub, Didn't touch the model for the sub, sadly, sadly, sadly, but it did fix a lot of the things but it did fix a lot of the things but it did fix a lot of the things involving like the feeding of the fish. involving like the feeding of the fish. involving like the feeding of the fish. And I was able to actually like play And I was able to actually like play And I was able to actually like play this version for a bit. But then you this version for a bit. But then you this version for a bit. But then you start to notice some of the start to notice some of the start to notice some of the the Gemini-isms. And the best I can put the Gemini-isms. And the best I can put the Gemini-isms. And the best I can put it is like it is like it is like it doesn't have big model smell. Big it doesn't have big model smell. Big it doesn't have big model smell. Big model smell is really hard to describe. model smell is really hard to describe. model smell is really hard to describe. It's a a type of thoughtfulness and It's a a type of thoughtfulness and It's a a type of thoughtfulness and generalization that models do when they generalization that models do when they generalization that models do when they have like a better understanding of the have like a better understanding of the have like a better understanding of the world that comes from the model being world that comes from the model being world that comes from the model being bigger. I talked about this a bit in bigger. I talked about this a bit in bigger. I talked about this a bit in terms of 3D in my previous video about terms of 3D in my previous video about terms of 3D in my previous video about Muse, where I talked about how the Muse, where I talked about how the Muse, where I talked about how the model's 3D capabilities don't reflect model's 3D capabilities don't reflect model's 3D capabilities don't reflect how smart or capable the model is. 3D is how smart or capable the model is. 3D is how smart or capable the model is. 3D is a thing that they train in separately, a thing that they train in separately, a thing that they train in separately, like they add it on.

like they add it on. like they add it on. This model clearly got more time with This model clearly got more time with This model clearly got more time with that 3D stuff, but it also has some that 3D stuff, but it also has some that 3D stuff, but it also has some stupid mistakes that I haven't seen stupid mistakes that I haven't seen stupid mistakes that I haven't seen other models do. Monsters spawn every other models do. Monsters spawn every other models do. Monsters spawn every minute. When you finish the wave, they minute. When you finish the wave, they minute. When you finish the wave, they drop a power up, and the power up lasts drop a power up, and the power up lasts drop a power up, and the power up lasts for 30 seconds. Why is there a power up for 30 seconds. Why is there a power up for 30 seconds. Why is there a power up that helps with my attack? I can't shoot that helps with my attack? I can't shoot that helps with my attack? I can't shoot anything after I get it, and they expire anything after I get it, and they expire anything after I get it, and they expire before I can use it. So, one of the before I can use it. So, one of the before I can use it. So, one of the power ups is legitimately useless power ups is legitimately useless power ups is legitimately useless because there will never be a monster in because there will never be a monster in because there will never be a monster in the tank when you have it on because it the tank when you have it on because it the tank when you have it on because it will disappear before the monsters come will disappear before the monsters come will disappear before the monsters come in. It's just like the type of silly in. It's just like the type of silly in. It's just like the type of silly thing that a human would not have done, thing that a human would not have done, thing that a human would not have done, that a smart model also would not have that a smart model also would not have that a smart model also would not have done, that a model that's goal is to done, that a model that's goal is to done, that a model that's goal is to just write code that seems good without just write code that seems good without just write code that seems good without thinking about it would absolutely do. thinking about it would absolutely do. thinking about it would absolutely do. Imagine that you were a part-time Imagine that you were a part-time Imagine that you were a part-time contributor to a game like this, and contributor to a game like this, and contributor to a game like this, and there was three power-ups made, you there was three power-ups made, you there was three power-ups made, you might throw out the idea of a power-up might throw out the idea of a power-up might throw out the idea of a power-up that does more damage and not even think that does more damage and not even think that does more damage and not even think about it and then add it. A smart model about it and then add it. A smart model about it and then add it. A smart model should have a good enough understanding should have a good enough understanding should have a good enough understanding of the system to know that that is of the system to know that that is of the system to know that that is stupid. Gemini doesn't. I think now is a stupid. Gemini doesn't. I think now is a stupid. Gemini doesn't. I think now is a better time than ever to start talking better time than ever to start talking better time than ever to start talking about the way this model behaves in about the way this model behaves in about the way this model behaves in real-world tasks. In particular, real-world tasks. In particular, real-world tasks. In particular, hallucination. The thing I just showed hallucination. The thing I just showed hallucination. The thing I just showed you wasn't hallucination. It's not like you wasn't hallucination. It's not like you wasn't hallucination. It's not like the model imagined this thing that the model imagined this thing that the model imagined this thing that wasn't the case. It saw a power-up wasn't the case. It saw a power-up wasn't the case. It saw a power-up system, it came up with an idea for the system, it came up with an idea for the system, it came up with an idea for the power-up system, and it put it in the power-up system, and it put it in the power-up system, and it put it in the game. It might be a stupid power-up, game. It might be a stupid power-up, game. It might be a stupid power-up, doesn't make sense, but it didn't doesn't make sense, but it didn't doesn't make sense, but it didn't hallucinate anything. According to hallucinate anything. According to hallucinate anything. According to Artificial Analysis, the model actually Artificial Analysis, the model actually Artificial Analysis, the model actually doesn't hallucinate much. On their AI doesn't hallucinate much. On their AI doesn't hallucinate much. On their AI Omniscience benchmark, which is their Omniscience benchmark, which is their Omniscience benchmark, which is their attempt to measure hallucination, Gemini

attempt to measure hallucination, Gemini attempt to measure hallucination, Gemini 4 Argon had a 15% hallucination rate, 4 Argon had a 15% hallucination rate, 4 Argon had a 15% hallucination rate, the lowest of any model scoring over a the lowest of any model scoring over a the lowest of any model scoring over a 45 on the intelligence index. And the S 45 on the intelligence index. And the S 45 on the intelligence index. And the S and D, you can see here in the official and D, you can see here in the official and D, you can see here in the official charts, Gemini 4 Argon has an charts, Gemini 4 Argon has an charts, Gemini 4 Argon has an impressively low hallucination rate. But impressively low hallucination rate. But impressively low hallucination rate. But the AI Omniscience bench is not just the AI Omniscience bench is not just the AI Omniscience bench is not just hallucinations. It's both incorrect hallucinations. It's both incorrect hallucinations. It's both incorrect answers as well as partial answers and answers as well as partial answers and answers as well as partial answers and refusals. So, if it gets the questions refusals. So, if it gets the questions refusals. So, if it gets the questions correctly, it gets points. If it gets correctly, it gets points. If it gets correctly, it gets points. If it gets them incorrectly, it loses points. If it them incorrectly, it loses points. If it them incorrectly, it loses points. If it says, "I can't answer this," it's says, "I can't answer this," it's says, "I can't answer this," it's neutral. When you look at the overall neutral. When you look at the overall neutral. When you look at the overall score, it's no longer first place, it's score, it's no longer first place, it's score, it's no longer first place, it's now in fourth because it gets few enough now in fourth because it gets few enough now in fourth because it gets few enough things right that its refusal to get things right that its refusal to get things right that its refusal to get things wrong does not really matter as things wrong does not really matter as things wrong does not really matter as much. It's just like, "Sorry, can't much. It's just like, "Sorry, can't much. It's just like, "Sorry, can't help." more often than others. Cool. help." more often than others. Cool. help." more often than others. Cool. Nice to see Google making their models Nice to see Google making their models Nice to see Google making their models hallucinate less. Still not a hallucinate less. Still not a hallucinate less. Still not a particularly good score. Sadly, from my particularly good score. Sadly, from my particularly good score. Sadly, from my experience, this benchmark is not that experience, this benchmark is not that experience, this benchmark is not that realistic because I have seen this model realistic because I have seen this model realistic because I have seen this model just be wrong a lot. Here is where we just be wrong a lot. Here is where we just be wrong a lot. Here is where we get into the real code work and where my get into the real code work and where my get into the real code work and where my actual breakdown of the model begins. I actual breakdown of the model begins. I actual breakdown of the model begins. I started by throwing it at TS Rust, a started by throwing it at TS Rust, a started by throwing it at TS Rust, a project I built that I talked about in project I built that I talked about in project I built that I talked about in other videos where I rewrote the entire other videos where I rewrote the entire other videos where I rewrote the entire chess compiler in Rust using Opus 5.5. I chess compiler in Rust using Opus 5.5. I chess compiler in Rust using Opus 5.5. I had a bunch of different models audit had a bunch of different models audit had a bunch of different models audit the code base and describe it, things the code base and describe it, things the code base and describe it, things that could be improved, what is it worth that could be improved, what is it worth that could be improved, what is it worth knowing in this code base. I would take knowing in this code base. I would take knowing in this code base. I would take all the claims it made, validate them, all the claims it made, validate them, all the claims it made, validate them, and then put them through a panel of and then put them through a panel of and then put them through a panel of judges to rate how good or bad the model judges to rate how good or bad the model judges to rate how good or bad the model did. GPT-61 Soul came out on the top did. GPT-61 Soul came out on the top did. GPT-61 Soul came out on the top making 38 claims that were all true. It

making 38 claims that were all true. It making 38 claims that were all true. It had zero wrong claims, and the claims had zero wrong claims, and the claims had zero wrong claims, and the claims were informative, accurate, and useful. were informative, accurate, and useful. were informative, accurate, and useful. So, all the judges gave it a pretty good So, all the judges gave it a pretty good So, all the judges gave it a pretty good score, averaged an 8.84. Opus 5.5 score, averaged an 8.84. Opus 5.5 score, averaged an 8.84. Opus 5.5 slightly lower, 8.78. It took way slightly lower, 8.78. It took way slightly lower, 8.78. It took way longer, 49 minutes versus 28 minutes, longer, 49 minutes versus 28 minutes, longer, 49 minutes versus 28 minutes, and it cost $24 instead of $4, and it and it cost $24 instead of $4, and it and it cost $24 instead of $4, and it got one of the claims wrong. Still a got one of the claims wrong. Still a got one of the claims wrong. Still a very good score, just made some mistakes very good score, just made some mistakes very good score, just made some mistakes and took too long and was too expensive. and took too long and was too expensive. and took too long and was too expensive. 6 Astra also had no wrong claims, was 6 Astra also had no wrong claims, was 6 Astra also had no wrong claims, was way faster at 20 minutes, still a bit way faster at 20 minutes, still a bit way faster at 20 minutes, still a bit too expensive. 61 Soul on X High, things too expensive. 61 Soul on X High, things too expensive. 61 Soul on X High, things improved. Also, funny, uh 6 Astra on Max improved. Also, funny, uh 6 Astra on Max improved. Also, funny, uh 6 Astra on Max scored worse than 6 Astra on X High. scored worse than 6 Astra on X High. scored worse than 6 Astra on X High. Once again, as I've been saying for a Once again, as I've been saying for a Once again, as I've been saying for a while, you should probably avoid the max while, you should probably avoid the max while, you should probably avoid the max reasoning levels. They do more damage reasoning levels. They do more damage reasoning levels. They do more damage than good. You might notice something's than good. You might notice something's than good. You might notice something's missing, though. Where is Gemini? missing, though. Where is Gemini? missing, though. Where is Gemini? Well, we hadn't scrolled down far Well, we hadn't scrolled down far Well, we hadn't scrolled down far enough. enough. enough. Yeah. Gemini for Argon, in my audit the Yeah. Gemini for Argon, in my audit the Yeah. Gemini for Argon, in my audit the code base bench, scored a 4.38, code base bench, scored a 4.38, code base bench, scored a 4.38, less than half of Opus's score and 61 less than half of Opus's score and 61 less than half of Opus's score and 61 Soul's score. It also took 43 minutes Soul's score. It also took 43 minutes Soul's score. It also took 43 minutes long, almost double the time 61 Soul long, almost double the time 61 Soul long, almost double the time 61 Soul took, and more than double it than it took, and more than double it than it took, and more than double it than it took on X High. On the bright side, it took on X High. On the bright side, it took on X High. On the bright side, it would have only cost $3, but on the less would have only cost $3, but on the less would have only cost $3, but on the less bright side, of its 36 claims, 24 were bright side, of its 36 claims, 24 were bright side, of its 36 claims, 24 were false. The majority of the things it false. The majority of the things it false. The majority of the things it claimed about the code base were wrong.

claimed about the code base were wrong. claimed about the code base were wrong. Do you understand how egregious that is? Do you understand how egregious that is? Do you understand how egregious that is? Duh. It means that when this model's Duh. It means that when this model's Duh. It means that when this model's doing real work in a real code base, doing real work in a real code base, doing real work in a real code base, it's as likely, if not more likely, to it's as likely, if not more likely, to it's as likely, if not more likely, to be wrong than right. And this has lined be wrong than right. And this has lined be wrong than right. And this has lined up with my experience using it. I I hate up with my experience using it. I I hate up with my experience using it. I I hate to be that guy, especially cuz I've to be that guy, especially cuz I've to be that guy, especially cuz I've likely burned some credibility with a likely burned some credibility with a likely burned some credibility with a lot of y'all with my Muse Wonder video. lot of y'all with my Muse Wonder video. lot of y'all with my Muse Wonder video. I don't care. Muse Wonder is a unique I don't care. Muse Wonder is a unique I don't care. Muse Wonder is a unique balance here where it is getting similar balance here where it is getting similar balance here where it is getting similar scores to Fable, way faster, and gets scores to Fable, way faster, and gets scores to Fable, way faster, and gets things wrong, but eight out of 33 is things wrong, but eight out of 33 is things wrong, but eight out of 33 is comparable to Fable 5.1's 11 of 33, and comparable to Fable 5.1's 11 of 33, and comparable to Fable 5.1's 11 of 33, and a whole different world than 24 of 36. a whole different world than 24 of 36. a whole different world than 24 of 36. I'm going to be so real, and we'll I'm going to be so real, and we'll I'm going to be so real, and we'll you'll see what I mean as we go deeper. you'll see what I mean as we go deeper. you'll see what I mean as we go deeper. I don't think anybody who finds this I don't think anybody who finds this I don't think anybody who finds this model on the same frontier level model on the same frontier level model on the same frontier level capability as things like Opus 5.5, capability as things like Opus 5.5, capability as things like Opus 5.5, Fable 5.1, 6 Astra, and 6.1 Soul, or Fable 5.1, 6 Astra, and 6.1 Soul, or Fable 5.1, 6 Astra, and 6.1 Soul, or even Muse Wonder. I don't think anyone even Muse Wonder. I don't think anyone even Muse Wonder. I don't think anyone who finds these models comparable is who finds these models comparable is who finds these models comparable is actually working on that real of stuff. actually working on that real of stuff. actually working on that real of stuff. Show me demos if you disagree. I would Show me demos if you disagree. I would Show me demos if you disagree. I would like to be wrong here. I want to have like to be wrong here. I want to have like to be wrong here. I want to have more faith in Google as a competitor, more faith in Google as a competitor, more faith in Google as a competitor, but the numbers aren't going to get but the numbers aren't going to get but the numbers aren't going to get better as we keep going. The next round better as we keep going. The next round better as we keep going. The next round was one where I audited a real bug in T3 was one where I audited a real bug in T3 was one where I audited a real bug in T3 code related to how the bulk remote code related to how the bulk remote code related to how the bulk remote update workflows were functioning. Opus update workflows were functioning. Opus update workflows were functioning. Opus 5.5 got the best score with an 822. It 5.5 got the best score with an 822. It 5.5 got the best score with an 822. It had five claims that were wrong of its had five claims that were wrong of its had five claims that were wrong of its 46. It was $26 to run, and took 20 46. It was $26 to run, and took 20 46. It was $26 to run, and took 20 minutes, but these are like good, useful minutes, but these are like good, useful minutes, but these are like good, useful numbers. 6.1 Soul got a lower score. It numbers. 6.1 Soul got a lower score. It numbers. 6.1 Soul got a lower score. It missed some of the high severity things missed some of the high severity things missed some of the high severity things Opus 5.5 found, but it didn't say Opus 5.5 found, but it didn't say Opus 5.5 found, but it didn't say anything incorrect. It took 10 minutes, anything incorrect. It took 10 minutes, anything incorrect. It took 10 minutes, and it cost a dollar. Insane. This is and it cost a dollar. Insane. This is and it cost a dollar. Insane. This is why I like 6.1 Soul so much as a review

why I like 6.1 Soul so much as a review why I like 6.1 Soul so much as a review model. Muse Wonder scored quite a bit model. Muse Wonder scored quite a bit model. Muse Wonder scored quite a bit lower. I think it missed a handful of lower. I think it missed a handful of lower. I think it missed a handful of claims, and it still had the same claims, and it still had the same claims, and it still had the same roughly error rate as Opus did, but it roughly error rate as Opus did, but it roughly error rate as Opus did, but it only took 6 minutes, and most only took 6 minutes, and most only took 6 minutes, and most importantly, in my opinion, it was super importantly, in my opinion, it was super importantly, in my opinion, it was super readable. So again, I like Muse Wonder readable. So again, I like Muse Wonder readable. So again, I like Muse Wonder cuz it's a different flavor, and finds cuz it's a different flavor, and finds cuz it's a different flavor, and finds useful stuff. Gemini for Argon is useful stuff. Gemini for Argon is useful stuff. Gemini for Argon is another flavor. It's not just a Claude another flavor. It's not just a Claude another flavor. It's not just a Claude distill like a lot of the Open Weight distill like a lot of the Open Weight distill like a lot of the Open Weight Labs are doing. So like, if you talk to Labs are doing. So like, if you talk to Labs are doing. So like, if you talk to something like GLM 5.3 Flash, Kimmy K3, something like GLM 5.3 Flash, Kimmy K3, something like GLM 5.3 Flash, Kimmy K3, or any of those other models, they all or any of those other models, they all or any of those other models, they all talk and act relatively similarly. All talk and act relatively similarly. All talk and act relatively similarly. All four of these models behave entirely four of these models behave entirely four of these models behave entirely differently. Gemini is too dumb to differently. Gemini is too dumb to differently. Gemini is too dumb to justify the flavor change. Muse Wonder justify the flavor change. Muse Wonder justify the flavor change. Muse Wonder Full is probably too expensive to Full is probably too expensive to Full is probably too expensive to justify it. We don't know cuz again, we justify it. We don't know cuz again, we justify it. We don't know cuz again, we don't have the price. But at least it's don't have the price. But at least it's don't have the price. But at least it's not lying 14 out of 40 times. Yeah. Muse not lying 14 out of 40 times. Yeah. Muse not lying 14 out of 40 times. Yeah. Muse has taken whatever I was hoping Gemini has taken whatever I was hoping Gemini has taken whatever I was hoping Gemini would do. would do. would do. Hopefully you're starting to see why I Hopefully you're starting to see why I Hopefully you're starting to see why I was nicer to Muse than people expected. was nicer to Muse than people expected. was nicer to Muse than people expected. Testing both the Muse model and the Testing both the Muse model and the Testing both the Muse model and the Gemini model at the same time really Gemini model at the same time really Gemini model at the same time really deeply reminded me of how much I don't deeply reminded me of how much I don't deeply reminded me of how much I don't like working with Gemini. Sadly, it gets like working with Gemini. Sadly, it gets like working with Gemini. Sadly, it gets worse. I didn't do a fancy visual for worse. I didn't do a fancy visual for worse. I didn't do a fancy visual for this one. This is an audit of my cloud this one. This is an audit of my cloud this one. This is an audit of my cloud product Lakebed. 61 Soul got a nine out product Lakebed. 61 Soul got a nine out product Lakebed. 61 Soul got a nine out of 10. Sonic 55 got an 8 and 1/2 out of of 10. Sonic 55 got an 8 and 1/2 out of of 10. Sonic 55 got an 8 and 1/2 out of 10, mostly cuz Sonic 5 on Mac's will 10, mostly cuz Sonic 5 on Mac's will 10, mostly cuz Sonic 5 on Mac's will just burn and burn and burn. So, just burn and burn and burn. So, just burn and burn and burn. So, it finds everything. Muse took a pretty it finds everything. Muse took a pretty it finds everything. Muse took a pretty big hit here going from the 825 from big hit here going from the 825 from big hit here going from the 825 from Opus 55 down to 5.75. It's not as big of Opus 55 down to 5.75. It's not as big of Opus 55 down to 5.75. It's not as big of a hit as Gemini gets, 2.25.

a hit as Gemini gets, 2.25. a hit as Gemini gets, 2.25. These are scores out of 10. Muse gave These are scores out of 10. Muse gave These are scores out of 10. Muse gave its findings a three. Opus, Fable, and its findings a three. Opus, Fable, and its findings a three. Opus, Fable, and Astra gave its findings a two. Of its 78 Astra gave its findings a two. Of its 78 Astra gave its findings a two. Of its 78 findings, 56 were wrong. That is a 70% findings, 56 were wrong. That is a 70% findings, 56 were wrong. That is a 70% failure [snorts] rate. This model is failure [snorts] rate. This model is failure [snorts] rate. This model is more likely to gaslight your agents than more likely to gaslight your agents than more likely to gaslight your agents than it is to give useful feedback in it is to give useful feedback in it is to give useful feedback in reviews. If you do decide to use this reviews. If you do decide to use this reviews. If you do decide to use this model for real work for some reason, for model for real work for some reason, for model for real work for some reason, for the love of God, please put the things the love of God, please put the things the love of God, please put the things it does through Soul or Opus or some it does through Soul or Opus or some it does through Soul or Opus or some model that doesn't have this egregious model that doesn't have this egregious model that doesn't have this egregious rate of lying. I can't fathom how any rate of lying. I can't fathom how any rate of lying. I can't fathom how any bench measured this model is not bench measured this model is not bench measured this model is not hallucinating a lot when I don't I've hallucinating a lot when I don't I've hallucinating a lot when I don't I've never seen it not hallucinate in my never seen it not hallucinate in my never seen it not hallucinate in my day-to-day real-world code work. In my day-to-day real-world code work. In my day-to-day real-world code work. In my final run where I had it go through the final run where I had it go through the final run where I had it go through the T3 code architecture, it got a 3.95 out T3 code architecture, it got a 3.95 out T3 code architecture, it got a 3.95 out of 10. of 10. of 10. And you can also see again Muse And you can also see again Muse And you can also see again Muse wonderful being not like as good if not wonderful being not like as good if not wonderful being not like as good if not better than things like Fable and Astra, better than things like Fable and Astra, better than things like Fable and Astra, but neck and neck with Astra while also but neck and neck with Astra while also but neck and neck with Astra while also being much more pleasant to work with is being much more pleasant to work with is being much more pleasant to work with is worth noting. Again, to compare to Muse, worth noting. Again, to compare to Muse, worth noting. Again, to compare to Muse, it got 29 true claims, eight partially it got 29 true claims, eight partially it got 29 true claims, eight partially true, four false. Argon got 31 true, 14 true, four false. Argon got 31 true, 14 true, four false. Argon got 31 true, 14 partially true, 47 false. Gemini for partially true, 47 false. Gemini for partially true, 47 false. Gemini for Argon had more false findings than Muse Argon had more false findings than Muse Argon had more false findings than Muse wonderful had true findings. What the wonderful had true findings. What the wonderful had true findings. What the It's not just like kind of bad or It's not just like kind of bad or It's not just like kind of bad or a little more likely to do something a little more likely to do something a little more likely to do something wrong. No, it's like an order of wrong. No, it's like an order of wrong. No, it's like an order of magnitude gap here. It is lying 10 times magnitude gap here. It is lying 10 times magnitude gap here. It is lying 10 times more than any of the other models I'm

more than any of the other models I'm more than any of the other models I'm benching other than Muse wonder the benching other than Muse wonder the benching other than Muse wonder the non-full version. That one's just non-full version. That one's just non-full version. That one's just garbage. Throw it out. Don't touch it. garbage. Throw it out. Don't touch it. garbage. Throw it out. Don't touch it. But like the only bad models in this But like the only bad models in this But like the only bad models in this bench were Muse wonder standard and bench were Muse wonder standard and bench were Muse wonder standard and Gemini 4 Argon and they're both garbage. Gemini 4 Argon and they're both garbage. Gemini 4 Argon and they're both garbage. As you saw in my Muse video, I'm a As you saw in my Muse video, I'm a As you saw in my Muse video, I'm a little bit more forgiving if the model's little bit more forgiving if the model's little bit more forgiving if the model's pleasant to interact with. No points pleasant to interact with. No points pleasant to interact with. No points here either, Google. here either, Google. here either, Google. Walls upon walls of text full of things Walls upon walls of text full of things Walls upon walls of text full of things that don't really matter that much. Not that don't really matter that much. Not that don't really matter that much. Not very pleasant to read. But if the code very pleasant to read. But if the code very pleasant to read. But if the code works, that's fine then, right? Well, I works, that's fine then, right? Well, I works, that's fine then, right? Well, I had it do some work around the SQL light had it do some work around the SQL light had it do some work around the SQL light connection layer in T3 code in hopes of connection layer in T3 code in hopes of connection layer in T3 code in hopes of stabilizing it for heavy users on stabilizing it for heavy users on stabilizing it for heavy users on platforms like Windows where we're platforms like Windows where we're platforms like Windows where we're having some connectivity issues. I having some connectivity issues. I having some connectivity issues. I already knew a lot of the fixes. I was already knew a lot of the fixes. I was already knew a lot of the fixes. I was just curious if Gemini could find them. just curious if Gemini could find them. just curious if Gemini could find them. So, there's lots of users reporting So, there's lots of users reporting So, there's lots of users reporting issues with instability in connections. issues with instability in connections. issues with instability in connections. A lot appear to be SQL light related. A lot appear to be SQL light related. A lot appear to be SQL light related. What should we prioritize to fix this What should we prioritize to fix this What should we prioritize to fix this issue? I even gave it a hint with async issue? I even gave it a hint with async issue? I even gave it a hint with async SQL light. This is not the right SQL light. This is not the right SQL light. This is not the right solution. The right solution is pooling solution. The right solution is pooling solution. The right solution is pooling and having read pools at a single right. and having read pools at a single right. and having read pools at a single right. It did a hunt. It did a lot of the It did a hunt. It did a lot of the It did a hunt. It did a lot of the classic Gemini-isms of just reading classic Gemini-isms of just reading classic Gemini-isms of just reading every single file synchronously. It just every single file synchronously. It just every single file synchronously. It just kind of went on and on with that.

kind of went on and on with that. kind of went on and on with that. It does correctly identify the split It does correctly identify the split It does correctly identify the split read pool and a single right connection read pool and a single right connection read pool and a single right connection piece. piece. piece. I tell it to make a PR for priority one, I tell it to make a PR for priority one, I tell it to make a PR for priority one, which is the first section it had in which is the first section it had in which is the first section it had in here. here. here. Uh 30 minutes later, way longer than it Uh 30 minutes later, way longer than it Uh 30 minutes later, way longer than it should have taken, I've been told by should have taken, I've been told by should have taken, I've been told by some people that apparently Gemini is some people that apparently Gemini is some people that apparently Gemini is way slower in T3 code. You might think way slower in T3 code. You might think way slower in T3 code. You might think that's our fault. It literally cannot that's our fault. It literally cannot that's our fault. It literally cannot be. We're using their ACP bindings that be. We're using their ACP bindings that be. We're using their ACP bindings that they provide on their servers that we they provide on their servers that we they provide on their servers that we embed in our app. We actually did a embed in our app. We actually did a embed in our app. We actually did a bunch of weird to make it work bunch of weird to make it work bunch of weird to make it work at all because Google's a special at all because Google's a special at all because Google's a special snowflake. We'll talk more about the snowflake. We'll talk more about the snowflake. We'll talk more about the Google special snowflake Google special snowflake Google special snowflake Regardless, 30 minutes is an Regardless, 30 minutes is an Regardless, 30 minutes is an unreasonable amount of time to make this unreasonable amount of time to make this unreasonable amount of time to make this PR which when we open, we will see PR which when we open, we will see PR which when we open, we will see is closed. is closed. is closed. Because this change should not have been Because this change should not have been Because this change should not have been 1,200 lines of code. This change 1,200 lines of code. This change 1,200 lines of code. This change should have been 30. At this point, I should have been 30. At this point, I should have been 30. At this point, I was done. Especially when I read the PR was done. Especially when I read the PR was done. Especially when I read the PR which by the way, we have skills in the which by the way, we have skills in the which by the way, we have skills in the repo for how to format the PR that are repo for how to format the PR that are repo for how to format the PR that are really simple. It's supposed to open really simple. It's supposed to open really simple. It's supposed to open with a simple two to three sentence with a simple two to three sentence with a simple two to three sentence description of the exact problem the description of the exact problem the description of the exact problem the user had that they're trying to fix. We user had that they're trying to fix. We user had that they're trying to fix. We don't get that. We get a summary. We don't get that. We get a summary. We don't get that. We get a summary. We split the SQLite read accesses and write split the SQLite read accesses and write split the SQLite read accesses and write accesses in this particular file in the accesses in this particular file in the accesses in this particular file in the orchestration projection and query orchestration projection and query orchestration projection and query services that read snapshots and services that read snapshots and services that read snapshots and standalone select queries. Is this a standalone select queries. Is this a standalone select queries. Is this a run-on sentence? It is. It's unreadable run-on sentence? It is. It's unreadable run-on sentence? It is. It's unreadable slop that didn't follow any of the repo slop that didn't follow any of the repo slop that didn't follow any of the repo rules. Just garbage. So, I was already rules. Just garbage. So, I was already rules. Just garbage. So, I was already pretty ready to close it. Then I saw how pretty ready to close it. Then I saw how pretty ready to close it. Then I saw how big it was. Then I saw my favorite big it was. Then I saw my favorite big it was. Then I saw my favorite thing. CI failed. It just didn't write thing. CI failed. It just didn't write thing. CI failed. It just didn't write the code right. The check and the lint

the code right. The check and the lint the code right. The check and the lint both failed and it never bothered to fix both failed and it never bothered to fix both failed and it never bothered to fix it or take care of any of the high or it or take care of any of the high or it or take care of any of the high or higher level comments that it got from higher level comments that it got from higher level comments that it got from our review bots. So, I gave up. I opened our review bots. So, I gave up. I opened our review bots. So, I gave up. I opened up Opus 5.5. I pasted the thread ID in up Opus 5.5. I pasted the thread ID in up Opus 5.5. I pasted the thread ID in the PR and said, "This is opened by the PR and said, "This is opened by the PR and said, "This is opened by Gemini for Argon. I think the PR is Gemini for Argon. I think the PR is Gemini for Argon. I think the PR is garbage and doesn't really fix the garbage and doesn't really fix the garbage and doesn't really fix the problems that it barely stated. Go problems that it barely stated. Go problems that it barely stated. Go through the T3 code thread, figure out through the T3 code thread, figure out through the T3 code thread, figure out where it went wrong, give me your honest where it went wrong, give me your honest where it went wrong, give me your honest thoughts on the work that Gemini did. thoughts on the work that Gemini did. thoughts on the work that Gemini did. You agree it's slop, close the PR." I You agree it's slop, close the PR." I You agree it's slop, close the PR." I agree it's slop and I closed it with a agree it's slop and I closed it with a agree it's slop and I closed it with a comment that gives the reasons. Where it comment that gives the reasons. Where it comment that gives the reasons. Where it went wrong. Turn one, the diagnosis was went wrong. Turn one, the diagnosis was went wrong. Turn one, the diagnosis was mostly correct, but it had no evidence. mostly correct, but it had no evidence. mostly correct, but it had no evidence. Gemini found the real mechanism, the Gemini found the real mechanism, the Gemini found the real mechanism, the synchronous database sync runs on the synchronous database sync runs on the synchronous database sync runs on the main thread. Yada yada yada. main thread. Yada yada yada. main thread. Yada yada yada. Turn two, it fixed the wrong half of its Turn two, it fixed the wrong half of its Turn two, it fixed the wrong half of its own diagnosis. The reader pool still own diagnosis. The reader pool still own diagnosis. The reader pool still opens database sync on the main thread. opens database sync on the main thread. opens database sync on the main thread. A slow read on a reader connection A slow read on a reader connection A slow read on a reader connection blocks heartbeats the same way it does blocks heartbeats the same way it does blocks heartbeats the same way it does on the writer. The only effect is less on the writer. The only effect is less on the writer. The only effect is less waiting on this one semaphore call and waiting on this one semaphore call and waiting on this one semaphore call and that matters only when a fiber yields that matters only when a fiber yields that matters only when a fiber yields inside a transaction. inside a transaction. inside a transaction. What it's saying is it didn't work. It What it's saying is it didn't work. It What it's saying is it didn't work. It also called out that it didn't check also called out that it didn't check also called out that it didn't check prior art. There was already two PRs prior art. There was already two PRs prior art. There was already two PRs that fixed these problems. You might that fixed these problems. You might that fixed these problems. You might remember the previous video with Muse remember the previous video with Muse remember the previous video with Muse where I pointed out how much I liked where I pointed out how much I liked where I pointed out how much I liked that Muse said, "I'm not going to write that Muse said, "I'm not going to write that Muse said, "I'm not going to write code. Here's a PR that fixes it. You code. Here's a PR that fixes it. You code. Here's a PR that fixes it. You should merge this instead." This model should merge this instead." This model should merge this instead." This model doesn't do that. In fact, this model doesn't do that. In fact, this model doesn't do that. In fact, this model will hallucinate its own incorrect will hallucinate its own incorrect will hallucinate its own incorrect solution instead. The correct answer was solution instead. The correct answer was solution instead. The correct answer was in the PR tab on our GitHub and not only in the PR tab on our GitHub and not only in the PR tab on our GitHub and not only did it not find that, it invented its did it not find that, it invented its did it not find that, it invented its own wrong answer. It also solved it with own wrong answer. It also solved it with own wrong answer. It also solved it with a regex to route requests. I wish I was a regex to route requests. I wish I was a regex to route requests. I wish I was kidding. It's so bad. It was so garbage.

kidding. It's so bad. It was so garbage. kidding. It's so bad. It was so garbage. According to Opus, Gemini can read code According to Opus, Gemini can read code According to Opus, Gemini can read code and explain how it works. It does not and explain how it works. It does not and explain how it works. It does not test its own theory. It never asked test its own theory. It never asked test its own theory. It never asked whether a reader pool on the main thread whether a reader pool on the main thread whether a reader pool on the main thread stops the event loop from blocking. It stops the event loop from blocking. It stops the event loop from blocking. It answered your loose question with a answered your loose question with a answered your loose question with a confident roadmap and a percentage that confident roadmap and a percentage that confident roadmap and a percentage that it invented. When I told it to do it invented. When I told it to do it invented. When I told it to do priority one, it built that literally priority one, it built that literally priority one, it built that literally and at maximum size without checking and at maximum size without checking and at maximum size without checking whether priority one was even correct. whether priority one was even correct. whether priority one was even correct. So, I told it to go make the changes and So, I told it to go make the changes and So, I told it to go make the changes and it was 21 lines of code. it was 21 lines of code. it was 21 lines of code. They are now merged. They'll be in the They are now merged. They'll be in the They are now merged. They'll be in the next nightly and people have a better next nightly and people have a better next nightly and people have a better experience with SQLite. Real quick crash experience with SQLite. Real quick crash experience with SQLite. Real quick crash out about the instability of this model out about the instability of this model out about the instability of this model and the anti-gravity stuff. If I get my and the anti-gravity stuff. If I get my and the anti-gravity stuff. If I get my Anthropic account banned, that sucks, Anthropic account banned, that sucks, Anthropic account banned, that sucks, but I can open a new one with a new but I can open a new one with a new but I can open a new one with a new email address and I don't lose too much. email address and I don't lose too much. email address and I don't lose too much. If my OpenAI account gets banned, If my OpenAI account gets banned, If my OpenAI account gets banned, slightly more annoying, but again, I slightly more annoying, but again, I slightly more annoying, but again, I open a new one, I'm good. I might sign open a new one, I'm good. I might sign open a new one, I'm good. I might sign in to both of those with my Google in to both of those with my Google in to both of those with my Google account, but if I'm banned, it's just my account, but if I'm banned, it's just my account, but if I'm banned, it's just my sign-in to Anthropic or OpenAI that's sign-in to Anthropic or OpenAI that's sign-in to Anthropic or OpenAI that's banned. If my Gemini account gets banned. If my Gemini account gets banned. If my Gemini account gets banned, I might lose my YouTube channel banned, I might lose my YouTube channel banned, I might lose my YouTube channel cuz my Gemini account is my Google cuz my Gemini account is my Google cuz my Gemini account is my Google account. So, I could lose my email, I account. So, I could lose my email, I account. So, I could lose my email, I could lose my billing system for my could lose my billing system for my could lose my billing system for my credit cards, I could lose my saved credit cards, I could lose my saved credit cards, I could lose my saved passwords in Chrome, I could lose my passwords in Chrome, I could lose my passwords in Chrome, I could lose my access to my YouTube channel, I will access to my YouTube channel, I will access to my YouTube channel, I will lose my access to everything else I sign lose my access to everything else I sign lose my access to everything else I sign in to with Google. There's a real risk.

in to with Google. There's a real risk. in to with Google. There's a real risk. They have promised that they will only They have promised that they will only They have promised that they will only ban you from Gemini and anti-gravity. ban you from Gemini and anti-gravity. ban you from Gemini and anti-gravity. They won't ban you from Google anymore. They won't ban you from Google anymore. They won't ban you from Google anymore. But, I've seen people getting their But, I've seen people getting their But, I've seen people getting their Google accounts banned still, so it's a Google accounts banned still, so it's a Google accounts banned still, so it's a risk. risk. risk. As such, integrating their official ECP As such, integrating their official ECP As such, integrating their official ECP into T3 code felt risky. We chose to do into T3 code felt risky. We chose to do into T3 code felt risky. We chose to do it because I was told it should be fine, it because I was told it should be fine, it because I was told it should be fine, but I can also tell they're not putting but I can also tell they're not putting but I can also tell they're not putting a whole lot of effort into this, which a whole lot of effort into this, which a whole lot of effort into this, which you can see in places like here, where you can see in places like here, where you can see in places like here, where the model just failed outright to the model just failed outright to the model just failed outright to connect to the gateway in the middle of connect to the gateway in the middle of connect to the gateway in the middle of a run after reading endless files for an a run after reading endless files for an a run after reading endless files for an hour and 2 minutes. hour and 2 minutes. hour and 2 minutes. And then it just failed and refused to And then it just failed and refused to And then it just failed and refused to continue, and I had to restart this continue, and I had to restart this continue, and I had to restart this entire thread with none of its existing entire thread with none of its existing entire thread with none of its existing history. Horrible. And then there's a history. Horrible. And then there's a history. Horrible. And then there's a bunch of silly little Googleisms, like bunch of silly little Googleisms, like bunch of silly little Googleisms, like they don't report the model and then let they don't report the model and then let they don't report the model and then let you pick effort in the ECP. All of the you pick effort in the ECP. All of the you pick effort in the ECP. All of the model slugs are paired with the effort model slugs are paired with the effort model slugs are paired with the effort levels. So, you have to pick for Argon levels. So, you have to pick for Argon levels. So, you have to pick for Argon high, medium, or low as a model. Every high, medium, or low as a model. Every high, medium, or low as a model. Every other lab gets these things right. ECPs other lab gets these things right. ECPs other lab gets these things right. ECPs are pretty simple standard to implement are pretty simple standard to implement are pretty simple standard to implement and integrate. Google is Google, and and integrate. Google is Google, and and integrate. Google is Google, and they got all of this wrong. It is they got all of this wrong. It is they got all of this wrong. It is embarrassing how miserable it was to embarrassing how miserable it was to embarrassing how miserable it was to integrate anti-gravity and how high a integrate anti-gravity and how high a integrate anti-gravity and how high a percentage of the bugs in T3 code are percentage of the bugs in T3 code are percentage of the bugs in T3 code are just for anti-gravity. It has just for anti-gravity. It has just for anti-gravity. It has consistently been a thorn in my side consistently been a thorn in my side consistently been a thorn in my side since we first integrated it, and even since we first integrated it, and even since we first integrated it, and even having Gemini 4 did not make it better.

having Gemini 4 did not make it better. having Gemini 4 did not make it better. It was a relief to have a reasonably It was a relief to have a reasonably It was a relief to have a reasonably usable UI for a Gemini model for the usable UI for a Gemini model for the usable UI for a Gemini model for the first time ever. I honestly thought it first time ever. I honestly thought it first time ever. I honestly thought it would make me like it more. But, now would make me like it more. But, now would make me like it more. But, now that I see it next to models that are that I see it next to models that are that I see it next to models that are competent, like Muse, Opus, even like competent, like Muse, Opus, even like competent, like Muse, Opus, even like Sonnet, honestly, Sonnet, honestly, Sonnet, honestly, it just hurts more. And now we get to it just hurts more. And now we get to it just hurts more. And now we get to the end, comparing it to the other the end, comparing it to the other the end, comparing it to the other models. This feels unfair. models. This feels unfair. models. This feels unfair. Feels unfair to the other models. This Feels unfair to the other models. This Feels unfair to the other models. This is just garbage. Don't ask me about how is just garbage. Don't ask me about how is just garbage. Don't ask me about how good the usage is on the $200 plan or good the usage is on the $200 plan or good the usage is on the $200 plan or anything. I don't care. I'm not going to anything. I don't care. I'm not going to anything. I don't care. I'm not going to push the limits enough to know. push the limits enough to know. push the limits enough to know. It's probably practically unlimited It's probably practically unlimited It's probably practically unlimited because you're not going to do a whole because you're not going to do a whole because you're not going to do a whole lot with it cuz you're going to get lot with it cuz you're going to get lot with it cuz you're going to get frustrated and give up. It's just It's frustrated and give up. It's just It's frustrated and give up. It's just It's just bad. I'm going to put my money just bad. I'm going to put my money just bad. I'm going to put my money where my mouth is here and show you just where my mouth is here and show you just where my mouth is here and show you just how committed I am to this. I don't how committed I am to this. I don't how committed I am to this. I don't trust any Gemini models nor Muse Wonder. trust any Gemini models nor Muse Wonder. trust any Gemini models nor Muse Wonder. To be clear, Muse Wonder full is fine, To be clear, Muse Wonder full is fine, To be clear, Muse Wonder full is fine, but the standard Muse Wonder as well as but the standard Muse Wonder as well as but the standard Muse Wonder as well as Gemini 3, 4, etc. They do not contribute Gemini 3, 4, etc. They do not contribute Gemini 3, 4, etc. They do not contribute useful code the T3 code code base. I useful code the T3 code code base. I useful code the T3 code code base. I would like to find some way to auto would like to find some way to auto would like to find some way to auto close all pull requests that are filed close all pull requests that are filed close all pull requests that are filed using models like Gemini 4 Argon or the using models like Gemini 4 Argon or the using models like Gemini 4 Argon or the normal Muse Wonder normal Muse Wonder normal Muse Wonder when they are filed to T3 code. I am not when they are filed to T3 code. I am not when they are filed to T3 code. I am not joking.

joking. joking. I will find a way to get this in later I will find a way to get this in later I will find a way to get this in later this week. I have wasted more time in this week. I have wasted more time in this week. I have wasted more time in tokens trying to vet code that was tokens trying to vet code that was tokens trying to vet code that was written by Gemini 4 than I would have written by Gemini 4 than I would have written by Gemini 4 than I would have spent writing the code with a smarter spent writing the code with a smarter spent writing the code with a smarter model in the first place. Gemini is not model in the first place. Gemini is not model in the first place. Gemini is not a useful model for anything in the code a useful model for anything in the code a useful model for anything in the code world other than maybe as like a first world other than maybe as like a first world other than maybe as like a first or second pass on a nice looking or second pass on a nice looking or second pass on a nice looking marketing page. As soon as I'm done with marketing page. As soon as I'm done with marketing page. As soon as I'm done with this video, I'm just not going to use this video, I'm just not going to use this video, I'm just not going to use Gemini for anything other than like Gemini for anything other than like Gemini for anything other than like fetching data for YouTube. This is one fetching data for YouTube. This is one fetching data for YouTube. This is one of the worst releases of the year. I of the worst releases of the year. I of the worst releases of the year. I don't care what the benchmarks say. I don't care what the benchmarks say. I don't care what the benchmarks say. I have been testing it constantly in real have been testing it constantly in real have been testing it constantly in real world work and I have not merged a thing world work and I have not merged a thing world work and I have not merged a thing that this model has shipped. This feels that this model has shipped. This feels that this model has shipped. This feels like a throwback to 2024 with some of like a throwback to 2024 with some of like a throwback to 2024 with some of the hallucinations I see this model the hallucinations I see this model the hallucinations I see this model making. I was played up to this point making. I was played up to this point making. I was played up to this point because I'm thankful for the Google because I'm thankful for the Google because I'm thankful for the Google employees I've talked to. They have employees I've talked to. They have employees I've talked to. They have generally been one of the more receptive generally been one of the more receptive generally been one of the more receptive labs to my feedback and I tried to share labs to my feedback and I tried to share labs to my feedback and I tried to share this feedback with them early. I've also this feedback with them early. I've also this feedback with them early. I've also heard rumors that there is a better heard rumors that there is a better heard rumors that there is a better snapshot like carbon or something that's snapshot like carbon or something that's snapshot like carbon or something that's coming soon. I don't know how much coming soon. I don't know how much coming soon. I don't know how much better it's going to be, but it needs to better it's going to be, but it needs to better it's going to be, but it needs to be a lot because this model is garbage be a lot because this model is garbage be a lot because this model is garbage and I have no intention to ever use it and I have no intention to ever use it and I have no intention to ever use it again other than for fetching data on again other than for fetching data on again other than for fetching data on YouTube. Tell me what y'all think. Am I YouTube. Tell me what y'all think. Am I YouTube. Tell me what y'all think. Am I overreacting massively or is this model overreacting massively or is this model overreacting massively or is this model actually just garbage? They told me it actually just garbage? They told me it actually just garbage? They told me it was good at code and it's not good at was good at code and it's not good at was good at code and it's not good at code, so I'm I'm pretty done with it.

code, so I'm I'm pretty done with it. code, so I'm I'm pretty done with it. Maybe you can find some cool uses. Let Maybe you can find some cool uses. Let Maybe you can find some cool uses. Let me know in the comments if you do. Until me know in the comments if you do. Until me know in the comments if you do. Until next time, next time, next time, I'm sticking with Muse. I'm sticking with Muse. I'm sticking with Muse. Okay, seriously, I'm probably just going Okay, seriously, I'm probably just going Okay, seriously, I'm probably just going to stick with Opus 5.5. You get the idea to stick with Opus 5.5. You get the idea to stick with Opus 5.5. You get the idea though. I just I'm out. Bye.