OpenAI DevDay, Rails World, Omarchy Meetups, & More

Yeah. Welcome.

here we go.

Welcome back to another episode of the Not Brothers Podcast. where today we're kinda gonna bounce around topics again. they're all loosely intermingled because I've just spent the past couple of weeks traversing between Rails World and OpenAI D developer day, a little Omarchy meetup. just been a good time just kind of connecting with people, talking about things. listening to keynotes and and seeing where things are. So I haven't had a chance to talk to Mark since really I I started that journey about two weeks ago. so we figured it'd be a good time to just talk through some of that and yeah.

Well, first off, I'm glad you're doing this. I mean you just got off a red eye like what, eight hours ago or something?

I don't know what time it all my time zones are fucked

Yeah.

up on my laptop right now, actually. So yeah, probably not even actually like four hours ago. so yeah, I came frig came straight back from San Francisco. Although I will say, you know, my other option was to come back today. I think it w left there at like two thirty and so it got back here at like seven something. And it's she kinda like blow the whole day. Yeah.

You either lose a whole day or you lose a night. Yeah.

Versus you know, the red eye last night I d I went to the Omarchy meetup out at Cloudflare's offices and then h went across the street, hung out with a couple of guys afterwards, and after you know, hung out with them for like forty five minutes, an hour, got an Uber to the airport, almost left one of my bags in Cloudflare's office. And then popped on the plane and went straight to sleep. So actually worked out pretty well.

Nice.

yeah. But you know, we were talking kind of pre-show a little bit about some of the the things that we should talk about and and changes that are afoot. Obviously this is gonna be very a AI pilled given that I've just been inundated with everything AI for the past couple of weeks. in chronological order, I suppose we would start with, you know, some of the things that came out of Rails World. So I went there with with Jake, our is he director of technology for us. leads our development teams and technology teams and put a really good time out there. Obviously we got to spend some time with With David and Spencer from the Omarchy Core team. and you know, David's keynote from Rails World was was a fun one. It was I knew going into it because I talked to him, like he like, It's gonna ruffle some feathers and it's gonna be a little spicy. Like, great, I love it. and that's exactly what happened. And the inter the most interesting thing is like The difference in the discourse online and the difference in the discourse there were like kind of polar opposites. And I'm not exactly sure why. in that like online was really negative. like they had a really negative reaction to to everything that was said, pretty much. and being like, Well, you know, Rails is dead and all these things. You're like, that's not exactly what we're saying here. We're saying is the is the paradigm has shifted clearly. The the winds have shifted and it behooves you to not stick to your old ways and shift with them. On site, you kind of had some people who, you know, question some of that stuff, but also, you know, could at least understand would it would understand those perspectives of, you know, hey, we can't we can't keep doing what we're doing when other people have superpowers because we will get passed up. And what that means for individual languages, what that means for frameworks, what that means for you know, anything at all, nobody fucking knows. you know, we were there. Obviously David was talking about how they they rewrote some of or all of Hay and Rust and it was so much more performant and it's and it's easier to run, like the you know, all those things. Which makes sense because Rust like at its core, Hey is a male server, and Rust was being incredibly fucking good at being a male server. It'd be incredibly bad at a lot of other things. So I think using the ability to use the right tools for the right jobs now becomes more possible than ever when and before you kind of always had to choose what camp you lived in. Like you're a PHP guy or you're a Rails guy or you're a a Java person, unfortunately. you know, all these other you know, sectors, because you can't learn all of them, you can't know all of them and be proficient at all of them. However In today's day and age, I kinda can. If I know good good development practices and principles and the the ways of building good user experiences, and just good application architecture in general, I could build it in whatever fucking language I want. I could a in fact, David rewrote well, he didn't rewrite, his agent rewrote, but just for fun, out of curiosity, rewrote the screensaver. So screensaver that we use for Omarchy is based on a project called TTE. that was originally a Python project. And Python is god awful at library management. That's why UV exists and all this other stuff. But what could happen is that if you had for whatever reason when it tried to execute, it grabbed the wrong Python binary, it could puke all over itself and then die. So we're like, well maybe we re rewrite that And something else. I had a an instance where I started down the path of write rewriting and go. It was pretty close. the agents just weren't able to get the age animations a hundred percent right. And then more recently with some of the newer ones, David rewrote it in Rust and it's substantially faster than Python, doesn't have the relia the reliability issues, all those things went away. and then after his keynote, he rewrote Fucker in Assembly and It was even faster. now insane. Like the the idea of writing an an application in today's day and age in in assembly is insane. But for something like that, like it's a pretty purpose built thing. We just need it to work. it draws characters on a screen. yeah, maybe it is just an assembly application and you know, we don't need any abstraction layers at all to have to work through. so it can be incredibly performant, it can be super small and it just ships as like a little binary. so those have been

My favorite line from my favorite line from his entire keynote was actually, what do you say, 10 years ago or so when he recorded himself doing some version of Rails and like introducing and his he was very, very excited. He was like, Look at all the things I'm not doing. Look at all what I'm not

Yeah.

doing. And then he played that, and then he's like, Look at all the things I'm not doing, talking about

Yeah.

agents in today's world. And it's it

I mean hasn't changed and I think that's the interesting thing that like Rails was born originally out of the the desire to not do a bunch of shit to be able to, you know, scaffold things and have frameworks where you could just build a ton of things, produce a ton of code, you never look at it. It was the same I same exact idea. It's just the way in which you produce it is different. And I think that there, you know, there's lots of issues that people take with with AI, right? There's reliability, there's the you know, the political bullshit, there's climate impact, there's you know, there's no shortage of of challenges that anyone could take with it. But you know, I think that to say it's not it's not an incredible tool that should be taken advantage of, I think it's just ignorant. and anybody that takes that stance, I think is just ignorant. it's just that. Th there are plenty of projects that are taking that stance now where it's you know, anti-AI. we've seen some of the projects that have benefited Omarchi, like Asahi. They took a stance that, you know, there's no AI contributions to reverse and I mean, you motherfuckers are reverse engineering Apple hardware. Let's not get haul all high and mighty here. and they're like, nope, no AI, whatever. I'm like, you what AI would be really, really fucking good at? Reverse engineering some fucking hardware. So

Yeah.

we have the Aurora team at on Omarchy that's basically a direct work of Asahi. And we're like, okay, it's like a Sahi, but use agents to pound on the front door. Why would you do that by hand? It it it makes no sense at all. anyways, yeah, I think that you know, no doubt it was it was I think a good keynote. kind of kick things offline. others can have their opinions and that's fine. and I think it it at least did what what I think a keynote should and that's get people talking about stuff. It started a conversation that's a necessary conversation amongst everybody to figure out like hey what the hell is does the future look like because we can't pretend it's gonna look the same as it does. It was also interesting going to some of the talks and not to you know definitely not to throw any shade at any of the people who did some of the talks, but there were a good number of the talks that were kind of the opposite where, you know, they were in some cases just pr very proud of, you know, how a library or something is more optimized for readability and easier to write and those sorts of things. And I kind of found myself walking away from those, at least at first, a little bit disappointed. Like, I kind of don't care about that anymore. I I would prefer to see, you know, how did you apply the library or how did you how were you able to put this into practice and be successful? I don't care as much about exactly what it looks like. Now, after thinking about it more, I think there's maybe a way that both of those win. In that you know, I still think that Ruby as a language is a very, very good language for agents because it is a simple syntax, it's it's a very efficient syntax, it's an easy-to-read and understand syntax. And while Maybe I don't read the code. It is nice that if I crack it open, I can at least it at least looks like fucking English. but even if I don't read the code, the agent still does. So, you know, I kind of equate it to a project that I built for for Oodle, it's called Mosaic, where we had a lot of people on the team generating artifacts, and what we found is it would take a lot of time and they were inconsistent. And it was because they were generating HTML, JavaScript, and CSS every time. So tons of like far more token usage, which led to burning more tokens, led to it taking longer to create them, and just led to more drift amongst them. So it created Mosaic, which basically allowed the agents to do all of their work in Markdown and have a single rendering engine that sort of an MDX style syntax could render through. And the Nice thing about that was now the agent just has to deal with Markdown and this MDX syntax, which is far more token efficient, which meant that everything happens faster, and also meant that the outputs are more consistent. And I think that like something like Ruby could still benefit from those sorts of things. So maybe there are benefits to optimizing libraries to being, you know, maybe human friendly, but also maybe there are specific things to making them agent friendly. Or perhaps what what is human friendly is just automatically agent-friendly. and we find ourselves in a similar world to like it's like SEO. Remember SEO used to be SEO used to be a very technical thing, and there's probably some that would still argue it's technical today, and I would argue it's not. SEO is really simple. Make good content that humans want to interact with and see, and you will have good SEO. That's it. That's all you have to do. I think we may see the same thing

So you

happen here.

you said earlier, and I I think this is probably the right way to look at it, from a from a non developer's perspective, the way I think about it is a a toolbox full of similar tools, but those tools have slightly different purposes. So think about like screwdrivers, right? A screwdriver is a screwdriver until you need a Phillips and you only have a flathead. Still screwdrivers, right? But one has a very specific purpose versus the other. I'm thinking about coded code languages.

Yeah, but if you try hard enough, that motherfucker can be in a hammer.

Fair, right? But like you so I think about code bases and syntax in very much the same way, right? You have specific tool sets that can be used for specific job types. And, you know, if you want that super any, you know, super tiny screwdriver that is used for for glasses repair, great. That's a c specific code syntax in language. It's a different tool that you can use for the for a different job type. You wouldn't want the standard, you know. screwdriver, you know, P number two or whatever the frick it is for repairing your sunglass your your glasses. It just doesn't work. It's not the same. And I think about code languages and syntax in in kind of that same light. because it, you know, as as a developer, you if you try if you spend long enough with a syntax, you can figure it out. Right. You it may not be your first your first native language or and it may not be your your favorite to look at as David Page Rust, but you can figure it out.

We ever looked at Rust?

No, I have not looked at Russ, but I don't think I want to now.

I'm not sure I can figure it out ever. Not sure

Yeah.

I would be I would have the patience to figure it out. it it's a god awful like it's a syntax that only the creator could love. and I'm sure it's for some fucking reason that makes sense and people I don't even know. Like the only thing I've ever known about Rust is that it's god awful to write, but god damn is it fast. Like whatever

Okay.

voodoo magic they have in there. it's extremely fast. And then you have you know other languages that are simpler to write, but not as fast. You also have one of the things that came out of Rails World that I is that Matt's the guy you know who created Ruby, he's working on his Spinel project or spinel, I don't know how to pronounce it. which basically is compiled Ruby, which was kind of interesting because I I I did an experiment with Something. I don't remember what. I grabbed something off the shelf and was like, hey, run this in Ruby Spinel and and in Rust and just kind of compare the two and see performance wise how do they do? And they were neck and neck. we're we're fighting for microseconds at this point. But they were neck and neck, which was pretty impressive considering that the non compiled Ruby version of it was incredibly Like it was destroyed by the Rust version in terms of performance. it was it was it was Oma Reel, the the the screen recording application that I wrote and I had it rewrite it, rewrote the whole thing in Ruby and then compiled it using Ruby Spinel. And it it worked, it was good, it was fast, it was just as good as as the current Rust backend or C plus plus back end that it exists in. So that's interesting. You know, it's also possible that languages and things could evolve through tooling to again, like I think Ruby as a syntax is a good syntax. I think it's a good syntax for agents even to understand, for all the reasons that it's a good syntax for humans to understand. so there is a world where like maybe the tools converge and what's good for agents is good for humans. I don't know. That's that that that could be the ultimate benefit too that we start scratching around and finding because it doesn't in my opinion, it doesn't benefit us to have something that is not performant for agents to work in, that maybe squeaks out a little bit of performance. So like if if we could find that, you know, hey, I could write the same application in half the time using Ruby with Ruby Spinel, I can write it in half the time with half the token usage, then I can the same Rust application and get comparable for performance. It's a fucking no brainer. I would I would say you should go the Ruby route and forget Rust because you're saving time and you're saving tokens and you're getting the same output.

So what what are your top takeaways from Rails World, let's say? Maybe that's how we organize some of these adventures that you've had over the last couple of weeks, because there's quite a bit.

Rails World the the top takeaway from Rails World would be that you know, the winds are shifting and if you're somebody who tied their identity to being a programmer It's incredibly important that you understand those shifting winds and shift with them. in whatever direction that means for you personally. I think that any programmer that just kind of tries to exist and, you know, continue handcrafting code forever, like if you want to do it as a hobby, that's fine. But I'm talking, you know, professionally and c being competitive in the marketplace. I I think those days are numbered if not over. now, you know, the knowledge that comes with that that has been a that has been built over time, I think is incredibly value, probably more valuable now than it's ever been. you know, that's how we make sure that these things are producing proper code or not leaving security holes or you know, all these other things is it's the experience itself, not necessarily even writing the code. Like I can exploit all kinds of stuff. I have no idea how. you know, what the code of Windows looks like, but I can tell you how to exploit some stuff in it. at least prior to patches. so I think that that would be my takeaway is like, you know, I think we're in weird times right now. And I don't think any one person knows what'll what we'll be looking at two years from now, or even a year from now. so I think maintaining curiosity and and exploring and experimenting along the way. not locking yourself in. That's why I think that the the stance of saying like, you know, no AI on a project, I'm like, that's a butt's a pretty ignorant stance because you really just don't know. I would have agreed with you a year ago. 100% would have agreed with you a year ago. Because the agents were fucking stupid. And they were even, you know, they were creating headaches for me where people would lob PRs for for a Marchie. And I'm like, this is dog shit. It like none of this makes any logical sense. You've engineered some solution to a problem that doesn't exist. And no. And now I just have to sift through this stuff. Now it's a little different. The you know, the there's still plenty of stupid things, but they're usually functional. it's usually the idea that's bad. But the you know, the quality of code that you're able to produce with agents, the complexity of things that you're able to throw at them, the length that they can run. entirely unassisted or or you know not interacted with is incredible. so I think

So let's talk about that for a second, because that that's a relatively that's probably the last two or three months. And we've we've been on record in this podcast probably every month saying something like, Yeah, we've we've probably hit the plateau. It's they're getting slightly incrementally better, but we're probably getting close to the point where the the improvements are gonna gonna slow down. We've said that many, many times from probably February to July. And then somehow,

I think that's still true. And I think we're

like July to October, it feels like the quality ha like that that quality gap has closed.

I think it has, but I don't think it's been through monumental leaps. I think it has been these small iterations and and changes, right? If I think back to And some of its market pressures, right? When open claw was introduced, it was built on top of Claude. And all of the harnesses got tuned really well to work on Claude. And then Anthropic Rug pulled everybody and said, You can't use your subscriptions. Everybody said, Go fuck yourself. We're going to OpenAI because they embraced it. But at the time, you know, if you tried to use Claude on OpenAI, I mean it would make you want to punch a fucking baby. They it was, you know, it would you you would be like, All right, make make a website that has a header and a footer and a team, you know, page or some shit. And it'd be like, Okay, I'm gonna make a website that has a header and a team page and a footer. Would you like me to continue with that? Like, yes, motherfucker. I just told you. It's like, okay, I'm gonna create the header now. Is can I create the file? You're like, for fuck's sake. Yes, please stop asking me questions. If you ask me one more question, I'm gonna lose my mind. And and that's how it was for you know, months. And then they released a new model where they were like, Hey, we tuned it to not fucking do that anymore. And, you know, didn't get it right that time, didn't get it right the time after that, but they've continued to get better. And I think that's some of what we're seeing. Like obviously the models are getting a little bit small, like they're getting a little bit smarter, they're getting a little bit bigger. But they're not getting monumentally smarter or bigger. What I do think they're getting is better at executing certain tasks. So they're figur they're better at spawning sub agents and executing multi-layered activities. They're better at long running tasks. So being able to give what's this one that I gave it? that's isn't even particularly long. This one ran completely untouched for over an hour. And you I don't even know how many tokens or sub agents that it spawned in that process, but made a substantial amount of of work. I've had some run for three hours. I've had some run for six hours, just chugging away on a problem throughout the entire day. And I think that that was not possible before. And you could have this if you had the same model, if it would just keep chugging away, you would get better results than we were able to achieve before. So I think that, you know.

So you so you think the the self improvement side and self pressure testing of being able to just continuously run, find a hole, plug it, continuously run, find a hole, plug it, that's really the difference that you've seen in the last couple of months.

Maybe it's hard to say. I don't know how these models are built from the ground up anyways. It's just it seems like, you know, there's only so much knowledge to suck in, right? So I think that there's only so smart that the the the model itself can be. There's only so many parameters that it can consider with every one token before it's just not performant enough to be useful. At that point, the question is, what are the gains after that? And I think there are a few areas where you can get those gains. One of which is just how the harnesses interact, how subagents work, how some of that tuning is that like makes it not stop and ask for permission, but still have proper safeguards that you know protect it from wiping your system or allowing someone to red team some application that's not theirs. So I think a lot of the benefits that we've seen, certainly some have come from the model improvements themselves. I think the others have come from a lot of harness improvements and just how the models interact more so

Mm-hmm.

than the the model itself, if that makes sense. not knowing

Yeah, that makes sense.

what the language is to s to, you know now somebody from OpenAI could probably give me the actual terminology to use to describe what the hell I'm trying to say. But I don't build models and I don't know. I think the the interesting thing is I think that this leans into what I what I believe is the next frontier of where we get massive performance increases and cost reductions. And that's mini models. I think that there's you know, we keep focusing on these big frontier models with, you know, Astra and Opus and and Fable and you know Whatever else comes after these. And, you know, those models are incredible. What do we what was the Gemini one that just er it's not Gemini? Google's new one.

I saw that come out, but I can't remember the name.

Something something four. Anyways, doesn't matter. I I think that those models are great. They will always have a good a good place and they will probably set the pace and standard for

Argon.

everything. Huh?

Gemini four Argon.

Argon. I was thinking Gorgon, but that's the new that's

Close.

the new AMD chip is the Gorgon Halo chip.

I was thinking stranger things, but you know.

so I think that you know when we think about like those big models, they have to hold the whole of the universe, right? They know you know, how many inches are in a foot and also who won. You know, World War Two and You know, who was the you know, or how many dynasties have there been in fucking China? I don't know. Like they know everything. which means that when I ask a question, all of the weights for all of that those answers are in that's being addressed in that token in some way, shape, or form. Might be completely irrelevant, but it's still there. So I think about you know some of the experiments that we've done even with like our vacation rental property and and other things where you can take a really small model, like a like the you know, a quinn I think it was like the point seven B one that I used, or like a Quinn two billion forever, a really, really small model that can run on my laptop, any laptop. can run a CPU compute and still be fine in some cases. And we train it on one thing, and that's how to answer any question that happens in our business. I don't need you to know anything else about anything else. I just need you to know how to answer the questions that are in our business. And you could say, like, well, we'll just take a really big smart model and we'll just feed all that in in the you know, in the pre prompt, or we'll give it a vector vector database and vector store and all those things, which we did, right? We had all those things. In tandem with this experiment. and what I found is the small model did better. Because the small model, when somebody asked a question weird or whatever, it didn't have resources to go get answers and and fabricate things. It only knew what it knew. And those the things that it knew are in the actual fucking weights. It's not something it has to go think about. And like maybe it consults the index, maybe it doesn't consult the index this time. Maybe it thinks, I've got this, let me just YOLO it. I know where, you know, the garage door opener is. And you're like, joke's on you. These houses don't have garages. The garages have been converted to game rooms. But it doesn't change the fact that we've had guests ask us, like, how do I open the garage? And and you know, with a A frontier model, sometimes the agent would be like, the garage door opener, the button is right inside to the right of the door, because that's where they fucking always are. But in our houses, they're not, they don't exist. so I think that's going to be a huge opportunity is how do you take, you know, how do you take the whole of your business and

Mm-hmm.

distill it down into something that can be used as post training data on any model, but also specifically smaller models that could then be self hosted, that could then cut down on your overall costs, increase accuracy, increase speed, all these things. Cause that's the other thing is these small models, we can run these small models at at hundreds and thousands of tokens per second. So you're talking like

Well then theoretically it's also far more secure because they are literally sandboxed and don't even have access to any of them.

Yeah, it can be air gapped as well. So if you're in a in a a regulated market like healthcare, or something like legal, you know, those sorts of places where maybe the data that you need to pass in or you'd like to pass in is somewhat sensitive in nature, those are great because you could establish a workflow where you could still have access to s to these sort of superpowers, while also still maintaining privacy and all the things that are Definitely necessary to do.

So related but not related to that, going in chronological order, you spent roughly a week at Rails World and then the following week you went

Yeah, there you

to an OpenAI development event. So talk about that OpenAI development event. What what was interesting there? What was what was announced? What was the vibe and energy like? what what was that event like?

Yeah, I went to OpenAI dev day. That was my first dev day that I've ever been to. and I guess last because they they talked about not calling it dev day last year or next year. But the It was a really cool event. I think one of the things that struck me immediately actually was in kind of the main hall. they had a lot of just like tables set up and seating areas and like it wasn't it wasn't just chairs facing a stage, although they had some up front. It was a it was more of a space that you could just I mean you could literally just hang out there and do some work while you're listening to some of the talks and things. So Thought that was really unique way to set up a space and just I mean it forced some conversations also because when you have these kind of long skinny tables and you have people sit down, you're naturally just sitting down beside and across from people. So it's pretty standard to just kind of be like, hey, nah, what are you working on? and make some introductions. I think that I mean that's probably part of the point, right? Is to spur collaboration and conversation.

How do we make a bunch of nerds talk to each other?

Yeah. How do we shove a bunch of introverts in one place and make interact? it was cool. I they they what'd they have? They had a bunch to announce. they announced dots, which is, you know, their version of Grokbot and you know Claude's desktop thing and But really, I mean it's more probably akin to what Buzz did with their agents in Buzz, where you you know you kind of start with a few agents and you can create more agents and those agents just kind of live and are basically coworkers. they have a little bit of a a spin to them. and they also have the advantage of just kind of snapping in with the chat with the ChatGPT connectors. So there are tons of connectors pre-built, so you can just kind of connect everything up and then You know, your dot can just help you with whatever it is you're working on. I think I gave you an example where Jake set his dot up and connected it to Slack. And one of the things I guess that it does is it just kind of chills in the background and it'll try to be helpful throughout the day. And Jake and I were talking about open claw enterprise, and I was like, I wonder if it has the ability to pass, you know, individual permissions or something like that. I don't remember what the question was. But he was like a little bit later that evening, it actually pinged me and was like, Hey, I saw you were talking to Ryan about this. it does, and here's the link to it. so that was pretty cool. I I thought that was a neat thing. You know, whether or not dots will be a game changer for me personally, I'm still not sure. I think first

Is it almost like more of a personal assistant sort of thing? Is that the idea based on whatever you're working on?

Sort of, yeah. But the the challenge for me is like figuring out, you you've got Grogbot, you've got Dots, you've got, you know, all these different assistant things that exist. And I'm trying to ratify in my head where where this fits with things like open claw and Hermes and and other ones. Because like for me, I like my open claw agent. it's open, I ha I give it its own box, it has the ability to do things that I need it to do. You know, it can access things like Basecamp using a the Basecamp CLI, which isn't available on ChatGPT, so now my my dot has no way of understanding what's going on in Basecamp or or acting as a an entity in Basecamp. So you just have all these different things. So I think that like for some users, maybe it'll just be, you know, it's a low barrier to entry to to establishing something like that. So I think it's a fantastic thing. And then I think for other users like me that are maybe po more power users or have more specific requirements, might be some the question is like, do I also use that? Or do I just ignore that and use the things that I'm already using? And I'm not sure. I could see some benefits for both. I could also see a world where like eventually they start to say the same shit and I kinda just need one agent to help me drive things, but who knows?

so I haven't even I haven't done a lot of exploration with with Dots yet, but the way that you're describing it sort of it reminds me of what Siri is attempting to be, which is sort of that low barrier to entry. I'm familiar with a lot of different applications and can help you with certain things with without you even asking. it sounds like that's what OpenAI is attempting to do is is maybe a way to draw other people into the universe and get that low barrier to entry of connectivity, which quite frankly hasn't been super duper easy to really figure out. Cause even, you know, Claude has connectors and OpenAI has connectors to different things and whatever. But to get it to actually be helpful without prompting it to be helpful is pretty much irrelevant. It doesn't just do those things in the background. So whereas Siri will, right? So if you're texting them texting someone with a new Siri and you say something like, you know, birthday event. It'll prompt you and say, Do you, you know, send an invite for a birthday event or or whatever right there on the screen, right? That sort of reminds me of what the point is of that. So instead of being a deliberate prompt, it's sort of listening and thinking and saying, Is this this? You want to help with this? so your example with Jake is kind of a perfect one of actually I did the research for you and here's the answer to your question that you actually didn't ask me.

Yeah. that was kind of the big the the big thing that they started with. They announced a they announced a new model, the GPT six point one soul model. Which from some testing looks pretty fucking impressive. it can on some cases I think achieve like almost astra level results, but for an absolute fraction of the price. So that's gonna be a really interesting model to do some I haven't really done anything with it yet. but it's gonna be a really interesting model to experiment with. It actually reminds me a lot. Of Opus five point five where you know, just recently Anthropic released Opus Pi point five and it's better than Fable. It's

Mm.

like just for what it does and the speed that it moves at and it's it's basically like a less verbose fable is what it feels like. which just makes it shut the fuck up and do its job that much

Yeah.

better and faster. It's fantastic. So it's faster, it's cheaper, and it seems to be as smart. So the question is like, is this the same thing where they've kind of just like taken Astra and distilled it to make it wicked wicked fast by comparison and extremely cheap. speaking of speed, they also announced Ultra Fast. It's only available on GPT A six Astra at the moment. So only the flagship model. But ultra fast, they've always had fast. Fast was 2x speed, I think it was 2x billing rate, right? so you just churn through your tokens twice as fast. They all they released kind of in tandem a $500 Pro Max plan, which gets I think 25X pro in terms of usage, whereas the Pro 200 was previously, I think, 10X or fifty, I think it was 10X. So you get a little a lot more usage. no five hour cooldown anymore. They got rid of that. And the $500 plan is the only one that can do ultra fast. ultra fast is 8x performance with a 6x bill rate. So it is incredibly fast. It can do like 300 tokens per second on GPT 6 Astra. But goddamn, does it a just like slurp tokens? it

That's insane that they can even offer that. It's already relatively fast, right? So when you think about

Yeah.

what it does and the amount of time that it it does it, and especially for the, you know, for models like Seoul and Opus in particular, that aren't as verbose as the the the super, super high end models, the fact that they can make those things even faster and apply it to the top model is kind of ridiculous.

you haven't seen. There's some things with what is it, Cerebus, Sara Cerebros? they're running like three thousand tokens per second, like crazy. Like you hit enter and the result is there and it's fantastic.

That is insane.

yeah. That's where that's the future that I want to be in, is you just you think at this like you can create at the speed of thought.

This is the this is the guy that runs ten gigabit a second at home.

Yeah. Who who does it? that's when those fifty gig lines start to look really attractive. But

Right.

I think the you know, I used it yesterday. I've got some banked resets. So I was like, well, let's see what we can do here. And you know, I had a a presentation I needed to pull together. I used the the hype app to do it and which is very agent friendly and I was able to Open up hype, and you know, I could tell it like, Hey, let's stub out this, go do this read, you know, go pull some information from here, pull this information from here, consolidate it. You know, here's kind of the structure I'm looking for, whatever. And then just hit go. And I could just sit there and watch it make slides. which you could always do, but it would not, it would make like one. And then maybe start the second one. And like it'd be a long process. It stubbed out, you know, twenty some slides, researching across probably fifty different places and generating a number of graphics like charts and things like that in about fifteen minutes. I was like, This is incredible.

That's another easy.

What was also incredible is it burnt about ten percent of my fucking weekly usage that is now twice as big

In fifteen minutes.

as it was like two and a half times of what it was before. So It definitely chewed through the tokens. Now, this is the the question I didn't ask you earlier. I don't know what to do with that. On one hand, I look at it and I say that's extra that's extremely expensive and you know, whatever. That's the logical end, right? It's so expensive, you shouldn't use it. maybe use it in emergency situations, whatever. The other side of me, the addict side probably, looks at it and says, Well, hang on. I'm moving eight times as fast. I'm only being built for 6x. So technically speaking, I'm not paying any more for token usage. I'm just moving faster. I'm actually paying yes less for token usage. Because If I produce the work eight times as fast, well I would have just produced the same work but turned through it eight times as slow.

Yeah, I mean it the math would I mean you would just have to compare actual token usage and and utilization of those things, right? But if it if it's a straightforward eight X is fast, eight same eight X number of token usage, you're right. It would be only it would be cheaper to do it that way.

Yeah, at that point I'm just I'm trading time.

Right. You're you're trading time and it becomes a you know, a scalability challenge, right? So how many five hundred dollar a month packages do you need to be able to satisfy whatever work demand you're trying to accomplish?

as of right now it looks like about seven.

Well, all right, let's talk about budgeting.

Yeah. But the question is, what do you get done? Like if you

Uh-huh.

can get everything you need done and more, I don't know, maybe maybe it's worth it.

I mean it it would be for certain application types, right? So in you know, maybe there's maybe there's something that they layer on top of this that's more teams oriented instead of just paying for for token usage. is there a way that you can, you know, create some of these things together from a from a team standpoint? I haven't explored their their pricing models and they change every twelve minutes. So there's that.

Yeah, I mean the Teams models are never gonna have advantageous pricing. They're always like the you don't have the access to even the new Pro Max plans on Teams. You can pay for ultra fast,

They want you to pay per token.

but you're paying per token. And you know, I'm just not doing that. I'll just keep stacking, you know, personal plans until forever, just like

To take them away.

many other people have. Yeah. but you know. Then it it it did it did create like a weird little moment in my head where I was like, this is fucking awesome. I was just able to do a whole bunch of work really, really quickly. And sure I would have been able to do that before at a s but it would have been a slower speed and it would have traded time for it. And you know, if I'm getting it if I'm getting it done faster, is it actually that much more expensive? Or am I just moving through time faster? Like have I discovered Have they just bestowed upon all of us time travel because what

At some point the human becomes the bottleneck though, right? Because the human has to be able to review, approve, do something relative to that. So as long as the

Maybe.

human can move as fast as it needs to move, or you've built other agents to to satisfy that workflow, then theoretically you've commit you've created an infinite loop.

Yeah, it depends on depends on the product that you're producing. But also like, you know, I just talked about one example now and granted I wasn't in a hurry to get that done. and I wasn't sitting here waiting for it, but you know, it took an hour to do the work that that I tasked it with. But if that work could have been done in ten minutes? I would have been I would have had the result and been able to use it in ten minutes instead of an hour.

Well, it goes back to some of the things we've talked about even in our own business, right? So how do we get to a place where we are quote unquote fully leveraged in a in AI? And maybe that moment never comes, but that aspiration is what where we should be driving towards. And I think what you're describing here is like, all right, well, if a normal deliverable, we may quote it at, I don't know, whatever, a week to produce. And instead of taking a week to produce it, it actually takes 15 minutes. Did you did you, to your point, did you invent time travel at that state? Or is now the new norm gonna be that things should take a day instead of a week? And so now every the the pressure is just compressed as opposed to what what things used to look like. I I think the latter is probably the more long-term outcome, short term, I think you get while the getting's good. And and you can, you know, you can get a lot more done in a in a fraction of the time and and have a bit of a buffer on those things.

I think it it becomes the time value of of it's the it's the old old question, right? Do you have more money than time or time than money? so for me I look at this like fast I really didn't think much about because it was just too X speed. You really didn't feel the difference. This is the first time I've tangibly felt the difference. And I'm like, I'm not sure I can go back.

Yeah.

which is a real problem. But then the question is like, is it worth it? Is the is the immediate next question that pops in my head is, you know, is it worth it? I'm like, it's pretty fucking awesome to be able to, you know, have a problem and and send it to an agent and have that agent reply almost instantaneously because it's running at eight X speed. and sometimes

Again, I think it depends depends on the work type and it depends on again, going back to like the bottlenecks, it does it does that create different bottlenecks? The answer's probably yes. What does that look like? If the bottleneck was the generation of the deliverable, whatever that is, and now that's not the bottleneck, what is? And then you you explore how to how to knock down that wall, whether it's with agents or not. But yes, I mean I think I think the short answer, my short answer is it is worth it so long as you can identify what those bottlenecks are.

I think the big one would be whatever it's it has to be something that like you're waiting on, right? So like you know, I have a background job that like classifies emails now because they have a lot of just junk emails that come in. And that doesn't need necessarily to be fast or performant because I don't ever see it. what I do need to be fast and performant is when I'm walking out the door and I'm trying to find information for where the Omarchi meetup is and I can't find it. So I kick it to an agent and say, Hey man, find this. And it comes back almost instantly with all of that information. eventually actually the bottleneck did.

Which is a true story that happened yesterday.

I think eventually the bottlenecks also probably become things like browser use, APIs, web, because like the agent can process and produce the result faster, but it can only research at the speed that it has access to research. so for some of your longer running tasks, you would maybe run into that as a bottleneck eventually too, is It can only research so fast because the internet can only move so fast. or you get rate limited or or whatever it is. But I hope that Anthropic sees this and now there's a speed war that starts where, you know, everybody's been focused on how smart these things are. We we've gotten away from how fast they are. And I would love to see a world where I can get you know, frontier models running at a thousand tokens per spec per second directly from the foundries. that would be incredible.

And I think we're on our way there, right? I mean, infrastructure has been the bottleneck of speed up to this point, which infrastructure is now catching up slowly but surely, right? And it's going at rapid pace, don't get me wrong. Like these these freaking facilities are popping up, you know, within like a month,

Everywhere.

month or two, at least getting online and then being optimized afterwards. It's it's crazy the speed at which they're they're creating. But, you know, you think back to literally this time last year, the supply and demand was way out of whack. So Therefore the models, even though they were less intelligent, were incredibly slow. And so you think to now this year, the idea of something fast earlier this year was still kind of monumental, right? So while it felt like a baby step, it was probably the best they could do with the infrastructure available at the time. Now it's like, well, we're gonna end up in a place it's it's forecasted anyway, with that we're gonna end up in a place where the supply far outpaces the demand for at least some period of time. It'll catch up, but we'll have way more compute than we can use for some period of time, because of all the all the things that are popping up everywhere. Is that true? I don't know.

I don't know, man. I don't think it's gonna happen for probably ten years.

That maybe. I don't know.

This is this this what we're seeing is definitely like it is the the what was it, the Jevons paradox, right? Where As fast as they can build these things, we need more compute. Like there's not enough GP, there's not enough silicon that exists today to satisfy the needs that we have, which is why we see, you know, price hikes as high as high as they are. It's why, you know, the new framework desktop that just came out is seven thousand dollars. And the one that, you know, you previously would have bought that was expensive for the time was like twenty eight hundred. crazy. Like the the memory that's in my laptop I bought for like a hundred and ninety dollars, I think. That shit's like

Think there's a lot of things that go that go into that though. Supply and demand is one of them, but I think you have well

Fine wins all of it. All they all they all figured out that they can sell it's it's much more profitable to sell memory to to commercial AI developers or it's more

Call call me a conspiracy theorist, but I think like diamonds, there's there's a supply and demand thing starting to happen. Or or is b it's in my opinion, it's beginning to happen in the technology space.

You think they're just hoarding silicon and not releasing it?

Yep. I don't know that the actual manufacturers are doing that, but I think there's a there on a on the supply side somewhere there's some greedy suppliers that are that are very likely causing price inflation across different things. Call me a s conspiracy theorist, but that's what I

I will, 'cause I don't think that's what's happening. I think they're all just figuring out that they're just they can make money. If I can sell it, the problem is the consumer gets fucked. I if the money has never been in the consumer side. And so when you had whatever the the manufacturer was of RAM, they were the largest man they were the largest manufacturer of consumer RAM in the world. They didn't go out of business. They just stopped selling to consumers. Because when you can sell for twice or three times the price to commercial Why the fuck wouldn't you? The problem is now everybody is just selling all of their memory and all of their chip allocations to enterprise. So on the consumer end, there's no memory available. there's only one place I think that manufactures consumer memory now. same with GPUs. You have these you have all these places scooping up all of the the Enterprise AI GPUs. So now ours are more expensive. So it sucks. The that's why like everybody's been after the RTX or the not the RTX sparks, the the DGX sparks. because they've been like the G B ten chip that's in them, if you went to buy the exact same system from Asus or Dell or other people, they've taken price hikes. They're like eight grand now. but DGX Spark was still selling for like thirty eight hundred up until recently. I think they've finally had take price hikes, and I've talked to some people about that. I'm like, why is this the case? And the reason is super simple, and that's that NVIDIA bought all of their memory before the memory hike. So they had this massive allocation of memory, and they're able to produce and sell these boxes at a much lower rate than anybody else because they're all buying memory at today's prices.

makes sense. I mean all pricing is based on futures essentially right so whatever's been

Sort of.

produced before unless you just have a greedy group that raises prices of inventory you already have, that price is set based on the futures that were bought at that point in time. And so today's products that are being created are based on the futures of of new things. So in this case chips. so that all that all jives and makes sense. All right let's talk about

Unfortunately, they're all out of stock now.

Right.

Everybody everybody bought up because they got they got wise to that. And I don't think there's a like you used to be able to get at Micro Center. Micro Center, all the microcenters around had like twenty five in stock. Now none of

Crazy.

them have any in stock.

All right, so the last stop of your west of the Mississippi tour was in a marshy meetup, right? In San Fran last night.

Yep.

How how was that? That's your that that the first meetup you've gone to? Because they haven't been going on all that long, right? Maybe a month or so?

a couple months, yeah. it was good. It was it was nice to meet up with some other people and kind of talk about, you know, their experiences and you know kind of see a swath of people from you know folks that have been using the thing since before it even had an ISO and they've been around for a long time to people who'd one guy who literally installed on his system there, like at the the place when he got there. So you had, you know, from sort of veteran users to the extent that veterans can exist,

Mm-hmm.

to brand new users. and you know, so I just talked, gave kind of a a state of where we're at and and some of the growth that we've seen and what we're doing and and just all that from my perspective. which is good, answered a bunch of questions and and got to see some Some lightning talks of some things that other people have produced that were pretty cool. so it's just neat it was a neat interaction to be able to spend some time. There've been a bunch of meetups and every single one of that I've been near, I've missed by like one or two days. So I was really happy that this one happened to be the day after dev day. So I like, Well I'll stick around for another day and at least go to this event because it's the one that I can finally make it to. I'm glad it was a fantastic event. And it was at Cloudflare's office, so I finally got to see the infamous wall of of lava lamps in person. So that was that was neat neat thing.

You have to sh send me pictures of that one. I haven't seen that.

I didn't take one, but there's plenty online.

I'll look it up. Well, we've covered a lot from a Rails World to OpenAI event to new models to hardware issues, conspiracy theories, and then a marching meetup. Till next time.

Till next time.