No, That's Not a Software Factory — Ryan Cooke, WorkOS

Well, thank you for Well, thank you for joining us for the joining us for the joining us for the final day of final day of final day of the conference. I know it's been a the conference. I know it's been a the conference. I know it's been a long long long week. But I'm happy week. But I'm happy week. But I'm happy to talk about a topic that's to talk about a topic that's to talk about a topic that's very close to my heart very close to my heart . Software factories . Software factories . Software factories . Hmm, this . Hmm, this . Hmm, this probably isn't a new probably isn't a new probably isn't a new concept to concept to concept to any of you. I think any of you. I think if you are all here, you are if you are all here, you are if you are all here, you are very familiar with very familiar with software factories. You know, software factories. You know, Ramp blogged about Ramp blogged about Ramp blogged about his inspections his inspections his inspections a few months ago. a few months ago. a few months ago. I think it was I think it was I think it was December of last December of last December of last year. I don't remember year. I don't remember year. I don't remember exactly when. A exactly when. A exactly when. A little time has passed. It little time has passed. It little time has passed. It really got the really got the really got the whole industry excited with thoughts whole industry excited with thoughts whole industry excited with thoughts like, "Oh, we like, "Oh, we like, "Oh, we can actually can actually can actually build software in a completely build software in a completely different way now that different way now that AI is incredibly AI is incredibly AI is incredibly capable of writing code." capable of writing code." capable of writing code." Hmm, many other Hmm, many other Hmm, many other companies followed companies followed companies followed suit. Hmm, and there suit. Hmm, and there suit. Hmm, and there is something like a is something like a is something like a standard standard standard configuration here. Uh, configuration here. Uh, configuration here. Uh, you have a sandbox where you have a sandbox where you have a sandbox where you put your code you put your code . Hmm, you put in . Hmm, you put in . Hmm, you put in there, uh, an AI agent, uh there, uh, an AI agent, uh , maybe a section of , maybe a section of , maybe a section of Claude's code or Claude's code or Claude's code or open source, uh, and open source, uh, and open source, uh, and with a query with a query with a query it generates a PR, and you it generates a PR, and you it generates a PR, and you merge it. Hmm, merge it. Hmm, merge it. Hmm, maybe you're already maybe you're already maybe you're already creating it yourself. Well, in creating it yourself. Well, in creating it yourself. Well, in WorkOS we took a WorkOS we took a WorkOS we took a slightly different approach. Um, slightly different approach. Um, slightly different approach. Um, and the motivation for what and the motivation for what and the motivation for what we're creating we're creating we're creating is that we is that we is that we see a lot of see a lot of see a lot of success metrics, um success metrics, um , that are really , that are really , that are really focused on focused on focused on results, results, results, right? For example, you'll right? For example, you'll right? For example, you'll see posts on X see posts on X see posts on X and blogs, um, that and blogs, um, that and blogs, um, that talk about talk about talk about PR percentage, or the number of PR percentage, or the number of PR percentage, or the number of takedown requests,

takedown requests, takedown requests, or how much code, um, AI or how much code, um, AI or how much code, um, AI generated, AI, um generated, AI, um , goes into , goes into , goes into production. Hmm, and these are production. Hmm, and these are production. Hmm, and these are just just just outcome measures that, um, outcome measures that, um, outcome measures that, um, you know, can you know, can you know, can actually actually actually hide hide hide how well how well how well these systems are working. these systems are working. Er, the percentage of PR or Er, the percentage of PR or the number of PR, hmm, could the number of PR, hmm, could the number of PR, hmm, could be the reason for the be the reason for the be the reason for the overall overall overall increase in PR. Uh, it's increase in PR. Uh, it's increase in PR. Uh, it's very difficult very difficult very difficult to distinguish, um, whether the to distinguish, um, whether the to distinguish, um, whether the outcome actually outcome actually outcome actually leads to the leads to the leads to the results. And so that's what results. And so that's what results. And so that's what we we we focus our focus our focus our efforts on when we think efforts on when we think efforts on when we think about the about the software factory. That's if software factory. That's if we were going to we were going to we were going to spend engineering time spend engineering time creating that creating that level of automation. level of automation. level of automation. Uh, and that Uh, and that Uh, and that takes a little takes a little takes a little work. Uh, work. Uh, full-time teams full-time teams dedicated to this. Um, dedicated to this. Um, dedicated to this. Um, how would we like to how would we like to how would we like to think about the success of think about the success of think about the success of this and does it this and does it this and does it actually create actually create actually create value for our value for our value for our organization or the organization or the organization or the engineering engineering engineering organization? And so organization? And so organization? And so we measure instead of we measure instead of we measure instead of just outputting code, just outputting code, just outputting code, we look at we look at outcome metrics. Um, and this is outcome metrics. Um, and this is really about really about really about whether whether whether we accelerate our ability to we accelerate our ability to we accelerate our ability to deliver features. We deliver features. We deliver features. We believe that the dream of a believe that the dream of a believe that the dream of a software factory is to software factory is to give each of give each of our engineers a our engineers a our engineers a small team of small team of small team of engineers to themselves. And engineers to themselves. And engineers to themselves. And so the consequence of that so the consequence of that so the consequence of that should be that we should be that we should be that we build more and build more and build more and deliver more, and deliver more, and deliver more, and that when we start that when we start that when we start working on working on working on complex features, complex features, complex features, they can be they can be they can be built much built much built much faster. So, what does it faster. So, what does it faster. So, what does it actually look like?

actually look like? Well, we started with the same thing Well, we started with the same thing you saw in other you saw in other you saw in other other factories. We other factories. We other factories. We started with a sandbox started with a sandbox started with a sandbox that we built on that we built on that we built on top of Cloudflare. We top of Cloudflare. We top of Cloudflare. We put an put an open source router there, um. open source router there, um. We pass on We pass on We pass on the clues to him. Um, and we the clues to him. Um, and we the clues to him. Um, and we very quickly very quickly very quickly ran into this ran into this ran into this situation where it wasn't situation where it wasn't situation where it wasn't yielding any more yielding any more yielding any more results. This results. This results. This wasn't a gradual wasn't a gradual wasn't a gradual increase or an increase or an increase or an exponential exponential exponential increase increase increase compared to compared to compared to engineers engineers engineers just running just running just running cloud code on their cloud code on their cloud code on their laptops. In fact, laptops. In fact, laptops. In fact, it was it was it was pretty indistinguishable to us. pretty indistinguishable to us. And so we started And so we started thinking about how thinking about how thinking about how can we embed can we embed can we embed our engineering our engineering our engineering processes into the processes into the processes into the factory itself? It is not enough for a factory itself? It is not enough for a factory itself? It is not enough for a factory factory factory to produce code. We to produce code. We to produce code. We actually wanted to actually wanted to actually wanted to take and take and take and automate a automate a automate a significant portion of the other significant portion of the other significant portion of the other work that work that work that our engineers do to our engineers do to our engineers do to build build build products. So we products. So we products. So we divided it into two divided it into two divided it into two parts. First, parts. First, parts. First, we have a system we we have a system we we have a system we call TARS. It's call TARS. It's call TARS. It's the way the way the way users interact with the users interact with the users interact with the coding agent, and coding agent, and coding agent, and it's built into the it's built into the it's built into the tools we tools we tools we use. So it's use. So it's use. So it's not just Slack, but not just Slack, but not just Slack, but Linear and GitHub as well. We Linear and GitHub as well. We Linear and GitHub as well. We subscribe to subscribe to subscribe to webhooks through TARS so that webhooks through TARS so that webhooks through TARS so that TARS can track the TARS can track the TARS can track the progress of projects progress of projects progress of projects in addition to generating in addition to generating in addition to generating source code. And source code. And source code. And then we created a then we created a then we created a separate system separate system separate system called Horizon. This is the called Horizon. This is the infrastructure orchestration level. It is

infrastructure orchestration level. It is much more much more much more similar to inspections, similar to inspections, similar to inspections, minions, and some minions, and some minions, and some other systems that other systems that other systems that other companies have created other companies have created other companies have created . But it is . But it is . But it is located in front of the located in front of the located in front of the MCP gateway. And I'm MCP gateway. And I'm MCP gateway. And I'm going to go into a little more going to go into a little more going to go into a little more detail detail detail about about about why this MCP gateway has been why this MCP gateway has been why this MCP gateway has been truly transformational for us truly transformational for us truly transformational for us . . So, by passing So, by passing actions that actions that actions that occur in other occur in other occur in other source source source code systems and code systems and code systems and project tracking systems to our webhooks project tracking systems to our webhooks , we start , we start , we start to achieve a to achieve a to achieve a level of autonomy where level of autonomy where level of autonomy where our factory does our factory does our factory does the work of developing the the work of developing the the work of developing the product, not just the product, not just the product, not just the code. So code. So code. So what is an example of this? In what is an example of this? In what is an example of this? In linear development, we linear development, we linear development, we can identify can identify can identify tickets that have tickets that have tickets that have dependencies. dependencies. dependencies. Thus, one ticket Thus, one ticket Thus, one ticket blocks another ticket. It is blocks another ticket. It is blocks another ticket. It is quite common for quite common for quite common for large large large units of work units of work units of work to be broken down into to be broken down into to be broken down into smaller ones. Because TARS smaller ones. Because TARS smaller ones. Because TARS receives webhooks receives webhooks receives webhooks after after after a ticket is completed, it can a ticket is completed, it can a ticket is completed, it can automatically automatically automatically intercept the intercept the intercept the next ticket in the next ticket in the next ticket in the loop. So we loop. So we loop. So we can, through our can, through our can, through our planning process, planning process, planning process, build something build something build something like a map of how like a map of how like a map of how we think this plan we think this plan we think this plan will be executed, will be executed, will be executed, and TARS can start and TARS can start and TARS can start executing it executing it executing it autonomously. Another thing autonomously. Another thing autonomously. Another thing we have is that we have is that we have is that between these steps, between these steps, between these steps, when the ticket when the ticket when the ticket is completed, we can is completed, we can is completed, we can ask TARS, " ask TARS, " ask TARS, " Can you re- Can you re- Can you re- evaluate the linear evaluate the linear evaluate the linear project and let project and let project and let me know if there are any me know if there are any me know if there are any new tickets missing?"

new tickets missing?" new tickets missing?" Because as you Because as you Because as you work, you will find out work, you will find out work, you will find out where there are gaps in where there are gaps in where there are gaps in your plan. And we your plan. And we your plan. And we want want want to keep our to keep our to keep our plan fresh plan fresh plan fresh by using the by using the by using the agent itself. Because the agent agent itself. Because the agent agent itself. Because the agent determines that determines that parts of what we parts of what we originally planned originally planned originally planned for the project and the goal for the project and the goal for the project and the goal we have in mind may be missing. we have in mind may be missing. So, we're bringing So, we're bringing this this this product product product engineering culture to WorkOS. This is where engineering culture to WorkOS. This is where engineering culture to WorkOS. This is where engineers engineers engineers are responsible for are responsible for are responsible for many of the many of the many of the product's functions. Today, product's functions. Today, product's functions. Today, we don't have we don't have we don't have product managers on the teams. product managers on the teams. product managers on the teams. Hmm, and one of the Hmm, and one of the Hmm, and one of the rituals within rituals within rituals within this this this product product product engineering process is engineering process is engineering process is to create a document to create a document to create a document at the top of the hill. at the top of the hill. This is a PRD, and it is This is a PRD, and it is designed to both designed to both designed to both define the purpose of define the purpose of define the purpose of this project and this project and this project and incorporate what incorporate what incorporate what our customers are talking about our customers are talking about our customers are talking about ? For example, ? For example, ? For example, where do we see the need where do we see the need where do we see the need for this unit of work? for this unit of work? for this unit of work? Hmm, he's looking at Hmm, he's looking at Hmm, he's looking at competitive analysis. competitive analysis. competitive analysis. For example, are there similar products on the For example, are there similar products on the For example, are there similar products on the market today market today that we can that we can draw inspiration from? We draw inspiration from? We draw inspiration from? We are starting are starting are starting to implement the early to implement the early to implement the early stages of design.

stages of design. stages of design. Hmm, and we outline, Hmm, and we outline, Hmm, and we outline, for example, what the for example, what the for example, what the main milestones are. And main milestones are. And main milestones are. And so, by so, by so, by consolidating and consolidating and consolidating and canonizing that canonizing that canonizing that information into a information into a information into a document, that's what document, that's what document, that's what our product our product our product engineers are doing, uh, engineers are doing, uh, engineers are doing, uh, across the company. Hmm, across the company. Hmm, across the company. Hmm, now we can now we can now we can ask to provide this ask to provide this ask to provide this resource to an agent, and the resource to an agent, and the resource to an agent, and the agent can break agent can break agent can break it down into units of it down into units of it down into units of work. Uh, and it was a work. Uh, and it was a work. Uh, and it was a really powerful really powerful really powerful way for us to take way for us to take way for us to take existing processes and existing processes and existing processes and code them into the code them into the code them into the factory itself. As I factory itself. As I factory itself. As I mentioned, it mentioned, it mentioned, it listens for webhooks. listens for webhooks. listens for webhooks. So I'll show an example of what So I'll show an example of what So I'll show an example of what it looks like, it looks like, it looks like, but he can see but he can see but he can see that the Hilltop document that the Hilltop document that the Hilltop document through line requests through line requests through line requests has been reviewed and has been reviewed and has been reviewed and approved, and because of approved, and because of approved, and because of its approval, he can its approval, he can its approval, he can automatically automatically automatically start working on start working on start working on that project. So we that project. So we that project. So we don't need a human to don't need a human to don't need a human to manage this manage this manage this agent at every agent at every agent at every step of the step of the step of the lifecycle. So we lifecycle. So we lifecycle. So we created an agent created an agent created an agent specifically for this specifically for this specifically for this part of the process. We part of the process. We part of the process. We call him PM ( call him PM ( project manager).

project manager). project manager). He does the first He does the first He does the first draft of hilltop draft of hilltop draft of hilltop based on the brief based on the brief based on the brief specification we specification we specification we give him. It give him. It give him. It adds context to adds context to adds context to this in addition to this in addition to this in addition to reading human reading human reading human feedback, and then feedback, and then feedback, and then takes the implementation and takes the implementation and takes the implementation and breaks it down into applications breaks it down into applications . And there is an opportunity for . And there is an opportunity for . And there is an opportunity for people to stay people to stay people to stay informed about every part of informed about every part of informed about every part of this process. I this process. I this process. I mean, very mean, very mean, very often a person often a person often a person intervenes or intervenes or intervenes or comments on an application comments on an application comments on an application generated by artificial generated by artificial generated by artificial intelligence, provides it with intelligence, provides it with intelligence, provides it with further instructions, further instructions, further instructions, or clarifies it. Hmm, or clarifies it. Hmm, or clarifies it. Hmm, but it's actually kind of like a but it's actually kind of like a cold start problem where cold start problem where we can overcome the we can overcome the we can overcome the difficulty of creating difficulty of creating difficulty of creating all these different all these different all these different resources. Okay, so what does resources. Okay, so what does resources. Okay, so what does it actually it actually it actually look like? Here's a project look like? Here's a project look like? Here's a project I started I started I started last month, two last month, two last month, two months ago. We are months ago. We are months ago. We are adding a new API to the adding a new API to the adding a new API to the products I products I products I manage called Vaults. manage called Vaults. And with the help of And with the help of the team here in Slack, I the team here in Slack, I the team here in Slack, I can start this can start this can start this project with a simple project with a simple project with a simple description, a few sentences description, a few sentences , of what I'm , of what I'm , of what I'm trying to achieve. trying to achieve.

Tarsyn creates all these Tarsyn creates all these resources for me. resources for me. resources for me. I don't need to I don't need to I don't need to create a project in create a project in create a project in Linear. I don't need to Linear. I don't need to Linear. I don't need to create a draft create a draft create a draft Notion document, all the Notion document, all the Notion document, all the solution logs, and solution logs, and solution logs, and open issues. Tars open issues. Tars open issues. Tars creates the first creates the first creates the first pass. And that gives me a pass. And that gives me a pass. And that gives me a simple structure simple structure simple structure that I can go into as a that I can go into as a that I can go into as a lead lead lead product engineer and start product engineer and start product engineer and start providing more providing more providing more specifications, specifications, specifications, finalizing areas finalizing areas finalizing areas that aren't clearly that aren't clearly that aren't clearly defined, knowing defined, knowing defined, knowing what we're trying to what we're trying to what we're trying to achieve with the achieve with the achieve with the project, and then project, and then project, and then hand it off to Tars to hand it off to Tars to hand it off to Tars to execute. So, here is an execute. So, here is an execute. So, here is an example project. And as example project. And as example project. And as I said, this I said, this I said, this sets the first sets the first sets the first milestones in our milestones in our milestones in our product development process. product development process. So, we've already done So, we've already done the review at the top. the review at the top. the review at the top. When that's done When that's done , we'll, uh, mark , we'll, uh, mark , we'll, uh, mark this request as this request as this request as complete, and Tarz complete, and Tarz complete, and Tarz will take it and will take it and will take it and move on to the move on to the move on to the next phase of the next phase of the next phase of the project. project.

So, we've found that So, we've found that a lot of teams are now a lot of teams are now a lot of teams are now doing more doing more project description and project description and top-up work because our top-up work because our top-up work because our agent can fill in agent can fill in agent can fill in all that information. This is all that information. This is all that information. This is a problem with a blank a problem with a blank a problem with a blank page while page while page while writing. If you writing. If you writing. If you can go into a can go into a can go into a document that already document that already document that already has information, and you has information, and you has information, and you know, these agents are not know, these agents are not know, these agents are not perfect, there are perfect, there are perfect, there are times when that times when that times when that greatly overstates what greatly overstates what greatly overstates what we are trying to we are trying to we are trying to achieve in this achieve in this achieve in this project, and we project, and we project, and we have to cut out a have to cut out a have to cut out a significant portion of the significant portion of the significant portion of the scope that it scope that it scope that it offers. But that's offers. But that's offers. But that's okay. It is much easier for an engineer okay. It is much easier for an engineer to add input data to add input data than to spend time than to spend time than to spend time configuring all configuring all configuring all these primitives these primitives these primitives themselves. And then themselves. And then themselves. And then we can we can we can continue to do continue to do continue to do this work through Slack, this work through Slack, this work through Slack, through Linear, and eventually through Linear, and eventually through Linear, and eventually it leads to PR it leads to PR requests on GitHub. Another requests on GitHub. Another requests on GitHub. Another thing this allows thing this allows thing this allows us to do is that us to do is that us to do is that we don't necessarily we don't necessarily we don't necessarily always need to always need to always need to use use use our our our coding agent to coding agent to coding agent to implement certain implement certain implement certain parts of the work. Um, Devin is parts of the work. Um, Devin is parts of the work. Um, Devin is very popular. very popular. very popular. We have a lot of people We have a lot of people We have a lot of people who really who really who really enjoy enjoy enjoy using Devin. using Devin. using Devin. They can point They can point They can point Devin to these tickets and the Devin to these tickets and the Devin to these tickets and the same documentation same documentation same documentation to give Devin the to give Devin the to give Devin the context needed context needed context needed to implement the to implement the to implement the different pieces of work.

different pieces of work. different pieces of work. The same goes for The same goes for The same goes for Claude's local code. Claude's local code. Claude's local code. If you're just If you're just If you're just using Opus in a using Opus in a using Opus in a local local local environment, it can environment, it can environment, it can pull in all pull in all pull in all this, uh, all this this, uh, all this this, uh, all this documentation through MCP and documentation through MCP and documentation through MCP and use it use it use it as context for its as context for its as context for its work. And of course, work. And of course, work. And of course, we also have people we also have people we also have people collaborating in these collaborating in these collaborating in these project channels. You will project channels. You will project channels. You will notice that our notice that our notice that our security team security team security team comes in and comes in and comes in and reviews new reviews new reviews new projects, gives projects, gives projects, gives their opinion on their opinion on security implications, etc. So, I security implications, etc. So, I hinted that we hinted that we created our created our own MCP gateway early in the development of our factory. We own MCP gateway early in the development of our factory. We own MCP gateway early in the development of our factory. We call it our call it our call it our context mechanism context mechanism . It connects . It connects . It connects to all of our to all of our to all of our internal systems, internal systems, internal systems, but also creates but also creates but also creates system-wide cues and system-wide cues and system-wide cues and context on context on context on how to navigate how to navigate how to navigate these tools and these tools and these tools and when to when to when to use them. use them. So, it's connected So, it's connected to Snowflake, our to Snowflake, our to Snowflake, our main main main data lake. We have data lake. We have data lake. We have semantic tables semantic tables semantic tables that we created in that we created in that we created in Snowflake that describe Snowflake that describe product usage or product usage or customer conversations. And then customer conversations. And then customer conversations. And then we can provide an we can provide an we can provide an agent to our MCP agent to our MCP server in the server in the server in the tool description list, tool description list, tool description list, these are the tables, this is these are the tables, this is these are the tables, this is the content that they the content that they the content that they contain, if you're contain, if you're contain, if you're trying to trying to trying to answer a answer a answer a question about this question about this question about this type of content, type of content, type of content, write a query for write a query for write a query for these tables. And it gives these tables. And it gives these tables. And it gives us a little bit of orientation us a little bit of orientation us a little bit of orientation for the agent to

for the agent to for the agent to understand how understand how understand how to use, how to use, how to use, how we use we use we use our tools, how our tools, how our tools, how we organized Linear in we organized Linear in we organized Linear in Snowflake, and gives some Snowflake, and gives some Snowflake, and gives some guidance on guidance on guidance on where to find where to find where to find information. What was information. What was information. What was really amazing about this was that really amazing about this was that we we we now use now use now use this MCP server for this MCP server for this MCP server for many other many other many other different internal different internal different internal tools. We're tools. We're tools. We're opening it up to opening it up to opening it up to people so they can people so they can people so they can query query query directly from Slack directly from Slack directly from Slack to conduct to conduct to conduct data or customer analysis. data or customer analysis. data or customer analysis. So while we initially So while we initially So while we initially envisioned this as envisioned this as envisioned this as just a way to just a way to just a way to connect our connect our connect our agent or agent or agent or coding agent to all of coding agent to all of coding agent to all of our systems, it's our systems, it's our systems, it's become an incredible become an incredible become an incredible enabler for a lot of enabler for a lot of enabler for a lot of our internal our internal our internal teams, and we're teams, and we're teams, and we're building a lot of building a lot of building a lot of other tools other tools other tools based on this MCP based on this MCP gateway. So, if you're gateway. So, if you're gateway. So, if you're just starting to just starting to just starting to think about a think about a software factory, it's worth software factory, it's worth investing in an investing in an investing in an internal internal MCP gateway server that connects MCP gateway server that connects MCP gateway server that connects all your tools all your tools all your tools and has descriptions that and has descriptions that and has descriptions that can tell the can tell the can tell the agent how agent how agent how to use those to use those to use those tools and how you tools and how you tools and how you specifically specifically specifically organized the organized the organized the information within them. And I information within them. And I information within them. And I think you'll find that think you'll find that think you'll find that it will eventually come in it will eventually come in it will eventually come in handy in many handy in many handy in many other other other use cases. As I use cases. As I use cases. As I mentioned, we mentioned, we mentioned, we are trying to are trying to are trying to automate other automate other automate other parts of our parts of our parts of our software development software development software development . We have . We have bug requests bug requests coming in via coming in via coming in via Slack. So TARS Slack. So TARS Slack. So TARS can can can listen to them through webhooks and be the listen to them through webhooks and be the listen to them through webhooks and be the first to first to first to sort and sort and sort and implement implement implement initial initial bug fix requests. We also have

bug fix requests. We also have many customers in many customers in many customers in shared Slack channels. shared Slack channels. We have found TARS to be We have found TARS to be quite useful for quite useful for quite useful for sorting through the sorting through the sorting through the support requests we support requests we support requests we receive through them. receive through them. Because he can Because he can review the code, he review the code, he review the code, he can often understand can often understand can often understand where a customer might where a customer might where a customer might encounter encounter encounter problems with our problems with our problems with our products from a products from a products from a code perspective. I showed code perspective. I showed code perspective. I showed you an example of how you an example of how you an example of how we organize the we organize the we organize the core functions core functions core functions around this. And around this. And around this. And the last thing that I the last thing that I the last thing that I think is really think is really think is really exciting is exciting is exciting is the attempt to get to the attempt to get to the attempt to get to self-improving self-improving software or self- self- managing software. So we managing software. So we managing software. So we use tar use tar use tar to actually to actually to actually build our own build our own sandbox infrastructure. So we sandbox infrastructure. So we want to move away want to move away want to move away from some sandboxes from some sandboxes from some sandboxes as a service and as a service and as a service and own own own this layer of this layer of this layer of infrastructure ourselves. Hmm, infrastructure ourselves. Hmm, infrastructure ourselves. Hmm, one of the reasons for this is one of the reasons for this is one of the reasons for this is that we that we that we want to have really want to have really want to have really deep control deep control deep control over over over session information and be session information and be session information and be able to able to able to move move move workloads between workloads between workloads between different parts of different parts of different parts of our infrastructure our infrastructure . So next we build . So next we build . So next we build our memory layer, and our memory layer, and our memory layer, and we want this to be an we want this to be an we want this to be an evergreen evergreen evergreen context of what context of what context of what each each each person in the company is working on, person in the company is working on, person in the company is working on, what team are they what team are they what team are they working on? What working on? What working on? What products is she products is she products is she responsible for? And then, responsible for? And then, responsible for? And then, at the organizational level, at the organizational level, at the organizational level, what are the semantics of the what are the semantics of the what are the semantics of the working OS and how working OS and how working OS and how we do our we do our we do our work? And we want to be work? And we want to be work? And we want to be able to able to able to connect both to connect both to connect both to our our software factory and software factory and extract that context extract that context extract that context from our factory and to from our factory and to from our factory and to our other our other AI tools.

So, I think that's So, I think that's our main our main our main takeaway from takeaway from takeaway from seeing how many seeing how many seeing how many people are putting people are putting people are putting effort into effort into effort into results, and where we results, and where we results, and where we think we can think we can think we can get this get this get this exponential exponential exponential human value from human value from human value from our factory. Can our factory. Can our factory. Can we actually we actually we actually provide value to provide value to provide value to our customers? We our customers? We our customers? We think a lot about think a lot about think a lot about delivery and delivery and delivery and customer impact. And so we customer impact. And so we customer impact. And so we want to code this want to code this want to code this in our own software factory in our own software factory . . Of course, we're Of course, we're Of course, we're concerned about, you know, concerned about, you know, concerned about, you know, introducing introducing introducing instability into instability into instability into our products. So our products. So our products. So can we measure, can we measure, can we measure, uh, defect rates uh, defect rates uh, defect rates and other, uh, you know, and other, uh, you know, recovery time metrics, uh. Uh, recovery time metrics, uh. Uh, and then a lot of and then a lot of and then a lot of anecdotal facts. anecdotal facts. anecdotal facts. For example, we want to For example, we want to For example, we want to see our see our see our engineers engineers engineers using TARS as a using TARS as a using TARS as a sign that it gives sign that it gives sign that it gives them value and them value and them value and speeds up their speeds up their speeds up their work. We want to work. We want to work. We want to see them see them see them move away from choosing to move away from choosing to move away from choosing to leave their leave their leave their on-premises system and on-premises system and on-premises system and move into a cloud move into a cloud move into a cloud sandbox. And then the sandbox. And then the sandbox. And then the real goal real goal real goal is that we is that we is that we can can can use this use this use this information because information because information because we own the we own the we own the infrastructure. We infrastructure. We infrastructure. We can see what is can see what is can see what is happening in happening in happening in the infrastructure. We the infrastructure. We the infrastructure. We use this use this use this information for information for information for self-improvement of the self-improvement of the self-improvement of the factory. We want factory. We want factory. We want her to study.

her to study. her to study. We want her to We want her to We want her to improve the way she improve the way she improve the way she writes code. We want writes code. We want writes code. We want to see where, um, our to see where, um, our to see where, um, our engineers can engineers can engineers can improve their improve their improve their skills. skills. skills. So much is So much is changing rapidly in artificial intelligence. It's kind of a changing rapidly in artificial intelligence. It's kind of a constant constant constant race to stay up to race to stay up to race to stay up to date with the latest, um, date with the latest, um, date with the latest, um, you know, tips and you know, tips and you know, tips and techniques. We can techniques. We can techniques. We can actually guide actually guide actually guide agents through sessions and agents through sessions and agents through sessions and see where the see where the see where the gaps are and how people are gaps are and how people are gaps are and how people are using, um, using, um, using, um, our factory. Where is our factory. Where is our factory. Where is the skill that we the skill that we the skill that we need to develop, hmm need to develop, hmm , because the agent made , because the agent made , because the agent made a mistake? Um, what a mistake? Um, what a mistake? Um, what skills are no longer skills are no longer skills are no longer relevant? Maybe relevant? Maybe relevant? Maybe the code has changed to the the code has changed to the the code has changed to the point that a skill point that a skill point that a skill we wrote 6 months we wrote 6 months we wrote 6 months ago is now outdated ago is now outdated . So this continues . So this continues . So this continues the verification. Semi- the verification. Semi- online, um, this is where online, um, this is where online, um, this is where we see a big we see a big we see a big advantage in advantage in advantage in actually owning the actually owning the actually owning the entire entire entire infrastructure infrastructure infrastructure ourselves. So, ourselves. So, ourselves. So, repeat it a third time, it's repeat it a third time, it's repeat it a third time, it's worth repeating. worth repeating. Sandboxes are great. Sandboxes are great. It's cool to launch It's cool to launch It's cool to launch an agent and open it up an agent and open it up an agent and open it up to to to the public. We the public. We the public. We really think about really think about really think about our software development practices and our software development practices and our software development practices and processes processes , and we , and we want to code them want to code them want to code them into automation. I'm into automation. I'm into automation. I'm Ryan. I'm one of the Ryan. I'm one of the Ryan. I'm one of the engineers here at Work-OS.

engineers here at Work-OS. engineers here at Work-OS. We have a booth right We have a booth right We have a booth right across from the across from the across from the performance area. If you performance area. If you performance area. If you build build software factories, I would be software factories, I would be happy happy happy to chat with you. to chat with you. to chat with you. I would really like to hear I would really like to hear I would really like to hear how you resolve the how you resolve the how you resolve the authorization issue. authorization issue. authorization issue. I didn't talk about it I didn't talk about it I didn't talk about it because we haven't because we haven't because we haven't figured it out ourselves yet, but figured it out ourselves yet, but figured it out ourselves yet, but maybe you have some maybe you have some maybe you have some understanding. So understanding. So understanding. So please come by please come by please come by our booth, our booth, our booth, talk to me, I'd be talk to me, I'd be talk to me, I'd be happy to happy to happy to chat. Thank you.