Generation Is Cheap, Review Is Expensive: How to Stop Shipping AI Slop — Gabriel Martinez, G2i

Hello everyone. My Hello everyone. My name is Gabriel name is Gabriel name is Gabriel Martinez. You can Martinez. You can Martinez. You can call me Gabo. I'm the call me Gabo. I'm the call me Gabo. I'm the engineering manager engineering manager engineering manager and lead supervisor and lead supervisor and lead supervisor of "hacking" at G2i, and I'm of "hacking" at G2i, and I'm of "hacking" at G2i, and I'm here to talk here to talk here to talk about hacking. I want to about hacking. I want to about hacking. I want to talk about talk about talk about hackwork because I'm quite passionate hackwork because I'm quite passionate hackwork because I'm quite passionate about about about this topic. And by this topic. And by this topic. And by "passionate" I "passionate" I "passionate" I mean "angry." mean "angry." mean "angry." The term "hackneyed" has already The term "hackneyed" has already The term "hackneyed" has already been worn out to the point of being obsolete, but been worn out to the point of being obsolete, but been worn out to the point of being obsolete, but what it refers to what it refers to what it refers to has existed for a long time. It's just that has existed for a long time. It's just that has existed for a long time. It's just that now it's now it's now it's happening at happening at happening at the speed of light. the speed of light. Therefore. A sloppy job is what Therefore. A sloppy job is what happens when happens when judgments are made. Hacking is judgments are made. Hacking is not AI-generated code not AI-generated code not AI-generated code . I'm not trying . I'm not trying . I'm not trying to define it as something to define it as something to define it as something specific and specific and specific and objective that's easy objective that's easy objective that's easy to detect, although there to detect, although there to detect, although there are certain signs. A hack is a are certain signs. A hack is a are certain signs. A hack is a result that result that result that appears complete appears complete appears complete before before before the thinking is complete. This is a work the thinking is complete. This is a work the thinking is complete. This is a work product where product where product where uncertainty uncertainty uncertainty was resolved was resolved was resolved by guesswork, by guesswork, by guesswork, compromises were left compromises were left compromises were left to chance, and to chance, and to chance, and responsibility was not responsibility was not responsibility was not taken. taken. The result was obtained, The result was obtained, of course, but the understanding was of course, but the understanding was not. And the real not. And the real not. And the real danger is not just that danger is not just that danger is not just that the machine the machine the machine made a bad made a bad made a bad choice. The danger is that choice. The danger is that choice. The danger is that the engineer the engineer the engineer didn't even notice that there didn't even notice that there didn't even notice that there was a choice. They was a choice. They was a choice. They never faced never faced never faced ambiguity.

ambiguity. ambiguity. They never They never They never felt limited. felt limited. felt limited. They never They never They never found edge found edge found edge cases that cases that cases that would change the design. The work would change the design. The work would change the design. The work came already " came already " smoothed out", and with smoothed out", and with smoothed out", and with it the unknown it the unknown it the unknown unknowns remained unknowns remained unknowns remained unknown. But unknown. But software is a software is a record of record of record of the choices made. Technical the choices made. Technical the choices made. Technical choices, product choices, choices, product choices, choices, product choices, operational choices, operational choices, operational choices, user choices. user choices. user choices. If we don't see these If we don't see these If we don't see these elections, we can't elections, we can't elections, we can't manage them. And if manage them. And if manage them. And if we don't manage them, we're we don't manage them, we're we don't manage them, we're not really not really not really doing doing doing engineering. We simply engineering. We simply engineering. We simply accept the result. accept the result. accept the result. We are just "code We are just "code We are just "code monkeys" pressing the monkeys" pressing the monkeys" pressing the merge button. I'm not a merge button. I'm not a merge button. I'm not a purist who came purist who came purist who came here to tell you here to tell you here to tell you that AI is bad. He's that AI is bad. He's that AI is bad. He's not bad, okay not bad, okay ? I'm here to describe ? I'm here to describe ? I'm here to describe some of the patterns that some of the patterns that some of the patterns that have led to our " have led to our " drunken" stupor drunken" stupor drunken" stupor on AI. It is quite on AI. It is quite on AI. It is quite understandable, given the understandable, given the understandable, given the successes achieved successes achieved successes achieved since December 25, how we since December 25, how we since December 25, how we ended up where we are ended up where we are . But we need to . But we need to . But we need to get this under control get this under control . We need to . We need to . We need to slow down. We slow down. We slow down. We need to need to need to sober up. Agents sober up. Agents sober up. Agents can seem can seem can seem magical, and the internet magical, and the internet magical, and the internet makes you makes you makes you believe it. They can believe it. They can believe it. They can write code, write code, write code, create create create screen wireframes, and, if screen wireframes, and, if configured correctly, build entire build entire build entire functions or systems.

functions or systems. functions or systems. But they cannot But they cannot But they cannot replace human replace human replace human work: deciding work: deciding work: deciding what is important, understanding what is important, understanding what is important, understanding what is being built, what is being built, what is being built, interpreting interpreting interpreting ambiguity, ambiguity, ambiguity, being responsible for the being responsible for the being responsible for the system in system in system in the long the long the long term, term, term, exercising judgment exercising judgment exercising judgment and restraint at the and restraint at the and restraint at the architectural level, and architectural level, and architectural level, and perhaps most importantly, perhaps most importantly, knowing what not to knowing what not to knowing what not to build. Hacking is build. Hacking is build. Hacking is not about AI as a not about AI as a not about AI as a tool, it's about tool, it's about tool, it's about lack of judgment lack of judgment . So, let's . So, let's . So, let's talk a little bit about what talk a little bit about what talk a little bit about what brought us here. In a brought us here. In a brought us here. In a world where anything is possible, world where anything is possible, world where anything is possible, deciding what to deciding what to deciding what to create becomes more create becomes more create becomes more important than important than ever. People got too ever. People got too ever. People got too carried away and carried away and carried away and built real built real built real monsters out of monsters out of monsters out of meaningless spaghetti meaningless spaghetti code. It's almost as if they code. It's almost as if they code. It's almost as if they forgot that every forgot that every forgot that every line of code is a line of code is a line of code is a commitment, and the commitment, and the commitment, and the best code is the best code is the best code is the absence of it. absence of it. absence of it. The cost of writing The cost of writing The cost of writing code is falling, but code is falling, but code is falling, but the cost of understanding the cost of understanding software is not. And I software is not. And I want to clarify this want to clarify this want to clarify this point. I'm not saying that point. I'm not saying that point. I'm not saying that every line of code needs to every line of code needs to every line of code needs to be personally be personally be personally read, read, read, approved, and approved, and approved, and understood individually. This has understood individually. This has understood individually. This has never been the never been the never been the standard. Large standard. Large standard. Large systems have always systems have always systems have always contained more contained more contained more details than one details than one details than one person can keep person can keep person can keep in their head. But the logic in their head. But the logic in their head. But the logic needs to be understood.

needs to be understood. needs to be understood. The main flows, boundaries, The main flows, boundaries, The main flows, boundaries, data model, and data model, and data model, and product behavior product behavior product behavior need to be understood. need to be understood. need to be understood. Good Good Good software has a software has a software has a certain harmony. Not in certain harmony. Not in certain harmony. Not in some sophisticated some sophisticated some sophisticated aesthetic sense, but aesthetic sense, but aesthetic sense, but in a practical one. The code in a practical one. The code in a practical one. The code should reflect should reflect should reflect the structure of the subject the structure of the subject the structure of the subject area. AI can area. AI can area. AI can generate code, but it generate code, but it generate code, but it cannot magically cannot magically cannot magically give give give integrity to a chaotic system. integrity to a chaotic system. integrity to a chaotic system. An agent can work An agent can work An agent can work much more effectively much more effectively much more effectively in a codebase that has in a codebase that has in a codebase that has structure, patterns, structure, patterns, structure, patterns, clear boundaries, and logical clear boundaries, and logical clear boundaries, and logical reasoning. But reasoning. But reasoning. But if the system is already a " if the system is already a " big pile of mud," big pile of mud," big pile of mud," the agent won't get you out of it the agent won't get you out of it the agent won't get you out of it . He'll just . He'll just . He'll just make this pile make this pile make this pile bigger and faster. Just bigger and faster. Just bigger and faster. Just because you can create because you can create because you can create another component doesn't another component doesn't another component doesn't mean the product mean the product mean the product needs it. needs it. needs it. Dijkstra had a great Dijkstra had a great Dijkstra had a great idea about this. idea about this. idea about this. If we counted If we counted If we counted lines of code, we would lines of code, we would lines of code, we would not perceive them as not perceive them as not perceive them as lines created, but as lines created, but as lines created, but as lines spent. This is lines spent. This is lines spent. This is the price of development. AI the price of development. AI the price of development. AI makes it easier to spend makes it easier to spend makes it easier to spend these lines. It these lines. It these lines. It alleviates alleviates alleviates file overhead, abstractions, and file overhead, abstractions, and file overhead, abstractions, and complexity. But complexity. But complexity. But in the end, the bill is in the end, the bill is in the end, the bill is yours to pay. This yours to pay. This yours to pay. This calculation manifests itself calculation manifests itself calculation manifests itself in the form of confusion, in the form of confusion, in the form of confusion, fragile processes, and fragile processes, and fragile processes, and functions that no one functions that no one functions that no one wants to touch because they don't fully wants to touch because they don't fully wants to touch because they don't fully understand. This understand. This understand. This results in results in results in interfaces that are interfaces that are interfaces that are technically capable of everything, technically capable of everything, technically capable of everything, but resemble a but resemble a but resemble a maze. Think of maze. Think of maze. Think of that special kind of that special kind of that special kind of bloated B2B software where bloated B2B software where bloated B2B software where every task every task every task requires training. Where there requires training. Where there requires training. Where there are 12 outputs on each screen are 12 outputs on each screen are 12 outputs on each screen , and no one , and no one , and no one can explain whether the can explain whether the can explain whether the complexity arose complexity arose complexity arose because of because of because of user needs or because the user needs or because the teams just teams just teams just kept kept kept adding more.

adding more. adding more. Teams must be held Teams must be held Teams must be held accountable for the accountable for the accountable for the work they work they work they do. do. do. Responsibility is the Responsibility is the key word here. key word here. key word here. Take Take Take responsibility for the responsibility for the responsibility for the product and have a way product and have a way product and have a way to associate it with to associate it with to associate it with positive or positive or positive or negative negative negative reinforcement. Just the reinforcement. Just the reinforcement. Just the realization that realization that realization that someone else someone else someone else will be checking your will be checking your will be checking your work will force work will force work will force engineers to pay engineers to pay engineers to pay a little more attention to a little more attention to a little more attention to detail and be more detail and be more detail and be more thorough. Especially thorough. Especially thorough. Especially if there are if there are if there are accountability metrics that accountability metrics that accountability metrics that can be made can be made can be made visible for each visible for each visible for each piece of work piece of work piece of work product. We are entering an product. We are entering an product. We are entering an era of bloated era of bloated era of bloated products with confusing products with confusing products with confusing interfaces interfaces interfaces that are no better than the that are no better than the that are no better than the worst B2B ERP systems worst B2B ERP systems you were once you were once you were once forced forced forced to use. to use. Today, anyone can Today, anyone can write an application code write an application code write an application code in an hour and get a in an hour and get a in an hour and get a working, clickable working, clickable working, clickable prototype. And that prototype. And that prototype. And that changed expectations. changed expectations. changed expectations. Because when people see Because when people see Because when people see software software coming out so coming out so quickly, they quickly, they quickly, they start to think that start to think that start to think that it it it can be created just as quickly. can be created just as quickly. can be created just as quickly. They see They see They see multiple requests multiple requests multiple requests turning into turning into turning into screens. They see a screens. They see a screens. They see a terminal session terminal session terminal session outputting something that outputting something that outputting something that looks real. And looks real. And , to be fair , to be fair , to be fair , sometimes it's , sometimes it's , sometimes it's real enough real enough real enough to be useful. to be useful. to be useful. Sometimes a prototype like this is a great Sometimes a prototype like this is a great way way way to explore an idea. But to explore an idea. But to explore an idea. But the problem is that the problem is that the problem is that stakeholders don't stakeholders don't stakeholders don't live in the codebase live in the codebase . Therefore, stakeholders . Therefore, stakeholders . Therefore, stakeholders who do not live in the who do not live in the who do not live in the codebase often codebase often codebase often perceive this perceive this perceive this prototype as proof that prototype as proof that prototype as proof that the hardest part is the hardest part is the hardest part is behind them. They don't behind them. They don't behind them. They don't think about the think about the think about the aspects where real aspects where real aspects where real experienced engineers experienced engineers experienced engineers earn their living.

earn their living. earn their living. This is everything that is outside the " This is everything that is outside the " happy path". happy path". happy path". Understanding how to Understanding how to Understanding how to integrate a integrate a integrate a new set of features seamlessly. new set of features seamlessly. new set of features seamlessly. Work related to Work related to Work related to clarifying requirements. clarifying requirements. Working on manual Working on manual testing of testing of testing of functionality to functionality to functionality to make sure it make sure it make sure it works as a works as a works as a person expects. And that person expects. And that person expects. And that the interface isn't complete the interface isn't complete the interface isn't complete garbage. They garbage. They garbage. They don't think about the don't think about the don't think about the most most most time-consuming moments at all time-consuming moments at all . About human factors. . About human factors. . About human factors. About the "last mile" About the "last mile" About the "last mile" where judgment is needed. where judgment is needed. This is where someone This is where someone has to read the code, has to read the code, has to read the code, test the behavior test the behavior , take , take , take responsibility for the responsibility for the responsibility for the tradeoffs, notice tradeoffs, notice tradeoffs, notice missing requirements, and missing requirements, and missing requirements, and decide whether decide whether decide whether this thing deserves this thing deserves this thing deserves to be part of the to be part of the to be part of the system. Here is a tweet system. Here is a tweet system. Here is a tweet that highlights the that highlights the that highlights the point I am point I am point I am trying to make. trying to make. trying to make. He quotes the creator of SQLite He quotes the creator of SQLite He quotes the creator of SQLite as saying on the subject of as saying on the subject of as saying on the subject of merge queries, "You merge queries, "You merge queries, "You say, 'Oh, it's say, 'Oh, it's say, 'Oh, it's free.'" No, it's free.'" No, it's free.'" No, it's not free. You're not free. You're not free. You're asking me to asking me to asking me to maintain this for maintain this for maintain this for you, document it you, document it you, document it for you, test it for you, test it for you, test it for you, support it for you, support it for you, support it for you for the for you for the for you for the next 25 years. It's next 25 years. It's next 25 years. It's not free. And it not free. And it not free. And it really touched really touched really touched me. I couldn't have me. I couldn't have me. I couldn't have said it better. said it better. said it better. It's easy to accept the request, It's easy to accept the request, It's easy to accept the request, but who should be but who should be but who should be responsible for this thing responsible for this thing ? Who is responsible ? Who is responsible ? Who is responsible for the consequences of what for the consequences of what for the consequences of what this means? So, this means? So, this means? So, external pressure from external pressure from external pressure from stakeholders stakeholders stakeholders combined with the pressure of combined with the pressure of combined with the pressure of competition from " competition from " masters of imitation" masters of imitation" masters of imitation" produces a lot of produces a lot of produces a lot of sloppiness. I'm sure sloppiness. I'm sure sloppiness. I'm sure most of you most of you most of you have seen this in real have seen this in real have seen this in real projects. Where bad projects. Where bad projects. Where bad engineers—those who don't engineers—those who don't engineers—those who don't check details, don't check details, don't check details, don't refine specifications, refine specifications, refine specifications, and aren't and aren't and aren't quality-oriented—end up quality-oriented—end up quality-oriented—end up producing more and producing more and producing more and faster. People who faster. People who faster. People who pay attention to pay attention to pay attention to detail and detail and detail and give their

give their give their all may all may all may seem to seem to seem to do less and do less and do less and move more slowly move more slowly . This leads to . This leads to productivity anxiety when, productivity anxiety when, in a team in a team in a team environment, environment, environment, engineers are pressured to engineers are pressured to engineers are pressured to merge PRs merge PRs merge PRs faster than they can faster than they can faster than they can evaluate them, leading to evaluate them, leading to evaluate them, leading to code being merged without code being merged without code being merged without full understanding. full understanding. full understanding. Short-term Short-term Short-term thinking aimed thinking aimed thinking aimed at a quick PR merge at a quick PR merge at a quick PR merge trumps trumps trumps long-term long-term long-term code health. I code health. I code health. I personally felt this personally felt this personally felt this anxiety as anxiety as anxiety as the engineers around the engineers around the engineers around me were merging changes at an me were merging changes at an me were merging changes at an incredible pace incredible pace , and I was constantly , and I was constantly , and I was constantly worried that I wasn't worried that I wasn't worried that I wasn't moving moving moving fast enough. And I can fast enough. And I can fast enough. And I can admit that it admit that it admit that it forced me to forced me to forced me to merge code that I didn't merge code that I didn't merge code that I didn't necessarily necessarily necessarily review with the same review with the same review with the same care that I care that I care that I usually do or should usually do or should usually do or should . I've seen . I've seen . I've seen horror stories of horror stories of horror stories of people merging people merging people merging code they don't code they don't code they don't fully understand fully understand fully understand or haven't even or haven't even or haven't even tested themselves tested themselves . After all, we are all human. . After all, we are all human. . After all, we are all human. We are driven by incentives. We are driven by incentives. We are driven by incentives. And if incentives And if incentives And if incentives make people make people make people close PRs faster, close PRs faster, close PRs faster, then we are in for a then we are in for a then we are in for a rude awakening. rude awakening. rude awakening. Rewards should be Rewards should be Rewards should be based on based on based on value creation, value creation, value creation, elegant, well- elegant, well- elegant, well- structured, structured, structured, pragmatic pragmatic pragmatic system design, and system design, and system design, and clarity of clarity of testing and assessment tools. It just annoys me terribly annoys me terribly when I see PRs that are when I see PRs that are when I see PRs that are several thousand several thousand several thousand lines long. A person lines long. A person lines long. A person cannot fully cannot fully cannot fully understand all the understand all the understand all the important aspects of important aspects of important aspects of such a great such a great such a great change. Our job as change. Our job as change. Our job as engineers is engineers is to adapt to to adapt to to adapt to this new world, this new world, this new world, breaking down tasks breaking down tasks breaking down tasks into simpler into simpler into simpler parts that facilitate parts that facilitate parts that facilitate understanding, simplify understanding, simplify understanding, simplify code surfaces, and gain code surfaces, and gain code surfaces, and gain control over our control over our control over our code bases.

code bases. It was predicted that AI It was predicted that AI would destroy jobs, would destroy jobs, would destroy jobs, and engineers would have to and engineers would have to and engineers would have to work at work at work at the speed of the speed of the speed of token generation. Therefore, there is a token generation. Therefore, there is a token generation. Therefore, there is a lot of pressure to lot of pressure to lot of pressure to produce more and produce more and produce more and faster. In reality, AI faster. In reality, AI faster. In reality, AI only produces more only produces more only produces more and faster, generating and faster, generating and faster, generating a bunch of symbols that a bunch of symbols that a bunch of symbols that take up more of a take up more of a take up more of a person's time. I've seen it person's time. I've seen it person's time. I've seen it happen in the most happen in the most happen in the most mundane mundane mundane situations. You give situations. You give situations. You give someone 10 points of someone 10 points of someone 10 points of scope of work, a few scope of work, a few scope of work, a few custom custom custom paths, something intentionally paths, something intentionally paths, something intentionally simple, and they simple, and they simple, and they come back with a 20- come back with a 20- page page page document. document. document. It may look It may look It may look polished with a polished with a polished with a fancy style, fancy style, fancy style, but now I have to but now I have to but now I have to spend hours spend hours spend hours figuring out if there's figuring out if there's figuring out if there's any real any real any real thought in it. This is thought in it. This is thought in it. This is AI-generated work, in the AI-generated work, in the AI-generated work, in the worst sense. worst sense. worst sense. She doesn't clarify She doesn't clarify She doesn't clarify the problem, she the problem, she the problem, she expands it. She took expands it. She took expands it. She took the uncertainty that the uncertainty that the workman had to resolve and the workman had to resolve and returned it as a returned it as a returned it as a larger artifact for larger artifact for larger artifact for other other other stakeholders. In code, a stakeholders. In code, a stakeholders. In code, a vague vague vague task task task turns into a turns into a turns into a huge PR. A huge PR. A huge PR. A small function small function small function becomes a stack of files.

becomes a stack of files. becomes a stack of files. The result looks The result looks The result looks like a completed job, like a completed job, like a completed job, but in reality the but in reality the but in reality the main burden main burden main burden has been shifted to the person has been shifted to the person has been shifted to the person who is checking. And this is who is checking. And this is who is checking. And this is exactly the point exactly the point exactly the point we need we need we need to be frank about. to be frank about. to be frank about. Generation is cheap as Generation is cheap as Generation is cheap as tea, but evaluation is tea, but evaluation is expensive. If AI expensive. If AI expensive. If AI creates more than creates more than creates more than humans can appreciate, humans can appreciate, humans can appreciate, it does not speed up the it does not speed up the it does not speed up the team's work. It team's work. It team's work. It creates a "bottleneck creates a "bottleneck creates a "bottleneck ". One thing that ". One thing that ". One thing that particularly annoys me is particularly annoys me is Twitter. It's full of Twitter. It's full of Twitter. It's full of self-proclaimed self-proclaimed self-proclaimed experts and experts and experts and clickbait who clickbait who clickbait who retell speeches retell speeches retell speeches or pass off or pass off or pass off well-known things as truth, as if it were well-known things as truth, as if it were well-known things as truth, as if it were some kind of brilliant some kind of brilliant some kind of brilliant novelty. "Ralph's Cycle" novelty. "Ralph's Cycle" novelty. "Ralph's Cycle" was one such was one such was one such case. They case. They case. They tried to present it as a tried to present it as a tried to present it as a breakthrough in agent breakthrough in agent breakthrough in agent systems, although it is systems, although it is systems, although it is just a while loop. just a while loop. just a while loop. For some reason, we completely forgot For some reason, we completely forgot that lines of code are a that lines of code are a that lines of code are a terrible terrible terrible performance metric, but performance metric, but performance metric, but decided that now decided that now decided that now the engineer who the engineer who the engineer who spends the most spends the most spends the most tokens is supposedly tokens is supposedly tokens is supposedly working working working the most efficiently. the most efficiently. the most efficiently. Previously, we broke down Previously, we broke down Previously, we broke down functionality into functionality into functionality into small, atomic small, atomic small, atomic parts that were easy to parts that were easy to parts that were easy to understand. understand. understand. Small enough to Small enough to Small enough to keep in mind and keep in mind and keep in mind and actually check.

actually check. actually check. Now we write two Now we write two Now we write two sentences and expect sentences and expect sentences and expect to get a whole to get a whole to get a whole ill-conceived set of ill-conceived set of ill-conceived set of features at once, features at once, features at once, without even understanding without even understanding without even understanding what we're asking for. what we're asking for. The key point I The key point I want to emphasize is that we want to emphasize is that we want to emphasize is that we must not forget the must not forget the must not forget the fundamental fundamental fundamental truths we have always truths we have always truths we have always known. 60 years of known. 60 years of known. 60 years of development experience have not development experience have not development experience have not been devalued by been devalued by been devalued by artificial intelligence. artificial intelligence. artificial intelligence. Modular code, Modular code, Modular code, proper patterns, and proper patterns, and proper patterns, and sound practices sound practices sound practices remain remain remain essential. A essential. A essential. A friend recently friend recently friend recently asked why I asked why I asked why I spend so much spend so much spend so much time on time on time on agent constraints, agent constraints, agent constraints, coding standards, coding standards, coding standards, architectural styles, architectural styles, architectural styles, and system structure. and system structure. His question was His question was simple: what's the point? simple: what's the point? simple: what's the point? If the code is written by an agent, If the code is written by an agent, If the code is written by an agent, why should we why should we why should we care what it care what it care what it looks like? To some looks like? To some looks like? To some extent I agree. extent I agree. Eventually, the time will come Eventually, the time will come when many when many when many implementation details will be implementation details will be implementation details will be completely hidden completely hidden completely hidden from people. This has always from people. This has always from people. This has always been the direction of been the direction of been the direction of software development software development software development . We . We . We went from went from went from machine code to machine code to machine code to assembly language, then to C, assembly language, then to C, assembly language, then to C, Java, and then to languages Java, and then to languages Java, and then to languages like Ruby, where the like Ruby, where the like Ruby, where the main goal was main goal was main goal was programmer comfort. Just programmer comfort. Just like in like in programming languages, where we programming languages, where we programming languages, where we have created abstractions have created abstractions have created abstractions for garbage collection and for garbage collection and pointer management, AI pointer management, AI abstracts away the abstracts away the abstracts away the imperative details.

imperative details. imperative details. This does not eliminate This does not eliminate This does not eliminate the need to understand how the need to understand how the need to understand how the system comes the system comes the system comes together. That's why together. That's why together. That's why diagrams are so important—not diagrams are so important—not diagrams are so important—not as decorative as decorative as decorative documentation, but as a documentation, but as a documentation, but as a way to make way to make way to make systems understandable. systems understandable. When agents When agents generate thousands of generate thousands of generate thousands of lines of code, people lines of code, people lines of code, people need a concise need a concise need a concise form for validation. form for validation. form for validation. Service boundaries, Service boundaries, Service boundaries, data flows, system states, data flows, system states, data flows, system states, and failure modes. A and failure modes. A and failure modes. A high-quality diagram high-quality diagram high-quality diagram allows you to see the allows you to see the allows you to see the system architecture, system architecture, system architecture, which is often hidden by the code which is often hidden by the code which is often hidden by the code . We can . We can . We can leave the core leave the core leave the core process or process or process or workflow as the object of workflow as the object of workflow as the object of verification, moving the verification, moving the verification, moving the implementation details to the implementation details to the implementation details to the periphery. That's why I periphery. That's why I periphery. That's why I advocate for advocate for advocate for declarative code and declarative code and declarative code and architectural patterns architectural patterns that create levels that create levels that create levels that people can that people can that people can actually understand. actually understand. actually understand. The goal is not just The goal is not just The goal is not just code generation. The goal code generation. The goal is human knowledge, is human knowledge, is human knowledge, human understanding, and human understanding, and human understanding, and human control. human control. human control. We need to We need to We need to optimize our work optimize our work optimize our work for this for this for this human level. human level. human level. Cognitive Cognitive Cognitive load is a load is a load is a tax that every tax that every tax that every engineer must pay engineer must pay engineer must pay before making before making before making significant changes. This is time significant changes. This is time significant changes. This is time spent spent spent understanding where the understanding where the understanding where the code should be, which code should be, which code should be, which pattern to choose, and what a pattern to choose, and what a correct implementation actually means . AI does not abolish this . AI does not abolish this . AI does not abolish this tax. If it is not tax. If it is not tax. If it is not curbed, it curbed, it curbed, it only intensifies it. The agent only intensifies it. The agent only intensifies it. The agent can generate can generate can generate five plausible five plausible five plausible patterns in five patterns in five patterns in five different styles. And different styles. And different styles. And now the human now the human reviewer has to reviewer has to reviewer has to reverse reverse engineer not only engineer not only engineer not only the code, but also the intent behind the code, but also the intent behind the code, but also the intent behind it.

it. it. With proper With proper With proper arrangements, intent is arrangements, intent is arrangements, intent is easier to discern. easier to discern. The answer to this is agreements, agreements, agreements, standards, standards, standards, documented documented documented processes, small processes, small processes, small areas for inspection, areas for inspection, areas for inspection, clear lines of clear lines of clear lines of responsibility, and responsibility, and responsibility, and definition of the definition of the definition of the result. The more result. The more result. The more obvious the path, the obvious the path, the obvious the path, the less room there is less room there is less room there is for for for negligence. I want negligence. I want negligence. I want to express special to express special to express special gratitude to Rails and its gratitude to Rails and its gratitude to Rails and its place in my heart. place in my heart. place in my heart. Rails is important because Rails is important because Rails is important because it shows what happens it shows what happens it shows what happens when a framework when a framework when a framework aggressively reduces aggressively reduces aggressively reduces the number of the number of the number of choices. When choices. When choices. When conventions conventions conventions reduce reduce reduce the workload, the workload, the workload, instead of 10 different instead of 10 different instead of 10 different ways ways ways to structure one to structure one to structure one concept, you concept, you concept, you only get one. only get one. It may not be It may not be ideal in someone's ideal in someone's ideal in someone's subjective opinion, subjective opinion, subjective opinion, but it's understandable and it but it's understandable and it but it's understandable and it works. There was a four-year gap in my career works. There was a four-year gap in my career works. There was a four-year gap in my career between my first and second between my first and second between my first and second experience with Rails experience with Rails . It was great to . It was great to come back after come back after come back after this hiatus and this hiatus and this hiatus and see that the same see that the same see that the same patterns and patterns and patterns and libraries libraries libraries remained the same remained the same remained the same . I could . I could . I could go back to the go back to the go back to the codebase right away and codebase right away and codebase right away and instantly understand instantly understand instantly understand the intent without the intent without the intent without having to having to having to understand the understand the understand the architecture. This architecture. This architecture. This allows engineers allows engineers allows engineers to easily move between to easily move between to easily move between projects with projects with projects with minimal minimal minimal adaptation time. This adaptation time. This adaptation time. This cannot be said about cannot be said about cannot be said about the ecosystems of other languages.

the ecosystems of other languages. the ecosystems of other languages. JavaScript. Environments with a JavaScript. Environments with a JavaScript. Environments with a single correct single correct single correct way to perform way to perform way to perform tasks are better for tasks are better for tasks are better for agent development because agent development because agent development because agents are not confused by agents are not confused by agents are not confused by dozens of dozens of dozens of competing approaches. This is well competing approaches. This is well competing approaches. This is well supported by the supported by the supported by the official official official Rails doctrine. She explicitly Rails doctrine. She explicitly Rails doctrine. She explicitly names optimization names optimization names optimization for for for programmer happiness and " programmer happiness and " arrangement over arrangement over arrangement over configuration" as configuration" as configuration" as core pillars. And core pillars. And core pillars. And describes the arrangements describes the arrangements describes the arrangements as a way to eliminate as a way to eliminate as a way to eliminate repetitive decisions repetitive decisions repetitive decisions and lower the threshold for and lower the threshold for and lower the threshold for entry. This is entry. This is entry. This is the reduction of the reduction of the reduction of cognitive cognitive cognitive load. Everything load. Everything load. Everything I talked about I talked about I talked about today led today led today led us to create Orc. Not us to create Orc. Not us to create Orc. Not because we think the because we think the because we think the answer is answer is answer is more agents, and more agents, and more agents, and certainly not because we certainly not because we certainly not because we believe believe believe engineering judgment engineering judgment engineering judgment can be automated can be automated . We created Orc because we . We created Orc because we . We created Orc because we kept seeing the same kept seeing the same kept seeing the same pattern of pattern of pattern of failure. AI could failure. AI could failure. AI could generate code, but the generate code, but the generate code, but the surrounding surrounding surrounding discipline was still discipline was still discipline was still lacking. lacking. lacking. Planning, review, Planning, review, Planning, review, error checking, error checking, error checking, context management context management . Over time, we noticed . Over time, we noticed . Over time, we noticed that we were constantly that we were constantly that we were constantly repeating the same repeating the same repeating the same human processes human processes human processes to combat to combat to combat sloppiness. Break down sloppiness. Break down sloppiness. Break down the work, manage the work, manage the work, manage context, weigh context, weigh context, weigh options, and make sure options, and make sure options, and make sure the result the result the result gets into the codebase in a gets into the codebase in a way that the team understands. Orc way that the team understands. Orc takes these lessons and takes these lessons and takes these lessons and embeds them into the embeds them into the embeds them into the workflow. This is not a magic workflow. This is not a magic workflow. This is not a magic box that box that box that relieves a person of relieves a person of relieves a person of responsibility for the responsibility for the responsibility for the correctness of the code. It is a correctness of the code. It is a correctness of the code. It is a system for system for system for maintaining such maintaining such maintaining such responsibility, where responsibility, where responsibility, where agents take on agents take on agents take on most of the most of the most of the work. The goal is work. The goal is work. The goal is not to not to not to produce more produce more produce more generated code.

generated code. generated code. The goal is to release The goal is to release software that remains consistent , tested, , tested, maintainable, and maintainable, and controllable after generation. After all, controllable after generation. After all, controllable after generation. After all, the future is not “vibe the future is not “vibe coding” faster and coding” faster and coding” faster and better. The future is better. The future is better. The future is orchestration with orchestration with orchestration with critical thinking. critical thinking. Thank you all for your time Thank you all for your time and remember: don't and remember: don't and remember: don't slack off.