Seeds and Germination: Software That Evolves in the Field

This is part 3 of a four-part series. Part 1 covered the split between agents, which have become general-purpose workers, and specialized tools. In this series, "worker" means a general-purpose agent, not a person. Part 2 argued that even organizations with air-gapped networks need a runtime where that worker can do its job. In part 3, I want to take that environment as a given and think about building software and applying it at customer sites.

The place where a product is built and the place where it is used operate under different conditions. Data shows up that the development environment never saw. Systems with the same name run different versions and configurations. The work described in documents differs from the work people actually do. A supplier builds common features, but making those features do their job in the field takes additional work.

Agents can take part in this field adaptation. While people come to understand the site and make judgments, an agent can inspect data, write integration code, and test changes. Separately, you can build features into the product itself that detect and respond to differences in the environment. Both can be described as "adapting to the field," but they are two different jobs and should be looked at separately.

The first is how people and agents build a product and apply it in the field. It is a question of who uses which tools to change what. The second is the structure by which a product responds to differences in environment and data while it runs. It is a question of which differences the product handles internally and what it keeps for the next run.

Field collaboration between people and agents applies even to ordinary software with no self-adaptive features. In-product adaptation can be implemented with rules and statistics, plus model calls where they are needed. The two approaches address different design problems, and they meet when experience gained in the field is fed into the next version of the product.

Seeds from outside, germination in the field

Once coding agents arrive on internal networks, the way SI (system integration) projects are delivered may change too. Picture a setup where Claude Code or Codex is used outside, and a separately built agent runs on the internal network.

The models, compute, and connectable tools can differ between the two environments. If internal resources are limited, matching the capability available outside may be hard. You need to divide up which work can be given to each environment, and decide how to connect them.

The external agent can be used to design the product's structure, implement common features, and test against a range of assumed conditions. The agent inside the site has access to real data, real systems, and the organization's working context. Reasoning capability and accessible information end up split across two environments, and the delivery model has to be designed on the assumption of that gap.

You can think of this division of labor as "seeds and germination." The seed is made outside, and it germinates in the field.

Here the seed means both the initial software you bring to a particular site and the structure that lets that software change to fit the site. Its basic features must already work, and it must already be defined what the inputs and outputs are, which parts can be adjusted, and how to check the results of an adjustment. Germination is the process of observing the real environment and turning that potential into concrete integrations and behavior.

Say you supply a product that analyzes operational data. Outside, you can prepare the analysis engine, a common data structure, examples for checking results, and a test method. In the field, you find out which collectors are in use, where the data lands, how service names are written, and who needs which results. This is where the differences you need to bridge to connect the common product to real work come to light.

In the field, you write scripts, config files, and memory files to fit the real environment. Scripts connect the site's systems, config files express how the environment differs, and memory files record facts already confirmed and points to watch out for during the work.

Information recorded this way spares the next person or agent working at the same site from investigating everything from scratch each time. Confirmed facts still need to be kept apart from tentative guesses, though, and it should be clear when and under what conditions each piece of information was obtained.

The better the supplier prepares in advance, the less uncertainty the field agent has to carry. When the available integration points, examples, error descriptions, and verification commands are clear, the work that needs coordinating can be broken into small pieces. On top of that, the scope handed to the agent should be set with the field model's capability in mind, and hard judgments and fixes should go to people or be escalated for additional support.

The seed of the seed, and the role of the FDE

Handling site-specific differences already matters in SI today. The experience of connecting customer systems and building software that fits their work does not vanish because agents have arrived. If anything, it becomes more important for people to translate that experience into tasks and verification criteria an agent can carry out.

This is where the role of the FDE, or Forward Deployed Engineer, who applies a common product in the field and develops it further, comes in. Palantir's official description emphasizes a process in which engineers close to the field and the core product team combine feedback to build new features. What I focus on is that solving field problems and improving the product are connected.

Extending the seed metaphor, the FDE is the person who uses a common design, the "seed of the seed," to make the seed brought to a site, and who helps it germinate alongside the field AI agent.

The "seed of the seed" means the design principles and common foundation that can be applied repeatedly across many sites. It also includes knowledge about what to keep structurally the same, which differences to leave to each site, and which results count as correct behavior. The FDE builds implementations that fit site conditions while understanding this foundation.

The FDE makes the problem the customer wants to solve concrete, decides which part of the product should solve it, and checks whether the results of a change fit the actual work. In turning customer requests into development work, the FDE also separates requirements that site-specific configuration can satisfy from defects in the common product.

The agent helps turn those judgments into execution. It can investigate data structures, draft adapters, compare results before and after a change, and document what was confirmed. The constraint that a person has to write every file by hand loosens, but someone still has to judge what needs to change and when a problem counts as solved.

The FDE also has to decide where changes live. Values that differ per customer can be kept as configuration and mappings, code that connects multiple systems as adapters, and investigation steps worth repeating as Skills and work procedures. Cases that made it possible to tell correct results apart are written up as evaluations and acceptance criteria. Once field requirements are broken down into these forms, the same features and knowledge can be reused even when the customer's agent changes.

The system integration and field coordination experience built up in SI can carry over into this FDE way of working. The role keeps the reusable foundation as a product, has people and agents build field fit together, and returns the results to the common product. I expect this kind of role to take up a growing share of development work.

To turn this direction into an actual business practice, time has to be set aside for feeding field work back into the product. If code patched in a hurry at a site survives only as a customer-specific branch, and the common product team never learns why, the next site will repeat the same work. A structure that holds the same people responsible for both field fixes and common product improvements raises the quality of the seed.

Structure for product change and accumulation

Adjustments that recur at every site are clues for changing the structure of the product itself. Some differences can be left open for people to configure, and others can be handled by the product directly by observing the data. Widening the scope of the latter requires designing what the product changes and what experience it keeps.

When designing a seed, it is hard to implement every site condition in advance. Instead, you design it to adapt to varied field situations by separating the skeleton that must stay solid, the parts where change is allowed, and the parts that accumulate change.

In this design, dividing responsibility among the three parts matters. The "solid skeleton" maintains the product's basic behavior and guarantees. The "parts that allow change" absorb conditions that differ per site. The "parts that accumulate change" carry confirmed experience into the next run and the next task.

The solid parts can also change during product development. But you need to separate what field work is free to shake loose from what must go through review by the common product. For example, adjusting a data field's name and changing the criteria that judge an analysis result as a success should not be handled with the same permissions.

Source code changes are not the only way to allow change. If configuration can fully express a site-specific difference, use configuration. Differences that a fixed transformation can resolve go into adapters, and a model can be used where semantic judgment is needed. You should be able to choose the simplest and most easily verified method based on what is actually different.

An extensible structure gives people or external workers touchpoints for changing configuration and processing methods. Automatic adaptation is the product's ability to observe real data and choose that configuration or processing method itself. Saving the chosen result and using it in the next run makes the effect of adaptation last. Making a place for change and automatically choosing and accumulating changes each have to be designed separately.

The idea of "tools that evolve in the field" starts at exactly this boundary. You have to decide how much the product handles on its own and at what point it hands off to a person or an external worker. The fact that a general-purpose agent can fix the product does not by itself complete automatic adaptation inside the product.

The adaptive analysis layer of an operational data analysis product

The work where I dealt with this problem concretely was the product's adaptive analysis layer. In operational data analysis, collectors and data formats differ from site to site. CPU usage arrives as a current utilization rate in one place and as cumulative usage time in another. To compute memory usage as a ratio, you also need to know which value is usage and which is the limit.

When the assumption that data arrives in the format the analyzer expects breaks, the analyzer can miss the information it needs no matter how good the downstream reasoning is. In the product, we implemented an adaptation layer to handle these input differences. We first experimented with a pipeline that explored data structures and synthesized analysis results, and later built an auto-mapping adapter that transforms input so the existing analyzers can use it.

The experimental synthesis pipeline first found fields and numeric data in the input and flattened them into a common shape, then looked at how values changed to tell cumulative values from point-in-time values. Cumulative values were converted to deltas for comparison, and it computed relationships where multiple metrics moved together or with a time lag. For logs, instead of only searching for particular error words, it used features such as spikes in volume or rarity to narrow down what to look at.

In the experimental pipeline, code did the data processing and statistical computation, and the model was called directly once, at the final interpretation step. The computed observations were passed to the model, which interpreted their meaning.

Analysis features that the product repeats internally like this can be implemented with the code and model calls they need. A general-purpose worker that combines many tools and an adaptation layer that handles differences in specific inputs each get an execution structure suited to its role.

After that, we implemented an auto-mapping adapter that converts metric formats the engine did not anticipate into the common structure the existing analyzers read. The adapter combines field naming rules, value ranges, whether a value is cumulative, and similar signals. The mapping itself does not use model calls. Because it provides a transformed view at read time without overwriting the original data, turning the mapping off falls back to the original path.

For example, it converts cumulative CPU time into a utilization rate, or divides memory usage by the memory limit to produce a ratio. We checked whether it could automatically reproduce mappings that developers had previously built by hand for specific data, and on the real data cases we examined, we verified agreement with the reference results. The earlier experimental synthesis path was later removed from the engine's execution modes. The auto-mapping adapter is a separate implementation focused on handling input differences.

What this implementation validated was finding input differences in some real data and connecting them to the common analysis structure. An evaluation comparing overall diagnostic quality was not completed at the time. Mapping correctness and the usefulness of the final diagnosis each have to be confirmed before the scope can be widened.

What actually struck me most in this work was a silent failure that real data exposed. Among the fields that looked like they could be memory usage, the initial implementation picked one, but instead of actual usage it chose a different field whose value was always 0. The mapping step succeeded and result rows were produced, but the computed memory utilization stayed at 0.

We found this error during development by comparing the original fields and transformed values against the reference results. We then changed the selection rule to check whether a value shows meaningful variation and to prefer candidates that correspond to actual usage. The experience of a person reviewing the run results and evidence and making the correction fed into the next implementation.

This case shows that the adaptation feature itself must be verified too. Once a product that used to accept only a fixed format starts accepting more varied input, a wrong reading of the input can turn into plausible-looking output. So you have to record what it adapted to, what it based its choice on, and which data it could not process. That record is the starting point for the next correction.

The scope of self-healing and self-evolution

Push this kind of adaptation further and you arrive at the questions of self-evolving agents and a self-healing layer. The core questions are how to detect failures, what to change within the allowed scope, and how to carry changes that worked into the next run.

To implement this as a concrete product, you first have to decide what is changing. The structure you need depends on whether it is choosing a different processing method within a single run, changing configuration, updating memory, or modifying code and tests. Only once the target and scope of change are clear can you decide how to implement it.

These layers can improve together, but a change in one does not imply a change in another. Fixing the rules that align input in that product, giving the field agent a better investigation procedure, and retraining a model differ in what they change and what results need checking. Rather than lumping them together under words like "learning" or "evolution," you should explain what is kept where and how it changes the next run.

In this post, I treat self-healing as restoring expected behavior through changes within an allowed scope when failing or deviating behavior is detected. For example, when an input field's name changes and the existing mapping fails, you could design a flow that finds new mapping candidates, checks that the existing analytical meaning is preserved, and then applies one. This is a direction the product's implementation described above could go further in.

I treat self-evolution as the process of keeping the changes that proved valid, using them in later work, and widening the range of new conditions that can be handled. This does not necessarily require retraining model weights. Keeping validated mappings, new adapters, better work procedures, and reproducible failure cases can on its own change how software handles the next environment.

If a different method is tried temporarily within one run and succeeds, that is adaptation within that run. If the next run recognizes the same condition and reuses the validated method, experience has accumulated. If that method is organized and shipped so other sites can use it, it becomes a product improvement. These three stages can be distinguished by where the change is kept.

The thing that generates change candidates does not have to be a single actor either. A small model inside the product can propose a new mapping, a general-purpose field agent can write a script, and a person can modify common code. How candidates are generated and the process for adopting a candidate as a valid product change each need to be designed.

Memory that keeps experience has the same problem. Information like "this service usually uses about this much memory" can be a starting point for the next analysis. But if that memory keeps you from looking at other current evidence, you can miss a new incident. Past baselines should be assumptions that current observations can revise.

In a follow-up design for the product, we considered separating the path that consults past baselines and known cases from the path that observes current data fresh. Reports and observation records would accumulate, while normal ranges and judgment criteria computed from those records would be recomputed periodically and the influence of old information reduced. This is still at the design stage, and whether accumulating experience actually improves diagnostic quality remains to be confirmed.

Keeping a lot of memory is different from accumulating useful experience. You need to be able to handle which environment a fact held in, whether that environment still holds, and what counterexamples have been found. Alongside the ability to adopt new experience, you also need the ability to correct and discard experience that is wrong or stale.

Ledgers and traces as the basis for correction

I think a product that deals with change in the field should record every run in a ledger and a trace, and be able to audit and correct based on those records. When something goes wrong, you need to be able to examine not just the result but how it was produced.

The execution ledger records what tasks and changes took place. The trace lets you follow the calls, decision rationale, and intermediate artifacts that led to a result. This record-keeping system is a design principle I intend to apply when extending the adaptation features described earlier.

In the auto-mapping example, the record should show which source field was chosen, which transformation was applied, and which data could not be processed. If something changed, you should be able to see the results before and after the change and the conditions used to verify it. If you store only an "analysis succeeded" status and the final report, it is hard to trace why memory utilization was computed as 0.

If a change affects later runs, you also need versions and a scope of application. You have to be able to decide whether a mapping confirmed on a particular server should apply to all servers or stay with a particular customer, and to roll back to an earlier state if something goes wrong. Fixing the feature and judging correctness also need to be kept apart, so that success is never manufactured by loosening the verification criteria to fit a change candidate.

These records also help bridge the capability gap between outside and inside. When a problem is hard to solve internally, you can gather the material the site allows to be taken out, go through the required approval process, and get help from outside. You build reproduction examples and error descriptions that can be handed over, and external agents and developers produce fix candidates.

A fix made outside has to be checked again under the real internal conditions, because gaps between the reproduction example and the site can remain due to data or system state the outside cannot access. The outside produces candidates, and responsibility then passes to the field to check applicability and results. Here too, the line between original material that may not leave the site and reproduction material that can be provided externally is set by the site's conditions.

Connecting records to correction requires a process for proposing, checking, applying, and if necessary reverting changes. Ledgers and traces provide the evidence each of those judgments needs. To the extent you leave the field open to change, you have to be able to explain the reasons for and results of that change.

Feeding field experience back into the product

The flow in which the experience of software that germinated in the field returns to the common product resembles evolution. Among the variations produced by differences in the field, you select what to keep in the next product.

Not every change made in the field needs to go into the common product. A specific customer's service names and paths, and field mappings valid only in that environment, can stay as that site's configuration. Transformations needed by several customers using the same collector can become reusable adapters. If failures repeated across several sites reveal limits of the common structure, the product's skeleton and design principles need to change.

The order of tool use and the judgment criteria that worked during investigation can be kept as Skills or work procedures. Which results were accepted as correct, and which failures can be reproduced and checked, become evaluation cases and acceptance criteria. Even if these assets never become part of the product code, they change the quality of work at the next site. Feedback only actually happens once you decide where configuration, adapters, working knowledge, verification cases, and common product improvements are each managed.

This selection process connects the two kinds of adaptation discussed earlier. People and agents can investigate new inputs or failures discovered inside the product and improve on them. Conversely, among the adjustments people kept making by hand in the field, the parts whose rules have become clear can be moved into the product's internal adaptation features. The flow is to learn from field work and have the product take over the repeatable parts.

What flows back is not just finished fix code. Reproduction examples showing the conditions under which a problem occurred, the reasons wrong candidates were ruled out, and tests that later changes must also pass need to come back with it. That is how the seed brought to the next site can carry earlier experience.

Anthropic's guidance on code modernization projects likewise recommends keeping workflows, completion criteria, and verification evidence as reusable assets alongside the finished code. What the next seed should carry is both the fixed feature and the method used to build and check it.

The seed of the seed also changes through this process. You may learn that a part you thought site-specific configuration could cover actually needs a common interface. You may also learn that a feature you expected to help every customer is only meaningful in certain environments. Field experience becomes grounds for redefining not only the implementation but also the scope the product takes responsibility for.

Trying changes to fit observed differences, verifying the results, and selecting and accumulating what holds up is what this post means by evolution. Field experience becomes product capability when you can confirm which changes worked and carry them into the next run and the next deployment.

This shift changes both how the product is positioned and how development projects are delivered. The supplier offers solid common features and a structure that can be adjusted in the field, together, as the product.

The FDE and the field agent apply that product to real work, and the product's internal adaptation features handle differences within a defined scope. Validated field experience flows back into the common foundation and changes the seed brought to the next customer.

In the next post, I will apply this perspective to the business strategy of a specialized monitoring product. I will look at what value the choice to build product touchpoints that general-purpose agents can use gives to customers' work, and what business opportunities building runtimes and accumulating field experience create for suppliers.