Replacing the Product Manager with an Agentic AI System

As Agentic AI systems and infrastructure become more and more ubiquitous, and its application becomes more accessible, there is a frenzied build-up of expectation that AI, specifically agentic systems, will quickly replace most human jobs.

While AI technology facilitator stock prices go through the roof, I was not totally convinced on the transformative powers of agentic systems. Is it as prodigiously capable as it is hyped out to be, and were our jobs really in danger?

So, I decided to try building an Agentic system to replace one such job that I know very well – Product Management (the premise being, although building a working system would jeopardize my employability, if it's happening anyway, i.e., AI taking over Product Management duties, at least I get to know about it first 😊)

On paper, Product Management looks like an “AI replace-worthy job”, as it is intrinsically based on data analysis. Most of the time of the Product Manager is spent in analyzing data from various sources - be it Financial Performance or Revenue data, market and competition data, data from requirements management system, etc, in order to make strategic decisions.

It took about a month of weekends and after-hours tinkering to build the structure of an Agentic AI system that would output a features and functionality roadmap for a portfolio of products for any market or markets, a major component of the Product Manager job description. I tested this on the Cybersecurity Exposure Management market, however, it is applicable to any product portfolio and domain. You can find the code in github here.

The Logic

Typically, in building a roadmap of features and functionality for a portfolio of products, the main objective of the Product Manager is to select a feasible set of requirements that maximize the value (Revenue) for customers within a specified budget and specified timeframe while taking acceptable risk.

There a several data driven decision points that the Product Manager has to take while making the roadmap, such as – how to split the budget between product and marketing/sales, what optimal resourcing is needed to build the roadmap, and which features to take up first or mandatorily and what can be taken up later, etc.

The Value Maximization System

The system consists of 4 agents – a Product Manager Decision/Orchestration agent, a Resource Manager, a Feature (Real) Value agent, a Perceived Value agent. Here is the schematic for the system:

The Product Manager Agentic system

a)       The Product Manager – The Product Manager agent is the orchestrator. It initializes the inputs for the other agents and runs a conditional loop for value maximization, when input the initial backlogs of features and capabilities and the overall budget. It decides on a budget split between Resourcing and Marketing and executes the loop. It also decides on mandatory features and step size. Theoretically it negotiates a complex gradient descent, where each agent serves the purpose of a sub-regression.

b)       The Resource Manager – the first handoff agent in the conditional loop, the resource manager utilizes resource cost and availability along with features information to build a timeline of features that can fit into the given budget, to create the features list passed to the other agents.

c)       The Real Feature Value agent – This agent finds the real value of each feature functionality. It utilizes a Jira MCP to find the relative importance of a feature or functionality. It also checks the market definition of the product to determine core features and functions, which is used to prioritize the features roadmap.

d)       The Perceived Value agent – This agent determines the perceived value of a feature, that is an amplification factor on the real value. The perceived value depends on various factors, such as the innovation, uniqueness and market demand for the capability, and is also affected by the market stage, the marketing budget and the targeted user persona. The perceived value agent used a RAG to search a repository of knowledge about the particular market, as well as retrieving information from the web. It then used a FAISS database to index the data and based on the feature and functionality information provided, it would retrieve the most relevant documents form the RAG. Finally the retrieval relevance is computed using a Langsmith evaluator and, if found to be relevant, it returned a normalized value score.

Results and Conclusion

I tried the system out on a fictional cybersecurity company targeting the Continuous Threat Exposure Management market. I used a GTP-4o-mini model to keep the budget low.

I put a limit on the number of optimization iterations to 50.

a)       The AI agents for Resource Manager and Perceived value performed efficiently retrieving the specific Resourcing costs and overall value from the set of features and functionality provided

b)       The Product Manager had some issues in taking decisions despite providing comprehensive prompts and suggestions on the behavior of the optimization function. Turns out that the Product Manager job is quite complex due to the interdependencies between the requested capabilities and their impact on the values. For example, a specific innovative feature would impact both the real value and the marketed perceived value and the value relation is not linear. While the output of the decision loop did converge/stop in most cases, it did not always guarantee an optimal output. Apparently, an LLM decision model is not very efficient in performing value optimization. A mathematical regression solver would likely do a better job at optimizing. Where the LLM shines is in finding the relationships (linear or non-linear) between variables via the Agent subsystem, without the need for additional model training. By modeling variable relationships as agents that could use RAGs, various integrations, or just the trained LLM to model the relations, breaks the problem down into manageable parts and does not require labeled datasets to train. In that sense, an Agentic system is similar to humans. If I added training and feedback loop as well to the Agents, it would be able to learn and perform better over time.

c)       There are several improvements that need to be done to the system to bring it to shape: including a better model for the HR and Finance functions to accurately reflect the future cost of resourcing, as well as the acceptable risks for an organization, the Cost of Capital, etc, that would influence decision. Also, as suggested earlier, creating a model for revenue to value relation across different verticals or horizontals would produce better results.

d)       The system, in its current state, (and with suggested modifications) while automating a lot of the leg work, still needs human supervision and decisions on the results produced. In other words, the Product Manager job seems to be safe, for the time being. However, the big learning was the sea changes that AI platforms like Colab/Copilot/Claude are bringing to speed up code output. It is not perfect, a human still needs to understand the complete logic of the code produced and tweak it as required, but putting together code from scratch in any language, to get an idea to market, is much easier now. The development time should have reduced 30-40%. In my opinion, that is the biggest benefit that AI is bringing to the table at this time.

Let me know what you think.

Outputs of the system after 10 iterations:

Current features count:6

Total feature value: $250.40

Total Perceived Value: $196.01

Total Value (sum of feature and perceived value): 446.41

Previous Decision: Increase marketing budget

Supplied budget for feature = 400000.0

Total months for this cycle: 13.0

Selected feature 'Provide vulnerability attack blocking' with cost $193,200.00.

Selected feature 'Provide customization option for risk score' with cost $28,000.00.

Selected feature 'Manage asset importance scoring' with cost $33,600.00.

Selected feature 'Provide secure user login via SSO' with cost $50,400.00.

Selected feature 'Export Exposure Management Reports' with cost $58,800.00.

Current Iteration: 10

Supplied Marketing Budget: $600,000.00

Adjusted Marketing budget:$572,400.00

Total Available Budget: $1,000,000.00

Feasible: True

Current features count:6

Total feature value: $306.40

Total Perceived Value: $248.24

Total Value (sum of feature and perceived value): 554.64

Previous Decision: Decrease marketing budget

--- Workflow Execution Result ---

Overall Feasibility: True

Total Value: $554.64

Final Marketing Budget: $572,400.00

Summary Reasoning:

### Comprehensive Roadmap Evaluation

#### Overall Feasibility

The evaluation indicates that the roadmap is feasible. The Resource Manager Agent confirms that the project can successfully incorporate the necessary features within the estimated budget.

#### Key Timelines and Resource Management

- **Estimated Cost**: The total estimated cost for the project is **$364,000**.

- **Initial Marketing Budget**: The initial marketing budget allocated was **$636,000**. After adjustments for targeting new user personas, the final marketing budget stands at **$572,400**.

- **Feature Selection**: The final selection includes five features, all of which are strategically chosen to maximize the overall value and address market needs.

#### Total Value

- **Total Feature Value Score**: **306.40**

- This score reflects a combination of innovative features aimed at enhancing usability and addressing critical market demands.

#### Final Marketing Budget

- The adjusted marketing budget of **$572,400** is significant for promoting the newly identified user personas and enhancing perceived value through targeted marketing efforts.

#### Total Feature Value

- The total feature value of **306.40** is derived from various innovative features, with specific contributions as follows:

- **Provide vulnerability attack blocking**: 84.80

- **Provide customization option for risk score**: 61.60

- **Manage asset importance scoring**: 56.00

- **Provide secure user login via SSO**: 48.00

- **Export Exposure Management Reports**: 56.00

#### Perceived Value

- **Total Perceived Value**: **248.24**

- This value reflects both feature contributions and the adjusted marketing budget aimed at new user personas.

- The perceived value from features stands at **$156.00**, and additional value from targeting new markets/personas adds another **$20.00**.

### Actionable Summary

- **Proceed with Development**: Given the positive feasibility assessment and a total feature value score of **306.40**, development should proceed with the outlined features.

- **Focus on Targeted Marketing**: Utilize the adjusted marketing budget of **$572,400** effectively to maximize outreach to the new user personas identified, specifically for the features of **vulnerability attack blocking** and **Exposure Management Reports**.

- **Monitor and Adjust**: Continuously monitor the perceived value from the features and adjust marketing strategies as necessary to ensure alignment with market demands and user needs.

This comprehensive evaluation indicates a well-structured approach to feature development and marketing strategy, setting a solid foundation for project success.

Feature Value Output:

Total Overall Value Score: 306.40

- Provide vulnerability attack blocking: Score 84.80

- Provide customization option for risk score: Score 61.60

- Manage asset importance scoring: Score 56.00

- Provide secure user login via SSO: Score 48.00

- Export Exposure Management Reports: Score 56.00

Perceived Value Output:

Total Perceived Value: $248.24

Adjusted Marketing Budget: $572,400.00