A proof of concept should answer the question that decides whether a planning system is worth adopting, not simply produce an attractive demonstration. It accepts a dataset, runs an algorithm and produces a solution that looks better than a spreadsheet plan. But the strongest proof of concept does more than demonstrate that software can return an answer. It establishes whether an optimization approach is worth developing and deploying, under conditions that resemble the real decision.
That requires asking the right questions before the work begins. Which outcome would make the project valuable? What existing method should serve as a comparison? Which constraints must be satisfied, and what would make the prototype unsuitable for production even if its objective score improves?
Begin with a measurable decision
Suppose a manufacturer wants to reduce changeover time. A first prototype might optimize job sequence on a single line using historical processing and setup data. The measure could be total setup time, but that cannot be the only success criterion. Every job must remain schedulable, delivery requirements must be respected and the planning effort must fit into the available decision window.
A clear evaluation question might be: can the candidate model produce feasible schedules within five minutes and improve total changeover time against the current planning method on representative historical weeks? This is a better question than “Can we build an AI optimization engine?” because the result has a defined meaning.
If a prototype is allowed an hour while the operational process has only five minutes, the demonstration may not answer the real question.
Choose a fair baseline
A baseline is not necessarily the previous spreadsheet total. It might be an experienced planner's schedule, a transparent dispatching rule or an established solver using a simple formulation. Where appropriate, compare more than one approach to understand whether sophisticated algorithms are actually needed.
Input data and evaluation conditions should be matched. If one method sees future information unavailable to another, the comparison is unfair. If a prototype is allowed an hour while the operational process has only five minutes, the demonstration may not answer the real question. Likewise, excluding difficult weeks because they make the prototype perform badly conceals exactly the cases that deployment must handle.
Test the difficult cases on purpose
The strongest test set includes ordinary workloads, busy periods, unusual exceptions and disruptions. It should contain representative constraints rather than a sanitized version of the problem. Some instances may be infeasible, and the system must communicate that accurately instead of producing an attractive-looking invalid plan.
For a recurring scheduling product, also test changes to an existing plan. Can the tool freeze work already in progress? Can it reoptimize without needlessly rearranging every task? Can users inspect the result and understand why a particular choice was made?
The strongest demonstration is not the most impressive screenshot but the one that answers the organization's actual adoption question.
The proof of concept should identify failure modes, not hide them. An algorithm that performs brilliantly most of the time but regularly exceeds its time limit may require substantial redesign before integration.
Measure the complete workflow
Solver runtime is only one part of the process. Data must be collected, validated, transformed into a model and interpreted after execution. A model that finishes in ten seconds may offer little operational advantage if every run requires several hours of manual data preparation.
The evaluation should record implementation effort and operational dependencies. Does the source data exist in reliable systems? What needs to happen when a run fails? Who will maintain the model as policies change? What are the privacy, security and deployment requirements?
A good prototype may conclude that the best first investment is better data, clearer rules or a simpler model. Those are valuable findings, even if they do not lead immediately to a software subscription.
Finish with a decision, not a presentation
A proof of concept should end with a short evidence report: what was modeled, which datasets were used, how results compared with the baseline, where the approach succeeded, where it failed and what remains uncertain. Ideally the outcome supports a clear choice: proceed, revise the approach, gather missing evidence or stop.
This is how an optimization experiment becomes useful to a business. It converts enthusiasm into a measured judgment about feasibility, value and next steps, without pretending that one attractive demonstration settles every practical question.
Design the test around the decision deadline
Suppose a planning team needs a usable schedule before a shift handover. One candidate solver produces a high-quality schedule in five minutes; another produces a slightly better one after an hour. Which result matters depends on when the schedule must be committed. A proof of concept should therefore test not just final objective values, but time to a feasible plan, time to a useful target, behavior under changes, and integration with real input data. Compare with the existing planning method on representative instances. The strongest demonstration is not the most impressive screenshot but the one that answers the organization's actual adoption question.