How to Choose the First Backlog Item for an AI-MSL Pilot, and What You Should Hold at the End of It

The item you pick for a pilot decides what the pilot can prove.

Pick a toy and you learn that AI can build a toy. Pick the thing that has been stuck for three quarters and you learn nothing in a week except that it was stuck for a reason. Most evaluations of AI software delivery fail at this step, before anyone connects a repository, because the item was chosen to be impressive rather than to be decidable.

This post is about that choice. It is written for the people the AI-MSL pilot is built for: a company with a product in production, a backlog it would otherwise hand to its own team or an outside vendor, and a business decision maker who will sit in the review at the end. The pilot runs one real backlog item, sometimes two, through the full lifecycle on your own code, with $2,000 in Dev credits on us and a named AI-MSL Engineer signing every gate. About a week, scoped before day one, reviewed on day seven.

What follows are the criteria that matter, the criteria that look like they matter and do not, how to weigh them, and the evidence you should be holding when the week ends.

The decision you are making

The pilot is not asking whether AI can write code. That question was settled a while ago, and the answer is not very useful. It is asking a narrower question that is useful: on this system, for this kind of work, what does a governed AI delivery loop produce, what does it take, and how does that compare with what the same item costs you today?

Every word of that question is set by the item. The system is whatever the item touches. The kind of work is whatever the item is. The comparison only exists if you know what the item would have cost you otherwise. So the choice of item is the design of the experiment. Treat it that way.

The criteria that matter

It would otherwise go to your team or your vendor

The pilot page puts it plainly: something you would otherwise give to your team or your vendor. A maintenance item, a modernization step, a smaller feature. Not a toy.

The reason is the comparison on day seven. If the item is one you would never have paid for, there is nothing to put next to the result. If it is one that is already in a sprint plan or a vendor’s statement of work, you know its price, and the review becomes a real decision rather than an opinion about a demo.

You can write the definition of done in a paragraph

Before anything starts, the item, the definition of done and the review date are agreed in writing with your AI Lifecycle Manager. That is not paperwork. It is the test the result will be held against.

If you cannot write what done looks like without a meeting, the item is not ready for a pilot. It may be a good item for the second week, once the assessment has shown you what is under it. For the first week, pick something where everyone in the room would agree on day seven whether it shipped.

You know what it costs you today

The day-seven review puts what the pilot took, in Dev credits and in our engineers’ hours, next to your time and cost for the same item today. You supply the second number. If nobody can say what the item would have cost in your current setup, the strongest output of the week is missing. Pick an item where that number exists, or can be reconstructed from a recent comparable piece of work.

It touches real code with history

The assessment runs against the repository as it is. That is where the pilot earns its keep, because a production system carries decisions nobody wrote down, and your codebase knows things your team forgot. An item in a clean corner of a new service tells you very little about how the loop behaves on the system that pays your salary. An item that crosses a module boundary, depends on an old integration, or sits near something fragile tells you a great deal.

It fits a week and the allowance

Your AI Lifecycle Manager checks that the item fits a one-week pilot and the credit allowance, and estimates it in Dev credits before anything starts. Work with that. The point of the first item is to get through the whole lifecycle, requirements to release, with a result you can judge. An item that would be half-finished on day seven proves nothing, however interesting the half is.

The criteria that look important and are not

The hardest item on the backlog. Teams reach for it because it feels like the honest test. It is the dishonest one. A hard item fails for reasons that have nothing to do with the delivery loop: unclear ownership, a dependency on another team, a design decision nobody has made. You will not be able to tell which failure you are looking at.

The item engineers are most curious about. Curiosity is good. It is also the wrong selection criterion, because the decision maker at the review needs a result they can price, not a result the engineers found elegant.

The item with the most stakeholders. More people in the definition of done means more time agreeing it and less time running it. Save it for when the loop has already been seen to work.

The safest possible task. A documentation-only change or a dependency bump is tempting because it cannot go wrong. It also cannot show you a specification being signed, a test failing before the code exists, or a change record you would put in front of a board. Safe is fine. Empty is not.

How to weigh them

The pilot page names three shapes of item, and each proves something slightly different.

A maintenance item proves the loop can work inside a system it did not build, under a definition of done the system already implies. It is the best choice when your concern is whether AI can be trusted near production code at all.

A modernization step proves the loop can change structure without breaking behavior, and shows you how the assessment reads your architecture. It is the best choice when the real question behind the pilot is a bigger piece of modernization work.

A smaller feature proves the loop can go from a business request to a specification to tested code with the business present. It is the best choice when the buyer is the product owner or the business, because they will see their own request turned into a record they can read.

If two items satisfy the criteria equally, pick the one whose cost today you know best. The comparison is the output that survives the meeting.

What you should be holding on day seven

This is the part most pilot plans skip, and it is the part that matters. Here is what the pilot leaves with you, whatever you decide next.

Working code, in your repository. A production-ready branch for each item, with its tests and documentation. Not a screenshot. A branch you can review, merge or leave alone.

The record behind each change. The request, the approved scope, the specification, the sign-offs by name at each gate, and the test results. Every action logged with a timestamp, an actor and the reasoning. This is the artifact to read most carefully, because it is what a governed loop produces that an assisted-coding session does not. If you want a sense of why that record looks the way it does, the ten differences between an AI-native lifecycle and the one you run today are a good place to start.

The assessment of your codebase. What the automated assessment found in your code and what it would do about it. It runs against the repository as it is, and it is yours.

The comparison. Your time and cost for the same item today, next to what the pilot took in Dev credits and in our hours. This is the number the decision maker came for.

A decision you can defend. Continue on a plan, scope the next piece of work, or stop and keep the code. Nothing continues on its own, which is why the business decision maker needs to be in the room on day seven and not briefed afterwards. The product manager’s job changes when AI builds the code, and the review is where that becomes visible.

A pilot that ends without those five things in your hands has not finished. A pilot that ends with them has given you a priced, recorded, reviewable answer to the only question worth asking: what would this do on our code, and what would it cost compared with what we pay now? That is also the question behind paying per change rather than for developer time, and the pilot is the smallest honest way to test it.

A short decision aid

Before you propose an item, check the four conditions the pilot needs: a product in production, a real backlog item, a business decision maker who will be at the day-seven review, and a repository you can connect yourself from your own login. If one is missing, the better first step is a conversation about plans, not a pilot.

Then put the candidate item through five questions.

• Would we otherwise pay our team or our vendor to do this?
• Can we write what done looks like in a paragraph, today?
• Do we know what it costs us now, in time and money?
• Does it touch code with real history, not a clean corner?
• Does it fit a week and the credit allowance, as scoped with the AI Lifecycle Manager?

Five yeses and you have the first item. Pick it, claim the pilot, and agree the definition of done before day one. The $2,000 in Dev credits is on us. The decision at the end is yours.