Machine Learning Project

Background and goal

This post covers different aspects of a Machine Learning (ML) project, looking at it from the project / program manager point of view.

• This post is being updated as new feedback is received and new experiences are lived.
• This post is being divided in different posts as its so large to digest.

Every initiative needs to be managed, projects is a vehicle to manage and implement initiatives. There are other ways to do it and they are valid ones. But in the environments where scalability and understanding about where the budgets are used project management is very valuable and used in the industry.

The goal: review a Machine Learning initiative from project management point of view.

Before to start the project

I understand that the answers to these questions are clear to the reader, but in case it’s not, help yourself:

1. For the ML project, should I use Agile or waterfall approach?
2. From the technology platform or solution, usually there is a clear decision about the one that is going to be used. If this is not the case, then take into account that your project, the technology will be evaluated and challenged.
3. If you need to have a conversation about the maturity of the customer organization in terms of ML or AI, you can use the Gartner maturity model as reference. So many CxO are familiar with this model, so it’s better to use their language.
4. If you are part of a factory or a portfolio where the basis are already stablished, so much of the work in terms of education, initial learning and major decisions, etc. is probably done.
5. I you are leading the first ML project, pay a lot of attention to the ML strategy of the organization, understand the journey defined on that strategy document or business case. Ensure that all relevant stakeholders are present on those documents.
6. If you are part of the first ML project, you have to pay a lot of attention about the technical expertise of the stakeholders, understand the strategy the organization has related to ML, understand what is build and what is still not build. Prepare yourself as PM to work a lot on evangelization of ML, look for the definition of a problem, dig into the strategy and decisions to be done by the organization, etc. Be ready to have stakeholders that will reject the change.
7. Is this the first Machine Learning project? Or you are implementing a specific project under a program/portfolio that is already running.
8. Understand the level of knowledge the stakeholders have about machine learning.
9. Project requirements: the project manager uses to receive a clear set of project requirements that are used to define the solution approach, the project deliverables and many other aspects of the final deliverables. It can happen that the agreed project to be implemented cover these requirements partially as the budget is capped and the initial version of the solution exceeds that budget.
10. Be familiar with the CRISP-DM methodology.

Project Definition sections

The main document a project manager handles is the project definition. This document defines what, how, when, why this initiative is going to be done. In many cases it’s the contract. In every case an approved project definition is a commitment of the organization.

In project based organizations project definition is a common deliverable, so I will review relevant sections digging into the ML singularities. I will not cover standard project approach.

Sections to be reviewed

• Project background and goal.
• Project scope
• Project approach
• Project deliverables
• Project estimation
• Project schedule
• Project assumptions
• Project risks
• Project constraints
• Equipment needs
• Project team and responsibilities
• Project budget

Project background and goal

There is nothing special for a ML learning project to be taken into account in this section.

Project scope

I have summarized all here: Considerations for the project scope of a Machine Learning project.

Project Approach

I have summarized all here: Project approach for a Machine Learning project.

Project deliverables

The main deliverables we should have are:

• The ML models with a success rate equal or over the requirements.
• The ML models implemented in production and being used by applications that consume the results provided by the ML model.
• Documentation about the discoveries done during the research of the ML model. This includes all rejected options and the reasons for rejection.
• Documentation about the selected ML models describing why this model is suitable, the assumptions, the parameters used, the data and data features (quality, quantity, distribution….).
• Monitoring reports and data ranges where the model works with the expected accuracy.
• Hand-over provided to operations, service team.

Project estimation

As in any type of new adventure, the first ML project for an organization is in part discover the efforts related to all the components you have to manage during an initiative like this.

Someone can give you a ball park estimate, but in reality you, as organization, has to invest and learn from that.

If you pretend to perform an estimation of effort in detail by components of the lifecycle without previous experience on the context where you are going to execute, then I’m 99% sure you will fail.

For the coming projects, we can start defining ranges of efforts that enable us to forecast the required resources for a ML project.

Project schedule

I have summarized all here: Project schedule for a Machine Learning project.

Project assumptions

To be completed.

Project risks

This is the list of ML related risks you can face:

• Lack of ML knowledge by business users or the organization.
• Use of proprietary infrastructure that delays the implementation of ML solution.
• Late integration of the ML solution within the applications team.

Project constraints

This in reality is linked to the context of your project, so it’s difficult to me to provide constraints related to ML. Many of the constraints come from the list of risks. Pay attention to them.

Equipment needs

To be completed.

Project team and responsibilities

The project team composition depends on the size of the project and the organization structure. But we can summarize this basic roles:

• Data scientific: Build ML models.
• ML engineer: produce ML models.
• Technical program/project manager: Ensure ML project meets business goals.
• Data engineer: build data pipes and work on data integration.
• Software developer: works on the integration of the ML solution into the software that will use it.
• Infrastructure engineer: provide the environments (typically deploy services).

In portfolio or value stream organizations it is common to see how a project manager drives several projects, and this is the reason sometimes they are called “program managers”.

Related to responsibilities, I use to work with a clear RACI chart for understanding who is responsible, accountable, consulted and informed in the projects.

I like to add client responsibilities too, sometimes in a separate table, but with details when discussing about the dynamics and communications expected between teams.

	Data scientific	ML engineer	TPM	Business Owner	Sponsor
Define requirements	C	I	I	R
Build ML models	R	C
Produce ML Models	C	R
Manage project			R	I	C
Validate Model	C	C	I	R	I

Domain knowledge is important on these projects. They will provide better data insight, better understanding of the data, the dynamics.

This will drive your project to have better data debugging, better definition of features, save time on different stages of the project, and probably better business value you create through your ML solution.

Project budget

Without context, we can say that the main resources required for a machine learning project are:

1. Infrastructure resources (platform services or data center space to be allocated).
2. Human resources (people directly engaged on the project).
3. Data (data required for the project and that need to be acquired).

In terms of guardrails or principles to build the budget, it basically depends on the organization, and be able to clarify these costs and how they fit with the expected return of investment, is the big question when working on a business case.

Data security and data privacy requirements can increase the complexity of the ML project and the needs of resources to attend these requirements. Be careful with this.

Other ideas to complete

To be defined.