This post is very boring, there are just questions and no answers.

I will start with 2 questions:

1. Which problem of my organization is the right one that investing on Machine Learning solution I will capture valuable return?
2. What are the features of a problem that makes it the appropriate one for a ML solution?

The answers:

For question #1, I do not have an answer. For question #2 I learned some suggestions from book Enterprise Artificial Intelligence and Machine Learning for Managers.

1. The problem are tractable with a reasonable scope and solution times.
2. Unlock sufficient business value and can be operationalized so you can capture the value.
3. Address ethical considerations.

Other secondary questions:

1. Do we have enough data to track the problem?
2. Do we know the economic value of the problem?
3. Can we measure the performance of the business function where we pretend to apply the Machine Learning solution with respect a baseline?
4. Do we have data sets that are fair?
5. Does have our potential solution the right balance with between fairness and bias?
6. Did we take into account potential safety issues?
7. Can the solution approach being explainable?
8. Is the solution approach transparent and easily understandable?
9. What is the advantage of using a Machine Learning solution instead of another solution?
10. Can we classify different issues or business cases in terms of priority?
11. Do the different business cases have relation with others?

There are multiple types of projects on machine learning, so the phases and steps are different. I will try to reduce to some basic type of projects.

Basic project plans (main phases)

Machine learning solution based on a Product

• Technology assessment = 2 – 3 days.
• Production trial = 8 – 12 days.
• Application deployment in production = 3 – 6 months.

Machine learning solution based on a platform

• Proof of concept, and prepare business case = 2 – 4 weeks
• Executive briefing with results = 2 – 4 hours.
• Production trial = 8 – 12 days.
• Application deployment in production = 3 – 6 months.

Six Sigma projects can be implemented using both approaches.

Solution Architecture

Some examples of architectures representation (just the main picture)

Architecture example based on C3.ai company.

Another example, from Microsoft:

Example of components of the Azure Machine Learning architecture.

Another example related to AWS:

AWS predictive maintenance example

As usual, the selection of the solution brand depends on the partnerships and the knowledge your organization has related to one or other brand, platform or company.

I continue taking some basic notes of the book “Enterprise Artificial Intelligence and Machine Learning for Managers“.

Tuning a machine learning model is an iterative process. Data scientists typically run
numerous experiments to train and evaluate models, trying out different features,
different loss functions, different AI/ML models, and adjusting model parameters
and hyper-parameters.

Feature engineering

Feature engineering broadly refers to mathematical transformations of raw data in order to feed appropriate signals into AI/ML models.

In real world data are derived from a variety of source systems and typically are not reconciled or aligned in time and space. Data scientists often put significant effort into defining data transformation pipelines and building out their feature vectors.

In addition, data scientists should implement requirements for feature normalization or scaling to ensure that no one feature overpowers the algorithm.

Loss Functions

A loss function serves as the objective function that the AI/ML algorithm is seeking to optimize during training efforts. During model training, the AI/ML algorithm aims to minimize the loss function. Data scientists often consider different loss functions to
improve the model – e.g., make the model less sensitive to outliers, better handle noise, or reduce over-fitting.

A simple example of a loss function is mean squared error (MSE), which often is used to optimize regression models. MSE measures the average of squared difference between predictions and actual output values.

These two linear regression models have the same MSE, but the model on
the left is under-predicting and the model on the right is over-predicting.

It is important, to recognize the weaknesses of loss functions. Over-relying on loss functions as an indicator of prediction accuracy may lead to erroneous model set points.

Regularization

Regularization is a method to balance over-fitting and under-fitting a model during training. Both over-fitting and under-fitting are problems that ultimately cause poor predictions on new data.

• Over-fitting occurs when a machine learning model is tuned to learn the noise in the data rather than the patterns or trends in the data. A supervised model that is over-fit will typically perform well on data the model was trained on, but perform poorly on data the model has not seen before.
• Under-fitting occurs when the machine learning model does not capture variations in the data – where the variations in data are not caused by noise. Such a model is considered to have high bias, or low variance. A supervised model that is under-fit will typically perform poorly on both data the model was trained on, and
  on data the model has not seen before.

Regularization helps to balance variance and bias during model training.

Regularization is a technique to adjust how closely a model is trained to fit historical data. One way to apply regularization is by adding a parameter that penalizes the loss function when the tuned model is overfit.

Hyper-parameters

Hyper-parameters are model parameters that are specified before training a model – i.e., parameters that are different from model parameters – or weights that an AI/ML model learns during model training.

Finding the best hyper-parameters is an iterative and potentially time intensive
process called “hyper-parameter optimization.”

Examples:

• Number of hidden layers and the learning rate of deep neural network algorithms.
• Number of leaves and depth of trees in decision tree algorithms.
• Number of clusters in clustering algorithms.

To address the challenge of hyper-parameter optimization, data scientists use specific optimization algorithms designed for this task (i.e.: grid search, random search, and Bayesian optimization). These optimization approaches help narrow the search
space of all possible hyper-parameter combinations to find the best (or near best) result.

The basis

• Support vector machine (SVM) is a supervised learning method.
• It can be used for regression or classification purposes.
• SVM is useful for hyper-text categorization, classification of images, recognition of characters…
• The basic visual idea is the creation of planes (lines) to separate features.
• The position of these planes can be adjusted to maximized the margins of the separation of features.
• How we determine which plane is the best? well, it’s done using support vectors.

Support vectors

The support vectors are the dots that determine the planes. The orange and blue dots generate lines and in the middle you can find what is called the hyper-plane.

The minimum distance between the lines created by the support vectors is called the margin.

The diagram above represents the more simple support vector draw you can find, from here you can make it more complex

Before to start

What is an error?

Observation prediction error = Target – Prediction = Bias + Variance + Noise

The main sources of errors are

• Bias and Variability (variance).
• Underfitting or overfitting.
• Underclustering or overclustering.
• Improper validation (after the training). It could be that comes from the wrong validation set. It is important to divide completely the training and validation processes to minimize this error, and document assumptions in detail.

Underfitting

This phenomenon happens when we have low variance and high bias.

This happens typically when we have too few features and the final model we have is too simple.

How can I prevent underfitting?

• Increase the number of features and hence the model complexity.
• If you are using a PCA, it applies a dimension reduction, so the step should be to unapply this dimension reduction.
• Perform cross-validation.

Overfitting

This phenomenon happens when we have high variance and low bias.

This happens typically when we have too many features and the final model we have is too complex.

How can I prevent overfitting?

• Decrease the number of features and hence the complexity of the model.
• Perform a dimension reduction (PCA)
• Perform cross-validation.

Cross validation

This is one of the typical methods to reduce the appareance of errors on a machine learning solution. It consist on testing the model in many different contexts.

You have to be careful when re-testing model on the same training/test sets, the reason? this often leads you to underfitting or overfitting errors.

The cross-validation tries to mitigate these behaviors.

The typical way to enable cross validation is to divide the data set in different sections, so you use 1 for testing, and the others for validations. For instance, you can take a stock data set from 2010 to 2017, use the data from 2012 as testing dataset and use the other divisions by year for validation of your trading model.

Neural networks

They can be used to avoid that errors are backpropagated. The neural network helps you to minimize the error by adjusting the impact of the accumulation of data.

The basis

• K-means clustering is an unsupervised learning method.
• The aim is to find clusters and the CentroIDs that can potentially identify the
• What is a cluster? a set of data points grouped in the same category.
• What is a CentroID? center or average of a given cluster.
• What is “k”? the number of CentroIDs

Typical questions you will face

• The number k is not always known upfront.
• First look for the number of CentroIDs, then find the k-values (separate the 2 problems).
• Is our data “clusterable” or it is homogeneous?
• How can I determine if the dataset can be clustered?
• Can we measure its clustering tendency?

Visual example

A real situation: identification of geographical clusters

You can create a K-means algorithm, where the distance is used to know the similarities or dissimilarities. Any pointers how a properties of observations are mapped so that you can decide the groups based on the K-means or hierarchical clustering.

Since k-means tries to group based solely on euclidean distance between objects you will get back clusters of locations that are close to each other.

To find the optimal number of clusters you can try making an ‘elbow’ plot of the within group sum of square distance.

http://nbviewer.jupyter.org/github/nborwankar/LearnDataScience/blob/master/notebooks/D3.%20K-Means%20Clustering%20Analysis.ipynb

.