Step #3 ✅: Creating & training my own machine learning models

Going into this deep-dive, I was expecting the implementation level to be one of the most interesting and important parts.

But as I kept going through papers and re-implementations, my perspective changed.

I focused first on creating my own linear models, tree-based models, and classical ML pipelines, and then also wanted to build clustering and probabilistic models because of how important they are conceptually.

I created my own (and trained on real datasets):

Linear Regression
Logistic Regression
Decision Tree
Random Forest
Support Vector Machine
K-Nearest Neighbors
Naive Bayes
K-Means
PCA
Gradient Boosting

I was expecting the software layer of building these models to be equally complex.

But through the implementation process, it became clearer that writing minimal machine learning implementations removes much of the interesting part. In hindsight, this should have been obvious.

The real complexity in machine learning usually comes from data quality, feature engineering, evaluation, tuning, and choosing the right assumptions for the problem, not just from writing the model code itself.

This is also magnified by the fact that, in contrast to my earlier hardware work, there are so many existing resources and reference implementations for classical ML models.

So instead of focusing my learning on implementation, where there are already tons of examples online, I wanted to create something new.