My Journey Through Machine Learning: From Early Projects to Current Innovations

Over the years, I have had the privilege of working on a variety of machine learning (ML) projects involving different approaches and algorithms. This journey has been both challenging and rewarding, pushing the boundaries of what can be achieved with AI and ML. Here, I will share some of the key projects and the insights gained along the way.

Early Projects: NLP and Beyond

Starting in the early days of my career, from 2011 to 2014, I worked heavily in the Natural Language Processing (NLP) space. Specifically, I trained classifiers and topic models for document matching. These projects primarily used off-the-shelf tools such as gensim, Scikit-learn’s SVM, and nltk’s Maximum Entropy classifier. This was a foundational period, where the primary focus was on building robust systems for handling large volumes of text data.

News Recommendation and Sentiment Analysis

When I started my own company in 2014, I began exploring more advanced techniques like Word2Vec and Recurrent Neural Networks (RNNs) and Long Short-Term Memory networks (LSTMs). These models were used to train document embeddings and build classifiers. An example of this work is a news recommendation engine, which aimed to provide personalized news articles to users based on their reading habits and interests. Although the business did not scale as expected due to the competitive landscape, the legacy API of the engine is still operational today, using fine-tuned BERT and other language model modifications.

Dynamic Filters and Social Media Popularity

Another exciting project involved predicting social media image popularity. Our app, Cornea AI, used dynamic neural art algorithms to transform any image into a variety of artistic styles. We trained an additional algorithm to predict scores for images using open image datasets with user scores available online. While the app gained significant media attention, it faced challenges in sustained growth and eventually had to be discontinued due to high operational costs.

Healthcare Applications: Head Injury Triage and Dental Cavity Detection

One of the more impactful projects in my career was the development of a head injury triage system. This system was designed to alert radiologists about potential emergencies, particularly in hospitals where CT scans were needed. By analyzing images, the system helped prioritize cases where a hemorrhage was detected. We deployed this system in two hospitals, although it fell short of achieving widespread adoption due to a lack of funding.

Simultaneously, we explored solutions for other healthcare challenges in India, such as tuberculosis (TB) detection on sputum slides and chest X-ray analysis. Another project involved building a technology stack for detecting dental cavities, which included FDA trials in the United States. However, the requirement for long trials and changing regulatory landscape posed significant hurdles, ultimately leading to discontinuation.

Current Focus: Retail Computer Vision

Today, I am leading a retail computer vision platform that leverages dense object detection and classification for recognizing packaged goods on retail shelves. This platform helps retail businesses achieve marketing compliance and manage out-of-stock situations efficiently. By analyzing images taken of retail shelves using deep learning techniques, the system provides valuable insights into product placement and stock levels.

Key Takeaways and Lessons Learned

Throughout these projects, several key lessons stand out. Firstly, the importance of choosing the right off-the-shelf models as a starting point cannot be overstated. Building upon these models and improving them through fine-tuning and additional features has been crucial for achieving better accuracy. Secondly, data collection and annotation are fundamental steps that significantly influence the performance of ML models. Lastly, adapting to changes in the regulatory and market landscape is essential to ensure the sustainability of projects, especially in fields like healthcare and FDA-regulated technologies.

Future Direction

Looking ahead, I am excited about the potential of ML in various domains, including retail, healthcare, and beyond. The future holds many promising opportunities, and I am eager to continue exploring innovative solutions to real-world problems using advanced machine learning techniques.