Classical ml Memes

Remember when we used to carefully engineer features and tune hyperparameters for our models? Yeah, those were simpler times. Now the entire AI/ML industry has basically decided that the solution to every problem is "just throw a transformer at it and add more GPUs." Need to classify images? Transformer. Text generation? Transformer. Predict stock prices? Believe it or not, also transformer. Meanwhile, good old classical ML techniques like random forests, SVMs, and logistic regression are sitting at the bottom of the pool, completely forgotten. Sure, they're interpretable, efficient, and actually work great for tons of problems, but who cares when you can burn through your entire cloud budget training a 175 billion parameter model instead? Deep learning and neural networks are also drowning down there, which is ironic since they were the hot new thing just a few years ago. But nope, not fancy enough anymore. If your solution doesn't involve attention mechanisms and can't be described with the phrase "large language model," are you even doing AI?