Posts

Burning Tokens

Image
Burning tokens is a double-edged sword. Powerful weapons can either give a lot of leverage or drag you down. It's easy to think that you are not shooting yourself in the foot with unlimited tokens. The thing is, you won't be shooting yourself in the foot; you will be expending a lot of company money and perhaps with lots of amplified waste. But only if you do it wrong or don't know what you are doing, and that's the very tricky part. Big Tech and big companies already have unsustainable token usage, like Uber . The list goes on and on: Microsoft , Walmart , Meta , JPMorgan , AT&T , and many others. For some, tokenmaxxing is over. Is it? Really depends on the cost of inference. Jev is showing that a classifier can be powerful and reliable, at least in the sense of low cost and speed, not saying anything about accuracy, which is not the score. The problem is not using tokens; that is the symptom. The problem is a lack of training; there is a huge AI enablement GAP ...

AI Slop

Image
AI Slop is everywhere. Technical debt always existed, but sure, slop will surpass it; somehow we managed to make things worse: Human-generated, handcrafted tech debt blended with synthetic AI generations. I won't say productivity is up because that's hard to measure; for sure, output is up, but so are bugs and incidents. The less you look at the code, the less you pay attention, the more slop you get. Attention is all we have; it's precious. The catch is, to produce more, you need to look less. It takes 5 minutes for the agent to produce the code and 5 hours for you to validate and understand. Multiply that by several PRs a day, and we have a big attention problem and slop proliferation. AI Slop Definition I think it would be fair to ask frontier LLM models what they define as ai slop. I had to ask to be max 2 lines; otherwise, it would be slop 🙂 GPT has the most interesting definition for me, specially because it mention judgement and quality. IMHO what we are also missin...

Lessons Learned from 500+ AI Pocs

Image
In the summer of 2024, I wrote 300+ AI POCs; in 2 years, I have 550+ AI pocs. Before I cover the lessons learned, we need to address somethings. I've been learning technology for the past 23 years since I started working professionally with software; I never stopped. I always learned multiple languages every year and always liked to explore , even before AI. For 19 years(since 2007), I have blogged about technology. Before AI, I always believed that in the software profession, you need to know what you are doing in order to do it right. I learned languages that I would probably never use in production, just for the fun and sake of learning. I always believed that when you're too narrow in your current needs, you would miss the big picture and opportunities. YES, I always learn and research problems I was solving, but never confined myself to only short-term features. So I was doing POCs before AI, and I will do them after AI. Now we need to understand why POC are getting a bad ...

AI Moves Fast. Decisions Move Slow.

Image
More tokens are not more intelligence. Still, it is easy to get hooked on token usage. Is it learning, experimentation, or just FOMO? Subscription models trained us to extract as much as possible from a fixed monthly payment. But when more usage creates more code to review, why do we need to max everything out? Are we getting more from AI? Or are we repeating an old consumption habit with a new resource? I was reading and watching Gergely Orosz’s conversation with Dex Horthy, “Context engineering with Dex Horthy” , and it made me think about fast loops, slow loops, token harder, and token smarter. Faster loops do not necessarily accelerate value delivery. Unless the lead time decreases and the value increases, we may be trapped in local optimization.

Handling Ambiguity in the Age of Agents

Image
Ambiguity is not simply having a difficult problem. It is not knowing exactly what the problem is, what matters most, or which direction will create value. It appears during requirements, scoping, analysis, and design—across the whole solution space. It also appears across teams: product, design, engineering, and the business. Ambiguity has always been one of the hardest parts of engineering. Agentic AI makes it worse at scale. A human engineer receiving an ambiguous request may push back, ask questions, make assumptions, or stall. An agent immediately produces something: code, commits, documentation, and a confident summary. The output looks like progress. It may have little to do with the outcome anyone wanted.  We always had this problem, but now agents are a huge employer.

Loop Engineering

Image
Software engineering keeps changing names for the same old problems. Before 2023, changes were introduced with some level os stability; now, changes happen much faster, to the point that we do not really understand the problems. We had those problems in the past, considering things like (agile/scrum, microservices,  devops), but never has it been a fast loop like this. First, we had scripting. Then automation. Then CI/CD. Then DevOps. Then Platform Engineering. Then AI Agents. Now we have Loop Engineering. However, companies have barely functional CI/CD due to a lack of incentives, poor vision, poor management, and other dysfunctions.  Loop engineering, the name is new. The problem is not. The problem is: how do you make a machine do useful engineering work without babysitting every step? That’s it. But "solution" and "waste" manifest in the same way, using the same tools. Loop Engineering is not just about prompting with a fancy jacket. It is not “write a bette...

Harness Engineering

Image
Harness Engineering is pretty trendy at the moment. Harness engineering is a way to better drive or operationalize an LLM model. The idea is that you are renting an LLM model as a service, but you own the harness. The harness is a way to be less dependent on the model (LLM). LLMs are not deterministic at all, and they are not general intelligence; they are pretty limited to their training data. Inference cost is very expensive, and the era of subsidizing is over .  AI was sold as a promise to solve engineering and elevate us to a new level of abstraction, and so far thats far from being a reality. I keep hearing people say "wait two years" every year. More and more engineers spend more time trying to drive LLMs to produce the right code. When that is well executed, we can see 10-30 % productivity gains across our industry. Such good execution results from proper testing, robust CI/CD, great automation, amazing observability, and attention to technical excellence. When that is...