Transitioning to using AI tools

Despite working in the applied AI field for the last couple of years, I’ll admit that for a while I was resisting using it in my daily life, or my coding. This post is about transitioning to comfortably using AI, while leaving room for growth.

At work, I found early on that LLMs could be extremely helpful for analyzing data in minutes, which would otherwise take a human hours or even days. But I have concerns about privacy, I have a lot of friends who are vehemently against AI (for various reasons I won’t go into here), and I didn’t have great ideas about use cases in my personal life where I thought AI could really help (and wouldn’t steer me in the wrong direction). For some things, it still felt better to ask humans for advice.

This post is about things I wish all eval tools provided (but many don’t).

Over the last couple of years, I’ve used several different eval tools, from home-grown and pre-release products, to open source and startup-built tools with wide adoption. Most of them had inconsistent features at best, and glaring omissions for what I thought were obvious features, at worst.

Now, to be fair, what I’m asking for from an eval tool is kind of a lot. I want something that can

Recently, I’ve been surprised to note how many companies aren’t just returning to offices (ugh, open offices are the worst!), but are still actively unwilling to do remote work well.

Here are some hints for things that I’ve learned can be helpful, after working remotely for a few years at a variety of companies - some that were remote-first or remote-only, and some that didn’t have mature practices around it: