Some observations on vibe coding

I get a lot of questions about working with LLMs. Something I hear people discussing a lot is whether AI-generated code is better, or worse, or just more evil than human-generated code. I don’t think it’s an either/or question. To me, it’s just another tool: like most tools, there are ways to use them for good, or as weapons.

I think vibe coding is great if you need to write a one-off script that isn’t intended to be part of production code. For example, I asked Claude to help me come up with ways to generate fairly large test data sets for my last project, and it decided to write me a script to do it. This worked beautifully for what I needed, and saved me a ton of time. I did have to iterate on the prompt a bit to get it to do what I wanted, and that was a little nerve-wracking, so I saved off the different versions of the code (because sure enough, sometimes it got worse in ways I didn’t expect).

I’ve been doing a lot of work around grief lately.

Both of my parents died in 2023. Two other friends also died in 2023 - both former coworkers. One was in his mid-thirties and had a rare brain tumor; the other died from COVID. 

My dad’s death last June came after several months in the ICU. He endured repeated rounds of dialysis, which he hated, and ventilator-associated pneumonia, which I hadn’t heard of before. He had opted for open-heart surgery, I think because he didn’t fully understand the risks given that he was a cancer patient for over 10 years, and like many people, believed that doctors are superheroes. I was with him, and held his hand when they turned off the machines at his request. 

Life is growth. If we stop growing, technically and spiritually, we are as good as dead.

—- Morihei Ueshiba

I started studying aikido about 12 years ago, shortly after I moved to San Francisco. I had no idea how much I would enjoy it, or how much it would change my life.

I didn’t know what it was when I first started, but the translation I like the best is “Path of Life Harmony.”

This is one of those posts that I’m writing mostly because I’m still frequently hearing people say they’re struggling with the question of how to interview data people. I’ve blogged and spoken previously about the misery of being an interviewee, so suffice it to say I have a ton of empathy for how awful it is to go through a bad interview process, and how disappointing it is to go through a long, grueling interview and not get an offer.

I decided to embark on this comparison mostly out of curiosity. No tool is perfect for all use cases, that’s why we have forks, and spoons, and sometimes when we’re camping, sporks. Although Pachyderm claims to use a git-style approach for data and code versioning, there are aspects of the Pachyderm approach (like forking) that aren’t exactly like git. So one thing I wanted to know is, how well does this analogy to git work for DVC? And are there use cases where one should definitely use DVC or Pachyderm?

Coming from a background in bench science, or what we affectionately referred to as “wetlab”, I like to test everything I do, and I like my tests to be fast and representative of what I expect to find when I run things “for real”.

Most people I’ve met who are newer to data engineering find that it’s not immediately obvious how to write and run tests for data things. It’s different enough from writing unit tests for web apps that there are some pitfalls to be aware of. This post is about that.

Recently, a coworker asked me what the difference is between data science and data engineering. She seemed to think that data engineering was “just analytics”, which is a definition I’ve often heard applied to data science, as well.

After I got over my initial horror that she had been wondering this for months and hadn’t asked sooner, I started to explain:

Data engineering is getting data, cleaning data, reshaping data, validating data, and loading it into databases. Data science is all of that, plus analyzing the data and figuring out how to display it in a way that makes sense, and sometimes also building models and doing machine learning.

Question: How come we don’t have more diversity at our company?

This is one of those posts, I’m writing it because I keep hearing people asking the same questions over and over. I hear these questions at meetups and in Slack groups every week, and I hate seeing people trying to reinvent the wheel when it’s really not necessary.

Answer: You haven’t tried that hard.

For the purposes of this blog post, let’s go with the hypothesis that there are plenty of nontraditional candidates out there like me. We didn’t major in computer science, but we learned how to code and we enjoy it. We’re coming from other kinds of backgrounds. For some of us, this is a second career. We have tons of transferable skills. You should be hiring more people like us.

Recently, someone asked for help with setting up postgres in docker and connecting to it with python.

While helping this person, I realized this is something that should be fairly straightforward with a simple set of instructions, but there aren’t a lot good beginner tutorials out there. So I decided to write this up because I’m sure it’s something other people would also find useful. A few years ago I wouldn’t have been able to do this even with a lot of googling (this time I only had to google a few things!).

I was chatting with some friends recently, and this question came up:

What does a good working relationship with your manager look like?

This is the list I came up with. I’m not saying I’ve been perfect at all of these myself as a manager, but it’s what I strive for, and it’s what I look for in a good manager.

1. They trust you
   • They ask for your input when appropriate
   • They promote your work to others in the company where appropriate
2. You trust them
   • No surprises
   • You feel safe asking them for help when you need it
   • They discuss your career goals with you, and help steer you toward them

This post is just an elaboration on what I mean by each of the things on that list.