The question I heard most at LDX3 New York was a simple one: how do you know if AI is actually helping your engineering team? Several leaders told me they could see code getting written faster. They were less sure what that meant for how quickly they could ship, how much work went into reviewing it, or what happened once it was in production.
I went to LDX3 in September for the first New York edition of LeadDev’s festival for engineering leadership. It brought together LeadDev, StaffPlus and LeadingEng at the Javits Center on September 15 and 16. It was part of a trip that also took me to LA and gave me a few more days in New York to meet clients and people from our network.

Why we went
LDX3 is a technical event, made by and for people who lead engineering teams, and that’s why we wanted to be there. AI is changing how software gets built, and we wanted to swap notes with people dealing with that every day. What was working for them? What were they still trying to figure out?
Some of what we heard was new to us. Some confirmed things we’d been seeing with our own clients and wondering whether it was just us.
Two days at the Javits Center
Things started the night before at the LeadDev community meetup, a nice way to meet a few people before it all got busy. The next day was a mix of talks, roundtables and short coffees with engineering managers, VPs, CTOs and founders. It ended on a boat on the Hudson, which is a pretty good way to close a day of talking about engineering leadership. As usual, some of the best conversations happened there, when people stopped presenting their strategy and started telling you what was really going on.

Six things I took away from the trip
1. Measuring the impact of AI is still an open question
Several leaders told me they were struggling to show what their teams were getting out of AI. Tickets closed, PRs reviewed and lines of code never said that much, and with AI they say even less. Some wanted a way to measure the impact without making their teams feel watched. Others wanted something they could actually show to a CFO.
2. Writing code got faster. Getting it ready for production still takes work
Architecture, security, testing and keeping the product running still take work. More than one person put it as a simple question: who wants to be the one waking up at 3 AM when it breaks?
That came up again when we talked about prototypes. A small team, or even a founder working alone, can get something working in days. Once more people start using it, the problems can be harder to diagnose and fix. Several people told me they expect to spend more time making quickly built products secure and stable enough to scale.
3. The time saved gets used up quickly
Some leaders said AI freed up a bit of headspace on their teams. They also said expectations rose quickly once people saw what the tools could do, and the time they saved was often filled with more work. One leader shared a question with me that I really liked: does more capacity always have to mean more work?
4. Teams are paying closer attention to what AI tools cost
Some teams told me they’d already switched tools because of price. Others were using less expensive models for routine tasks and saving more capable ones for work that needed them. That’s part of figuring out the return, too. Once you know what a tool costs, you still need to know what it’s helping your team do.
5. Senior judgment is hard to hire for
One engineering leader told me he’d interviewed around 50 senior engineers in three months and found only two with the mix he was looking for. He wanted people who were comfortable working with AI and could also break down an ambiguous problem, make sound design decisions and write a spec the team could build from. His story stuck with me.
6. Knowledge walks out the door
This one came from a lunch in LA, before LDX3 even started, and it followed me all the way to New York. When a team finally figures out how to work well with AI, how much of that stays with the team when people move on? A lot of what people learn is hard to read back from the code or the tooling: the dead ends, the small decisions, knowing when an output needs a second look.
What we’re taking home
The teams I learned the most from on this trip were changing how they worked around AI. They were asking what to measure, where engineers still needed to exercise judgment and who would own what made it into production.
It’s given us one more question to bring into conversations with clients. Where is AI helping your team ship sooner, and where is it adding work for the people who review, test and run what gets shipped?
I keep coming back to that lunch in LA. Once a team finds a way to work well with AI, how do they pass on what they’ve learned?
Thanks to the LeadDev team for putting together the first New York edition, and to everyone who shared a coffee and their time with me. Hope to see you at the next one!





























