The Roads Get Paved

TLDR: The returns on investment for “constraining” jobs, ML R&D in particular, is much greater now that companies have the capability to automate some core parts of it. Thus, they will “pave the roads”¹ to make it easier for the systems to succeed.

We often try to estimate the year by when AI can automate drop-in remote workers in full, but I claim that getting to “purely text-based software engineering” is actually still very important, even if the model lacks the capability to do things like participate in the team meetings that human SWEs are required to go to. Not because SWE is currently done in this purely text-based way, but because that will be the most efficient way to do it in the near future. Given the “jagged” capabilties profiles of models, people trying have their AIs be drop-in remote workers face are bottlenecked by tasks involving seeing, clicking, etc.. Smart companies will naturally instead just continually work on “constraining” (systematizing, scoping, structuring) the entirety of research engineering until it fits nicely within a box in which the LLMs can automate. The environment will adapt to the systems’s jagged capabilities, especially if you think that the systems capabilities will remain jagged.

As noted by Tyler Cowen, the Epoch folks, and ~everyone else who is paying attention at this point, we have systems which are extremely competent in some domains, and less competent in others. Whether they’ll catch up in these other domains is hotly debated, but I claim that factor is not load-bearing for some questions around just how transformative they will be.

A point in favor of this line of reasoning is that lots of great smart people will be quick to notice that even if AI progress stopped yesterday we’d get years of gains in productivity as a result, as the technology diffuses. This must be in spite of the jagged capabilities profiles. Partially this is just the slower-to-adapt firms automating the tasks that humans were doing long after they could be automated, but partially this is also because there are so many improvements even the bleeding-edge firms can make to the way that they make use of the AI systems currently available.

I don’t actually know how the horses vs. carriages debate played out at the time (or if there was one), but a reasonable argument would indeed have been “horses are so much more versatile, able to handle edge cases and many more terrains than carriages, which need a well-trodden path”. And indeed all-terrain capabilities took much much longer to develop than the simpler form of carriages. But instead, once we recognized the power of the carriage, we just changed the environment a ton to make use of the remarkable capability of wheels.

Footnotes

1. Title analogy h/t a coworker ↩