“Some organizations have already discovered that the cost of tokens for coding assistance is much higher than the cost of human software developers,” Herschel said.
As more agents work together, “your financial risk only grows. It’s like giving your teenager your credit card… I’m sure you will learn a lot, but mostly from the bill,” said Robert Thanaraj, a senior director analyst at Gartner and a co-speaker at the Mumbai keynote.
Companies should track costs in prototyping, such as finding the cost of an individual agent per completed task, Thanaraj said. “It’ll help you to evaluate different large language models or help you to go with a more affordable option, such as smaller language models or open weights model.”
