Maxwell gives sub-optimal code a runtime glow-up

It may not be enough to just use LLMs to write your code when you can also use them to adjust it while your program runs. DoubleAI, a Tel Aviv, Israel-based startup founded in 2023, on Monday introduced Maxwell, a code execution model (CEM) that evaluates, optimizes, and rewrites binary code during execution.

Named for Maxwell's Demon, a creature imagined by physicist and mathematician James Clerk Maxwell in 1867 as part of a thought experiment on entropy, the Maxwell CEM preprocesses algorithms and machine instructions to maximize performance on available hardware.

Maxwell is similar in concept to a just-in-time compiler, but provides more flexibility and formally verifies the equivalence of replacement code. Source code isn't touched.

REG AD

"Both Maxwell and a JIT compiler work while the program runs and both produce new machine code," said doubleAI co-founder Gal Beniamini, who holds a PhD in theoretical computer science, in an email to The Register. "The difference is that a compiler can't do high-level reasoning. It can't, for example, change the algorithm. Maxwell can and does.

REG AD

"One of the reasons why JIT compilers are so narrow is that, even though they can be data-dependent, the changes they make must be deterministic and correct 'by construction,' since JIT compilers are 'unthinking' classical algorithms. 

"Maxwell brings the expressive power of frontier AI into the 'JIT' setting. In addition, Maxwell relies on formal verification to write actual, hard proofs that show that its optimizations are sound. This expands the scope of its allowed optimisation far beyond that of any algorithm or compiler."

Beniamini pointed to sorting as an example. The compiler, he said, will make a general-purpose sorting algorithm run as fast as it can, but it won't swap it out for a different algorithm.

Compilers, said Beniamini, make decisions before the program runs and have to be narrow and conservative in their choices. 

Maxwell, he said, works with more information, isn't just a one-pass operation, and operates at a higher level.

"It sees the exact hardware the program has landed on, and the actual shapes and types of the data flowing through it (information that is only available during runtime!)," Beniamini explained. "It also sees the state of the environment, what else is running in the same program, and which actual bottlenecks express themselves, and under which conditions. This allows it to specialize in ways a compiler never can."

Since Maxwell operates continuously, it can keep improving programs as new bottlenecks arise. And it can "reason" in a way that compilers cannot.

Maxwell's approach wouldn't be possible if the model rewrote code in a way that introduced errors on the fly, so the formal proof capability is not optional.

REG AD

According to doubleAI, Maxwell has made data pipelines more than 38x faster and AI inference kernels more than 4.5x faster.

LTX-2.5, an open world model that generates 4K video, managed to do so almost twice as fast with the help of Maxwell. Maxwell's optimizations reduced the data being exchanged between GPUs and optimized the kernel selection to drop unnecessary token padding.

Maxwell's optimization capability varies depending on the source code and the hardware.

"Maxwell does best on code that is algorithmic, and which can be tailored to hardware in a non-trivial manner," said Beniamini. "That being said, even code that isn't like that, but is written in a high level language like Python, can usually enjoy far reaching optimisations due to the available overhang. The same is true for PyTorch graphs, which (even under Torch's compiler) tend to be extremely far from optimal."

It can also help with code security. Beniamini said Maxwell can eliminate issues like memory errors by proving their non-existence as part of the formalization process.

"Anecdotally, we have found such issues in Python itself, through the use of Maxwell – i.e., cases in which Python was not memory safe, and where Maxwell proved that to be the case," he said.

And better still, Maxwell's tracing and profiling don't incur much if any overhead on either the CPU or GPU.

"The maximal overhead of Maxwell's tracing and profiling is 2 percent (when it isn't tracing, that falls to zero, of course), and the optimisations that Maxwell yields far outweigh that," said Beniamini. "We also apply Maxwell to the runtime itself, so that we get back a much larger than 2 percent 'flat' speedup, across the board. So in short: it's never slower."

REG AD

The installation process is similarly light-touch: Devs install the runtime and run their code, with the Maxwell model hosted by doubleAI. This arrangement, however, does require some amount of trust.

Amnon Shashua, co-founder and CEO of doubleAI, founder and departing CEO of Mobileye and a professor of computer science at the Hebrew University of Jerusalem, said in a statement that AI has changed how software is written and now is changing how it runs.

"CEMs, and Maxwell in particular, are agents on behalf of the hardware as opposed to code agents that are on behalf of the human user," he said. "In other words, they are the first AI that works for the machine, not the programmer." ®