DeepCoder delivers top coding performance in efficient 14B open model

Researchers at Together AI and Agentica have launched DeepCoder-14B, a brand new coding mannequin that delivers spectacular efficiency corresponding to main proprietary fashions like OpenAI’s o3-mini. 

Constructed on high of DeepSeek-R1, this mannequin offers extra flexibility to combine high-performance code technology and reasoning capabilities into real-world functions. Importantly, the groups have totally open-sourced the mannequin, its coaching knowledge, code, logs and system optimizations, which may help researchers enhance their work and speed up progress.

Aggressive coding capabilities in a smaller package deal

The analysis group’s experiments present that DeepCoder-14B performs strongly throughout a number of difficult coding benchmarks, together with LiveCodeBench (LCB), Codeforces and HumanEval+.

“Our mannequin demonstrates robust efficiency throughout all coding benchmarks… corresponding to the efficiency of o3-mini (low) and o1,” the researchers write in a blog post that describes the mannequin.

Curiously, regardless of being skilled totally on coding duties, the mannequin reveals improved mathematical reasoning, scoring 73.8% on the AIME 2024 benchmark, a 4.1% enchancment over its base mannequin (DeepSeek-R1-Distill-Qwen-14B). This means that the reasoning expertise developed via RL on code will be generalized successfully to different domains.

Probably the most hanging side is attaining this degree of efficiency with solely 14 billion parameters. This makes DeepCoder considerably smaller and doubtlessly extra environment friendly to run than many frontier fashions.

Improvements driving DeepCoder’s efficiency

Whereas growing the mannequin, the researchers solved a number of the key challenges in coaching coding fashions utilizing reinforcement studying (RL).

The primary problem was curating the coaching knowledge. Reinforcement studying requires dependable reward indicators indicating the mannequin’s output is appropriate. Because the researchers level out, “In contrast to math—the place plentiful high-quality, verifiable knowledge is available on the Web—the coding area suffers from a relative shortage of such knowledge.” 

To handle this drawback, the DeepCoder group carried out a strict pipeline that gathers examples from completely different datasets and filters them for validity, complexity and duplication. This course of yielded 24,000 high-quality issues, offering a stable basis for efficient RL coaching.

The group additionally designed an easy reward operate that solely offers a constructive sign if the generated code passes all sampled unit checks for the issue inside a selected time restrict. Mixed with the high-quality coaching examples, this outcome-focused reward system prevents the mannequin from studying methods like printing memorized solutions for public checks or optimizing for easy edge circumstances with out fixing the core drawback.

The mannequin’s core coaching algorithm relies on Group Relative Coverage Optimization (GRPO), a reinforcement studying algorithm that proved very profitable in DeepSeek-R1. Nonetheless, the group made a number of modifications to the algorithm to make it extra secure and permit the mannequin to proceed bettering because the coaching extends for an extended time.

Lastly, the group prolonged the mannequin’s context window iteratively, first coaching it on shorter reasoning sequences and progressively growing the size. Additionally they developed a filtering methodology to keep away from penalizing the mannequin when it created reasoning chains that exceeded the context limits when fixing a tough immediate. 

The researchers clarify the core concept: “To protect long-context reasoning whereas enabling environment friendly coaching, we included overlong filtering… This system masks out truncated sequences throughout coaching in order that fashions aren’t penalized for producing considerate however prolonged outputs that exceed the present context restrict.” 

The coaching was progressively scaled from a 16K to a 32K context window, and the ensuing mannequin might additionally resolve issues that required as much as 64K tokens.

Optimizing long-context RL coaching

Coaching massive fashions with RL, particularly on duties requiring lengthy generated sequences like coding or advanced reasoning, is computationally intensive and sluggish. A serious bottleneck is the “sampling” step, the place the mannequin generates doubtlessly hundreds of tokens per instance within the batch. Variations in response size imply some responses end a lot later than others, leaving GPUs idle and slowing down the whole coaching loop. 

To speed up this, the group developed verl-pipeline, an optimized extension of the open-source verl library for reinforcement learning from human feedback (RLHF). The important thing innovation, which they name “One-Off Pipelining,” rearranges the response sampling and mannequin updates to scale back the bottlenecks and accelerator idle time.

Their experiments confirmed that one-off pipelining offered as much as a 2x speedup for coding RL duties in comparison with baseline implementations. This optimization was essential for coaching DeepCoder inside an affordable timeframe (2.5 weeks on 32 H100s) and is now open-sourced as a part of verl-pipeline for the neighborhood to make use of and construct upon. 

Enterprise impression

The researchers have made all of the artifacts for coaching and working DeepCoder-14B accessible on GitHub and Hugging Face beneath a permissive license.

“By totally sharing our dataset, code, and coaching recipe, we empower the neighborhood to breed our work and make RL coaching accessible to all,” the researchers write.

DeepCoder-14B powerfully illustrates a broader, accelerating development within the AI panorama: the rise of extremely succesful but environment friendly and overtly accessible fashions. 

For the enterprise world, this shift signifies extra choices and better accessibility of superior fashions. Chopping-edge efficiency is not solely the area of hyperscalers or these prepared to pay premium API charges. Fashions like DeepCoder can empower organizations of all sizes to leverage subtle code technology and reasoning, customise options to their particular wants, and securely deploy them inside their environments. 

This development can decrease the barrier to entry for AI adoption and foster a extra aggressive and modern ecosystem, the place progress is pushed via open supply collaboration.