Close Menu
NCIJ Network NCIJ Network
    What's Hot

    Global climate talks have collapsed. Here’s what just happened. – POLITICO

    November 22, 2026

    Pip review will call for overhaul rather than scrapping benefit for under-25s | Benefits

    November 22, 2026

    The Guardian view on Serbia’s new political era: a generational opportunity must not be wasted | Editorial

    November 22, 2026
    Facebook X (Twitter) Instagram
    Trending
    • Global climate talks have collapsed. Here’s what just happened. – POLITICO
    • Pip review will call for overhaul rather than scrapping benefit for under-25s | Benefits
    • The Guardian view on Serbia’s new political era: a generational opportunity must not be wasted | Editorial
    • Global left rallies in Paris to make ‘not so radical’ case for taxing billionaires – POLITICO
    • UK taxpayer funds held in offshore tax havens top £1tn, latest HMRC data shows | Tax avoidance
    • Theresa May’s blast from the past calls Tories back to their unhappy place | Theresa May
    • ‘Just wrong’: British right distances itself from Trump after Putin ‘friend’ post | Politics
    • I lost my dad, mum and sister to the same aggressive brain cancer
    • About
      • Our Team
      • Editorial Policy
      • Editorial Independence
      • International Support
    • Trust & Standards
      • AI Usage Policy
      • Conflict of Interest Policy
      • Corrections Policy
      • Ethics Policy
      • Fact-Checking Policy
      • Source Protection
    • Get Involved
      • Guide for Sources
      • Support Independent Journalism
    • Legal
      • Cookie Policy
      • Privacy Policy
      • Terms of Use
    Facebook X (Twitter) Instagram
    NCIJ Network NCIJ Network
    Sunday, November 22
    • Home
    • World
    • Ai
    • Business
    • Politics
    • Health
    • Crypto
    • Science
    • Technology
    • Cybersecurity
    • Defense & Security
    • Economy
    • Energy
    • Europe
    • More
      • Fact Check
      • Investigations
      • Opinion & Analysis
      • Environment
    NCIJ Network NCIJ Network
    Home»Artificial Intelligence

    Sakana AI Researchers Introduce PC-ALM, a Layer-Local Alternative to Backpropagation That Trains 1000-Layer Networks

    NCIJ NETWNCIJ NETWORKBy NCIJ NETWNCIJ NETWORKSeptember 14, 2026 Artificial Intelligence No Comments5 Mins Read
    Share
    Facebook Twitter LinkedIn Pinterest Email

    Backpropagation is a global algorithm: a forward pass, then a backward pass, then a weight update, each locked behind the previous one. Brains have no known mechanism for that kind of network-wide phase locking, which is why local-learning alternatives such as predictive coding (PC) keep drawing research interest. Sakana AI researchers propose Augmented Lagrangian Predictive Coding (PC-ALM), a variant of PC that keeps every update layer-local yet recovers backprop-aligned credit signals. The research team reports training residual MLPs up to 1000 layers within about 2 percentage points of backprop on MNIST.

    Is it deployable? Yes, as research code: an MIT-licensed JAX reference implementation runs on CPU and reproduces the paper’s width-depth grid. It is a training method, not a model, and has only been tested on small image benchmarks.

    Why standard PC stalls in deep, narrow networks

    PC treats every hidden activation as an optimization variable and penalizes the squared mismatch between each layer’s activation and the prediction arriving from the layer below. Inference is gradient descent on that energy; learning is a Hebbian-like weight step. The catch is that supervision enters at the output and must diffuse through a chain of local compromises. In deep, narrow networks the credit signal fades long before it reaches the input. Innocenti et al. characterized this PC-BP gap as a function of width and depth, and it is worst when width is smaller than depth.

    What PC-ALM changes

    PC-ALM starts from the constrained view of training: minimize the supervised loss subject to hi=σ(Wihi−1)h_i = sigma(W_i h_{i-1}) at every layer. PC is the quadratic-penalty relaxation of that problem. PC-ALM uses the augmented Lagrangian instead, attaching a Lagrange multiplier λi∈ℝdisuch thatdim(λi)=dim(hi)lambda_i in mathbb{R}^{d_i} quad text{such that} quad text{dim}(lambda_i) = text{dim}(h_i) to each layer constraint while keeping PC’s penalty. Setting λ = 0 recovers PC exactly.

    Inference alternates 2 local steps: a primal gradient step on the activations, and a dual step λi←λi+αrilambda_i leftarrow lambda_i + alpha r_i that accumulates the layer’s prediction error. Completing the square shows each primal step is a standard PC step with the prediction target shifted by −λi/ρ-lambda_i/rho. After T steps the weight update acts on the composite signal λi+ρrilambda_i + rho r_i. The research team read this as a PI controller per layer: the prediction error is the proportional term and the multiplier is the integral term. α = 0 gives PC; α = ρ with the inner problem solved exactly gives the classical method of multipliers.

    Exact backprop gradients in the linear case

    LeCun observed in 1988 that the Lagrange multipliers of a constrained network equal the backprop adjoints at a KKT point. The team proves that in linear PC networks, under a spectral-radius stability condition, PC-ALM converges to that KKT point: activations return to their forward-pass values while each λilambda_i integrates to the exact BP adjoint. The per-mode stability bound is ηhσi2(2ρ+α)<4eta_h sigma_i^2 (2rho + alpha) < 4, which reduces to PC’s condition at α = 0. Unlike PC’s monotone gradient flow, PC-ALM’s iteration matrix has complex eigenvalues that produce damped oscillations; α sets their frequency but not their decay rate.

    Results

    The research team sweeps residual MLPs with width and depth from 8 to 128 on Fashion-MNIST and MNIST under the mean-field parameterization of Innocenti et al., training for 1 epoch. With an inference budget of T = 2L, PC-ALM matches backprop across every width, depth, and activation (identity, tanh, ReLU), while PC drops sharply in deep, narrow cells. The repo’s reference cell (width 32, depth 32, ReLU, Fashion-MNIST) reports 78.66% test accuracy for BP, 68.13% for PC, and 77.75% for PC-ALM, with gradient cosine to BP rising from 0.604 to 0.909.

    The research extends the picture: 1000-layer residual MLPs on MNIST (width 32, ReLU, 5 epochs) stay within roughly 2 points of BP, and PC-ALM improves over PC on every benchmark tried, including ResNet-18 on CIFAR-10 and Tiny ImageNet.

    1000Layer Alternative Backpropagation Introduce LayerLocal networks PCALM researchers Sakana Trains
    NCIJ NETWNCIJ NETWORK
    • Website

    Keep Reading

    Google AI Previews Tool Retrieval in the Gemini API for Large Tool Libraries

    Best Personal AI Agents in 2026: OpenClaw, Hermes Agent, Gemini Spark and More, Compared on Memory, Security and Cost

    A Team of Researchers from Columbia and CertiK Introduce TensorLean for Formal Verification of GPU Kernels

    OpenAI Ships GPT-Live-1.1 for Voice Agents, Cuts Idle Cost to $0.02/Min

    Meet Thinking Machines Inkling 2 Mini: 26B Looped MoE Runs on a 32GB Mac

    Baseten Releases Bison-1 and Groundhog-1, Topping Decision Model Index

    Add A Comment
    Leave A Reply Cancel Reply

    Editors Picks

    Global climate talks have collapsed. Here’s what just happened. – POLITICO

    November 22, 2026

    Pip review will call for overhaul rather than scrapping benefit for under-25s | Benefits

    November 22, 2026

    The Guardian view on Serbia’s new political era: a generational opportunity must not be wasted | Editorial

    November 22, 2026

    Global left rallies in Paris to make ‘not so radical’ case for taxing billionaires – POLITICO

    November 22, 2026
    Latest Posts

    Review: Jan Carson’s ‘Few and Far Between’ Explores the Legacy of the Troubles in Northern Ireland

    August 22, 2026

    Crypto Biz: Treasury ‘Not-QE’ Fuels Bitcoin Rally

    August 22, 2026

    Android Car Malware Spreads Through Built-In Updaters for Ad Fraud, Proxy Botnet

    August 22, 2026

    Subscribe to News

    Get the latest sports news from NewsSite about world, sports and politics.

    NCIJ Network is an independent digital news platform delivering trusted investigative journalism, European and global news, in-depth analysis, and fact-based reporting with accuracy, transparency, and integrity.

    Facebook X (Twitter) Instagram Pinterest YouTube

    Global climate talks have collapsed. Here’s what just happened. – POLITICO

    November 22, 2026

    Pip review will call for overhaul rather than scrapping benefit for under-25s | Benefits

    November 22, 2026

    The Guardian view on Serbia’s new political era: a generational opportunity must not be wasted | Editorial

    November 22, 2026

    Subscribe to Updates

    Get the latest creative news from FooBar about art, design and business.

    Type above and press Enter to search. Press Esc to cancel.